Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

A new algorithm, MD-LAMBO, claims that subspaces generated from the gradients of a cubic-regularized quadratic model give sufficient function decrease and, except for one subspace family, drive the gradient norm to zero on bounded-below twi

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 20:55 UTC pith:4K5UPIWJ

load-bearing objection Genuinely novel cone subspace construction, but the paper's half-decrease and global convergence proofs have load-bearing gaps; worth refereeing and major revision, not desk reject. the 3 major comments →

arxiv 2509.08256 v1 pith:4K5UPIWJ submitted 2025-09-10 math.OC cs.NAmath.NA

Model-Driven Subspaces for Large-Scale Optimization with Local Approximation Strategy

Yitong He , Pengcheng Xie This is my paper
classification math.OC cs.NAmath.NA MSC 90C3090C5665K05
keywords large-scale optimizationsubspace methodsmodel-based optimizationtrust-region methodscubic regularizationderivative-free optimizationtruncated conjugate gradientglobal convergence
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to show that the subspace in which a large-scale optimization problem is solved can be built from the model itself—specifically, from the gradients of a quadratic-plus-cubic approximation evaluated on a few directions. It introduces sixteen such 'truncated subspaces,' proves explicit formulas and dimensions for them, and embeds them in a trust-region framework called MD-LAMBO. The central theoretical result is a global convergence theorem: for any subspace in the catalog except one pair, the gradient norms of the iterates have a subsequence going to zero, provided the projected Hessians satisfy a balance condition infinitely often. The derivative-free section offers a class of models with the same expected decrease as gradient models, and numerical profiles show the subspace choice changes practical performance.

Core claim

The paper's claim is that model-driven subspaces—sets spanned by gradients ∇p_k(s) of the model p_k(s)=∇f(x_k)^T s + ½ s^T H_k s + (σ/3)‖s‖³, evaluated over a low-dimensional 'inspiring region' W—are a viable substrate for large-scale optimization. Two functors produce sixteen subspaces: one takes the union of one-dimensional spans of these gradients, the other takes the span of the union. In the important two-dimensional case W=span{∇p_k(0)}, the truncated subspace S_l^{(1,σ)} is a parabolic wedge, and the first two truncated conjugate-gradient iterates stay inside it, so solving the reduced problem loses no more than half the maximum model decrease (Theorem 2.8). With an eigenvalue-balance

What carries the argument

The functor F from models to subspaces, S=F(p_k) = ∪_{s∈W} span{∇p_k(s)} (type 1) or span(∪_{s∈W} ∇p_k(s)) (type 2), with the cubic-regularized model p_k whose gradient includes the extra term σ‖s‖s. This machinery produces a low-dimensional 'truncated' set that still contains the relevant search directions; the two-dimensional wedge S_l^{(1,σ)} is the case where the paper proves truncated-CG iterates stay inside, giving the ½-decrease bound that feeds the convergence theorem.

Load-bearing premise

The proof depends on a balance condition for the model Hessian's eigenvalues that the paper does not show holds for any concrete problem class, and on the trust-region radius tending to zero, which the algorithm does not guarantee.

What would settle it

A concrete test: implement MD-LAMBO (Algorithm 2) with subspace S_l^{(1,σ)} on a bounded-below, twice differentiable function with Lipschitz Hessian, e.g., f(x)=sin(x₁)+∑_{i≥2} x_i², and record the trust-region radii and gradient norms. If the radii do not shrink to zero and the gradient norms do not decrease below any ε>0, the paper's Lemma 3.3 fails, taking Theorem 3.1 with it.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Large-scale unconstrained problems can be solved on 2- to 5-dimensional search spaces without sacrificing the theoretical guarantee of reaching a stationary point.
  • Derivative-free implementations can use interpolation models on these subspaces with the same expected decrease as exact-gradient models.
  • The 16-subspace catalog gives practitioners a way to trade per-iteration cost against versatility: truncated low-dimensional spaces matched the performance of classic full-dimensional ones in the reported tests.
  • The one excluded subspace family marks a boundary of the convergence theory and is the place to look first for a counterexample.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The eigenvalue-balance assumption (Assumption 1) is not scale-invariant; a practical corollary the paper does not state is that rescaling the model Hessian or preconditioning the subspace could make the condition hold more often.
  • Theorem 3.1 states a liminf result on gradient norms, which is weaker than the phrase 'global convergence' used in the abstract; for nonconvex problems the full sequence need not converge even if liminf is zero.
  • The truncated-step trick of returning to the boundary of the subspace (Algorithm 3, Step 5) suggests a general mechanism for keeping iterate directions inside designed cones; extending Lemma 2.5 to higher-dimensional truncated subspaces is a testable next step.
  • The numerical comparison against SGD and L-BFGS on 380 classical problems is suggestive, but whether the advantage carries over to high-dimensional machine-learning losses is an open empirical question.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a class of 'model-driven truncated subspaces' for large-scale optimization, defines 16 such subspaces generated by gradients of quadratic or cubic-regularized models over various inspiring regions, and embeds them in a subspace trust-region/ARC-type algorithm called MD-LAMBO. It reports dimension computations for these subspaces, a modified truncated conjugate-gradient solver (Algorithm 3), sufficient-decrease results, and a global convergence theorem. A derivative-free variant is supported by P-fully quadratic model arguments and by two stochastic comparison theorems. Numerical experiments compare the 16 subspace variants on 380 test problems using performance profiles and truncated Newton step errors.

Significance. If the theoretical claims were established, the paper would make a useful contribution to subspace-based optimization: the taxonomy of model-driven/truncated subspaces and the dimension analysis in Section 2.1 are concrete and nontrivial, and the numerical comparison across 16 subspace designs is a valuable empirical study. However, the central convergence claim is not supported by the proofs as written. The sufficient-decrease result for the featured truncated subspace omits a key hypothesis of the lemma it invokes; the main convergence theorem is stated without assumptions that the proof actually needs; and the stochastic model comparisons in Section 2.3 are consequences of the imposed noise model rather than derived properties of fully quadratic interpolation. These are load-bearing gaps, not presentation issues, so the advertised theoretical guarantee cannot be accepted in its current form.

major comments (3)
  1. [Theorem 2.8, Section 2.2] Theorem 2.8 is stated for any Δ_k>0, g_k∈R^2, and symmetric B_k, and its proof asserts 'By Lemma 2.5, y_{k,1}, y_{k,2} ∈ \hat{S}'. Lemma 2.5, however, requires the additional condition ∥g_k∥²/(g_kᵀB_kg_k) ≤ (1/2)√(σ∥g_k∥) and positive definiteness of ∇²f(x_k). When that condition fails, Algorithm 3's Step 5 is triggered and the output need not coincide with the standard TCG solution, so Yuan's Theorem 2.7 cannot be invoked. Thus inequality (11) is not established for S^{(1,σ)}_l. Since Corollary 2.10 and Lemma 3.3 rely on (11), the sufficient-decrease foundation of Theorem 3.1 is unsupported. The claim that 'all type-2 truncated subspaces' also enjoy the 1/2 decrease is asserted without proof.
  2. [Theorem 3.1 and Lemma 3.3, Section 3] Theorem 3.1 as stated supposes only Assumption 2 and S_k ≠ S^{(1,σ)}_{s1,s2}, yet its proof uses Assumption 1, Assumption 3, and the unstated condition Δ_k→0. Lemma 3.3 explicitly assumes Δ_k→0, but Algorithm 2 does not establish this; in a standard trust-region update, successful steps can increase Δ_k. Moreover, Assumption 1 — the condition (Σλ_i²)(Σ1/λ_i) ≤ c with c∈[1/4,1/2] — is strong and is not shown to hold for any nontrivial problem class, and Assumption 3 essentially postulates that models satisfying it recur infinitely often. Without these additional hypotheses, the claimed global convergence does not follow from the proof.
  3. [Theorem 2.16 and 2.17, Section 2.3] Equation (15) posits that the gradient difference between the true quadratic model m_k and the interpolation model \hat{m}_k is exactly κ_eg Δ² c b/∥b∥ with c∼U[0,1] and b standard normal. The equalities in Theorems 2.16 and 2.17 are then direct consequences of E[b/∥b∥]=0 and the symmetry of bᵀŷ. This is a result by construction, not a property derived from the P-fully quadratic definition, which only bounds the norm of the gradient error. The paper does not justify why a general interpolation model's error should have this specific zero-mean isotopic distribution; the theorems therefore do not establish the claimed practical closeness between gradient-based and derivative-free models.
minor comments (4)
  1. [General presentation] There are numerous formatting and typographical issues: 'Correpsonding' in the footnote, garbled symbols in Tables 1–2, and inline mathematical expressions that are difficult to parse. The notation S is overloaded (subspace vs. set of vectors), and the caption under Algorithm 3 is incomplete ('In Algorithm 3STCG' followed by an unnumbered step list).
  2. [References] The introduction credits Moré's trust-region paper as [22], but Reference [22] is Varah's 1980 paper; the intended Moré reference appears to be missing. Reference [23], cited for the Johnson–Lindenstrauss lemma, is a computational advertising survey; a standard JL reference would be more appropriate.
  3. [Theorem 2.9 proof] The proof of Theorem 2.9 contains algebraic and dimensional slips, e.g., the line '= 1/4 Σλ_i s_i²' followed by claims involving ∥Qᵀs*∥ and the expression 'Pq i=1 y_i²/λ_i²' as a norm. The chain of inequalities should be rewritten carefully; as written it is hard to verify the constant c₁=1/(4c).
  4. [Section 4] The numerical comparison reports iteration counts but not CPU time or per-iteration cost, despite the paper's motivation being computational savings. Details on the choice of σ, σ_k, and trust-region parameters for the 16 variants are not given, making it difficult to assess whether the comparisons are parameter-fair.

Circularity Check

3 steps flagged

One auxiliary 'prediction' pair (Theorems 2.16-2.17) is built into the assumed zero-mean noise model; the main convergence theorem is not circular but has unproved intermediate links (omitted Lemma 2.5 hypothesis; unproved Delta_k -> 0).

specific steps
  1. self definitional [Section 2.3, Eq. (15) and Theorems 2.16-2.17]
    "Hence, it is reasonable to assume that the difference of the two models' gradients behaves like the product of a uniformly distributed random variable and a standard normal distributed random variable, i.e., ∇m_k(s)=∇\hat m_k(s)+κ_eg Δ^2 c b/∥b∥ ... We claim that under above assumptions, the expectation of m_k(0)-m_k(\hat s) exactly equals to the expectation of \hat m_k(0)-\hat m_k(\hat s)."

    The theorem is an algebraic identity implied by the zero-mean/symmetric noise model, not a derived property of P-fully quadratic models. In the proof the key step is E_{c,b}(κ_eg Δ^2 c b̂^T \hat y)=0, which holds only because b_i are i.i.d. N(0,1); Theorem 2.17's P(b^T \hat y≤0)=1/2 is the same symmetry. If the noise were not centered, neither conclusion would follow. Thus the claimed support for using P-fully quadratic models as substitutes is a consequence of Eq. (15) by construction, rather than an independent statistical finding.

  2. other [Section 2.2, Theorem 2.8 proof and Lemma 2.5]
    "By Lemma 2.5, y_{k,1}, y_{k,2} ∈ \hat S. Hence, if the problem is two-dimensional, the iteration sequence will never go to Step 2 and Step 5, and the output sequence will coincide with the sequence obtained by the standard TCG method in solving problem (5)."

    This is not circular, but the derivation chain has a missing link: Lemma 2.5 requires ∥g_k∥²/(g_kᵀ B_k g_k) ≤ (1/2)√(σ∥g_k∥), while Theorem 2.8 states the half-decrease bound for any Δ_k>0, g_k∈R² and symmetric B_k without that condition. When the condition fails, the CG iterates can leave \hat S_l^{(1,σ)}, Algorithm 3 Step 5 is invoked, and the output need not coincide with Yuan's TCG solution, so inequality (11) is unproved for the subspace featured in the experiments and needed by Lemma 3.3.

  3. other [Section 3, Lemma 3.3 and Theorem 3.1]
    "Lemma 3.3. Suppose Assumption 3 is satisfied and Δ_k → 0 as k → ∞ ... Since Δ_k → 0 as k → ∞, we conclude that for sufficiently large N, there must be ∥∇f(x_{k_N})∥ ≤ 2ε."

    This is an omitted-support issue, not circularity: the assumption Δ_k → 0 is not established by Algorithm 2 and can be violated on successful trust-region steps. Without it, the model-gradient error term κ_eg Δ_{k_N}² need not vanish, so the proof of Theorem 3.1 does not go through. This weakens the claimed 'global convergence' but does not make the theorem equivalent to its own inputs.

full rationale

The main subspace construction and convergence framework are not circular: the subspaces are defined explicitly, their geometry is derived from the definitions, Theorems 2.6-2.7 are quoted from Yuan [30] as external results, and Assumptions 1-3 are stated hypotheses rather than conclusions derived from themselves. The paper's own prior work appears mainly in benchmark/test-set details and definitions such as the truncated Newton step error, not as the load-bearing justification of the central theorem. The clearest circularity is in the derivative-free section: Theorems 2.16-2.17 assume a zero-mean symmetric stochastic model for the gradient difference and then 'prove' that expected decreases coincide and that one decrease exceeds the other with probability 1/2; both conclusions are immediate consequences of the assumed symmetry, so they are true by construction rather than independent evidence for fully quadratic models. In addition, two non-circular but load-bearing proof gaps exist: Theorem 2.8's half-decrease proof omits Lemma 2.5's key hypothesis, so the sufficient decrease underpinning Theorem 3.1 is not established for truncated subspaces; and Lemma 3.3 assumes Δ_k→0 without proving it. These gaps are correctness risks and are weighed separately. Overall score 4 reflects one real by-construction auxiliary 'prediction' while the central convergence claim still has independent mathematical content.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 0 invented entities

The central convergence claim rests on Assumptions 1 and 3, which essentially postulate the sufficient decrease and the recurring good models that the proof needs. The subspace construction itself is the only fully new content. The stochastic model (15) is an invented device used to manufacture the results of Theorems 2.16 and 2.17.

free parameters (4)
  • sigma (subspace construction) = user-chosen, often 0
    Sets the cubic term in the model p_k(s)=Q_k(s)+sigma*||s||^3/3 used to generate subspaces. The paper claims most subspaces are independent of its exact value.
  • sigma_k (adaptive cubic coefficient) = updated by ARC rule
    Used in the subproblem model m_k; the update rule is referenced to the ARC method but not explicitly specified.
  • Trust-region parameters (gamma1, gamma2, eta1, eta2, Delta0)
    Standard hyperparameters of the trust-region framework; not fitted to data.
  • c in Assumption 1 = assumed in [1/4,1/2]
    The constant in the eigenvalue inequality (sum lambda_i^2)(sum 1/lambda_i) <= c; the interval is exactly what makes the Cauchy-decrease constant c1 in [1/2,1].
axioms (6)
  • domain assumption Assumption 2: f is bounded below, twice continuously differentiable, Hessian bounded by kappa_H and Lipschitz with constant L.
    Standard regularity for trust-region convergence proofs; invoked in Section 3.
  • ad hoc to paper Assumption 1: (sum lambda_i^2)(sum 1/lambda_i) <= c with c in [1/4,1/2] for the projected model Hessian.
    This eigenvalue inequality is strong and unverified; it is exactly what yields the Cauchy decrease (14) used for convergence.
  • ad hoc to paper Assumption 3: for every successful iteration, some later successful iteration has a model satisfying Assumption 1.
    No mechanism in Algorithm 2 guarantees this recurrence; the convergence proof requires it.
  • ad hoc to paper Before Theorem 2.1, grad Q_k(0) != 0, Hessian Q_k(s) positive definite, and grad Q_k(0), Hessian Q_k(s) linearly independent.
    Used to simplify rank and dimension arguments for the subspace descriptions; not shown to hold at every iteration.
  • ad hoc to paper Stochastic gradient model (15): grad m_k(y) = grad \hat{m}_k(y) + kappa_eg Delta^2 c b/||b|| with c~U[0,1] and b_i iid N(0,1).
    This distributional assumption is asserted without justification and is the reason Theorems 2.16 and 2.17 hold.
  • ad hoc to paper Lemma 2.5 condition: ||g_k||^2/(g_k^T B_k g_k) <= (1/2) sqrt(sigma ||g_k||).
    The half-decrease theorem for S_l^(1,sigma) depends on this inequality, which is not guaranteed by the algorithm.

pith-pipeline@v1.3.0-alltime-deepseek · 23969 in / 16840 out tokens · 170790 ms · 2026-08-04T20:55:58.421462+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Model-Driven Subspaces for Large-Scale Optimization with Local Approximation Strategy." pith.science (2026). https://pith.science/paper/4K5UPIWJ

@misc{pith2026250908256,
  author       = {Pith},
  title        = {Pith review of: Model-Driven Subspaces for Large-Scale Optimization with Local Approximation Strategy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4K5UPIWJ}},
  note         = {Machine review of arXiv:2509.08256}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Solving large-scale optimization problems is a bottleneck and is very important for machine learning and multiple kinds of scientific problems. Subspace-based methods using the local approximation strategy are one of the most important methods. This paper discusses different and novel kinds of advanced subspaces for such methods and presents a new algorithm with such subspaces, called MD-LAMBO. Theoretical analysis including the subspaces' properties, sufficient function value decrease, and global convergence is given for the new algorithm. The related model construction on the subspaces is given under derivative-free settings. In numerical results, performance profiles, and truncated Newton step errors of MD-LAMBO using different model-driven subspaces are provided, which show subspace-dependent numerical differences and advantages of our methods and subspaces.

Figures

Figures reproduced from arXiv: 2509.08256 by Pengcheng Xie, Yitong He.

Figure 1
Figure 1. Figure 1: S (1,σ) l and its corresponding affine plane Write s ∈ span{∇pk(0)} as s = tg where t ∈ R. Then S (1,σ) l = [ t∈R span{(1 + σt2 ∥g∥)g + tBg}. Establish the affine coordinate system using {O, g, Bg} as the frame. Let g be the x-axis, Bg be the y-axis and O the origin. By representing each point with its affine coordinates, S (1,σ) l can be written as S (1,σ) l =  l [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Partition of subspace Sˆ (1,σ) l Proof. Proof Since Sˆ (1,σ) l ⊆ Sˆ (1,0) l , it suffices to prove the result for σ > 0. First of all, since span{gk} ⊆ Sˆ (1,σ) l and yk,1 = yk,0 + α0d0 = ∥gk∥ 2 g ⊤ k Bkgk (−gk), we know that yk,1 ∈ Sˆ (1,σ) l . By Bk = P ⊤ k ∇2f(xk)Pk, ∇2f(xk) is positive definite implies that Bk is also positive definite. Establish the affine coordinated system using {O, gk, Bkgk} as the… view at source ↗
Figure 3
Figure 3. Figure 3: Relationships between our subspaces For each p ∈ P and a ∈ A, let Na,p = min{n ∈ N, f n acc ≥ 1 − τ}, then Na,p is the minimal number of iterations such that the solution achieves the accuracy τ . For a fixed value of τ , we define the indicator Ta,p Ta,p = ( 1, if f N acc ≥ 1 − τ for some N, 0, otherwise, and the relative number of iterations ra,p ra,p =    Na,p min {Na,p ˜ : ˜a ∈ A, Ta,p ˜ = 1} , if T… view at source ↗
Figure 4
Figure 4. Figure 4: Performance profile of different subspaces [PITH_FULL_IMAGE:figures/full_fig_p022_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Truncated Newton Step Error momentum term of 0.9 was employed to accelerate convergence, and a damping factor (weight decay) of 10−4 was applied to improve stability. The method was run in a stochastic setting with uniformly sampled data points at each iteration. We also implemented the limited￾memory BFGS algorithm [18], which maintains a memory of the most recent 20 curvature updates to approximate the i… view at source ↗
Figure 6
Figure 6. Figure 6: Performance profile of different algorithms [PITH_FULL_IMAGE:figures/full_fig_p025_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Distributed Gradient-Regularized Newton Method: Scheduled Consensus and O(epsilon^{-1}) Global Iteration Complexity

    math.OC 2026-05 unverdicted novelty 7.0

    DisGrem achieves O(epsilon^{-1}) global iteration complexity for driving gradient norm below epsilon in decentralized convex optimization via scheduled consensus and vanishing regularization, matching centralized rate...

Reference graph

Works this paper leans on

35 extracted references · 20 canonical work pages · cited by 1 Pith paper · 3 internal anchors

  1. [1]

    Gheribi, Michael Kokkolaras, and S´ ebastien Le Digabel

    St´ ephane Alarie, Charles Audet, A ¨ ımen E. Gheribi, Michael Kokkolaras, and S´ ebastien Le Digabel. Two decades of blackbox optimization applications.EURO J. Comput. Optim., 9:100011, 2021. ISSN 2192-4406. doi:https://doi.org/10.1016/j.ejco.2021.100011. URL https://www.sciencedirect.com/science/article/pii/S2192440621001386

    arXiv 2021

  2. [2]

    Quadratic Models

    Charles Audet and Warren Hare. Quadratic Models. InDerivative-Free and Blackbox Op- timization, pages 172–175, 2017. URLhttps://api.semanticscholar.org/CorpusID: 125349659

    work page 2017

  3. [3]

    Scalable subspace methods for derivative- free nonlinear least-squares optimization.Math

    Coralia Cartis and Lindon Roberts. Scalable subspace methods for derivative- free nonlinear least-squares optimization.Math. Program., 199(1-2):461–524, 2023. doi:10.1007/s10107-022-01836-1

    work page doi:10.1007/s10107-022-01836-1 2023

  4. [4]

    Adaptive cubic regularisation methods for unconstrained optimization

    Coralia Cartis, Nicholas Gould, and Philippe Toint. Adaptive cubic regularisation methods for unconstrained optimization. part I: Motivation, convergence and numerical results. Math. Program., 127:245–295, 04 2011. doi:10.1007/s10107-009-0286-5

    doi:10.1007/s10107-009-0286-5 2011

  5. [5]

    A randomised subspace gauss-newton method for nonlinear least-squares, 2022

    Coralia Cartis, Jaroslav Fowkes, and Zhen Shao. A randomised subspace gauss-newton method for nonlinear least-squares, 2022. URLhttps://arxiv.org/abs/2211.05727

    Pith/arXiv arXiv 2022

  6. [6]

    Random subspace cubic-regularization methods, with applications to low-rank functions, 2025

    Coralia Cartis, Zhen Shao, and Edward Tansley. Random subspace cubic-regularization methods, with applications to low-rank functions, 2025. URLhttps://arxiv.org/abs/ 2501.09734

    Pith/arXiv arXiv 2025

  7. [7]

    Q-fully quadratic modeling and its applica- tion in a random subspace derivative-free method.Comput

    Yiwen Chen, Warren Hare, and Amy Wiebe. Q-fully quadratic modeling and its applica- tion in a random subspace derivative-free method.Comput. Optim. Appl., 89(2):317–360, 2024

    work page 2024

  8. [8]

    Andrew. R. Conn, Philippe L. Toint., Annick Sartenaer, and Nicholas I. M. Gould. On iterated-subspace minimization methods for nonlinear optimization. In Adams L. and Nazareth J. L., editors,Linear and Nonlinear Conjugate Gradient-Related Methods, pages 50–78. SIAM, Philadelphia, 1996

    work page 1996

  9. [9]

    Conn, Katya Scheinberg, and Lu ´ ıs N

    Andrew R. Conn, Katya Scheinberg, and Lu ´ ıs N. Vicente.Introduction to Derivative-Free Optimization, volume 8 ofMPS-SIAM Series on Optimization. MPS/SIAM, Philadelphia,

    work page

  10. [10]

    Dolan and Jorge J

    Elizabeth D. Dolan and Jorge J. Mor´ e. Benchmarking optimization software with perfor- mance profiles.Math. Program., 91:201–213, 2002. doi:10.1007/s101070100263. 25

    doi:10.1007/s101070100263 2002

  11. [11]

    Larson, Matt Menickelly, and Stefan M

    Kwassi Joseph Dzahini, Jeffrey M. Larson, Matt Menickelly, and Stefan M. Wild. A noise- aware scalable subspace classical optimizer for the quantum approximate optimization algorithm, 2025. URLhttps://arxiv.org/abs/2507.10992

    work page arXiv 2025

  12. [12]

    Wild, and Pengcheng Xie

    Kwassi Joseph Dzahini, Stefan M. Wild, and Pengcheng Xie. YATSOp: Yet another test set for optimization.https://github.com/POptUS/YATSOp, 2025. GitHub repository, accessed 2025-08-30

    work page 2025

  13. [13]

    John Wiley & Sons, Ltd, USA, 2000

    Roger Fletcher.Restricted Step Methods, chapter 5, pages 95–109. John Wiley & Sons, Ltd, USA, 2000. ISBN 9781118723203. doi:https://doi.org/10.1002/9781118723203.ch5. URLhttps://onlinelibrary.wiley.com/doi/abs/10.1002/9781118723203.ch5

    work page doi:10.1002/9781118723203.ch5 2000

  14. [14]

    Stochastic first- and zeroth-order methods for non- convex stochastic programming.SIAM J

    Saeed Ghadimi and Guanghui Lan. Stochastic first- and zeroth-order methods for non- convex stochastic programming.SIAM J. Optim., 23(4):2341–2368, 2013

    work page 2013

  15. [15]

    RSN: Randomized Subspace Newton

    Robert M. Gower, Dmitry Kovalev, Felix Lieder, and Peter Richt´ arik. RSN: Randomized subspace newton, 2019. URLhttps://arxiv.org/abs/1905.10874

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  16. [16]

    Expected decrease for derivative-free algorithms using random subspaces.Math

    Warren Hare, Lindon Roberts, and Cl´ ement Royer. Expected decrease for derivative-free algorithms using random subspaces.Math. Comput, 94(351):277–304, 2025

    work page 2025

  17. [17]

    New subspace method for unconstrained derivative-free optimization.ACM Trans

    Morteza Kimiaei, Arnold Neumaier, and Parvaneh Faramarzi. New subspace method for unconstrained derivative-free optimization.ACM Trans. Math. Softw., 49(4), December

    work page

  18. [18]

    Liu and Jorge Nocedal

    Dong C. Liu and Jorge Nocedal. On the limited memory bfgs method for large scale optimization.Math. Program., 45:503–528, 1989. URLhttps://api.semanticscholar. org/CorpusID:5681609

    work page 1989

  19. [19]

    Michael J. D. Powell. Restart procedures for the conjugate gradient method.Math. Program., 12(1):241–254, 1977. ISSN 1436-4646. doi:10.1007/BF01593790. URLhttps: //doi.org/10.1007/BF01593790

    work page doi:10.1007/bf01593790 1977

  20. [20]

    D. C. Sorensen. Newton’s method with a model trust region modification.SIAM Journal on Numerical Analysis, 19(2):409–426, 1982. doi:10.1137/0719026. URLhttps://doi. org/10.1137/0719026

    doi:10.1137/0719026 1982

  21. [21]

    Scalable Second-Order Optimization Algorithms for Minimizing Low-rank Functions

    Edward Tansley and Coralia Cartis. Scalable second-order optimization algorithms for minimizing low-rank functions, 2025. URLhttps://arxiv.org/abs/2501.03718

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  22. [22]

    J. M. Varah. Stability restrictions on second order, three level finite difference schemes for parabolic equations.SIAM J. Numer. Anal., 17(2):300–309, 1980. doi:10.1137/0717025. URLhttps://doi.org/10.1137/0717025

    work page doi:10.1137/0717025 1980

  23. [23]

    Woodruff

    David P. Woodruff. Computational advertising: Techniques for targeting relevant ads.Found. Trends Theor. Comput. Sci., 10(1–2):1–157, 2014. ISSN 1551-3068. doi:10.1561/0400000060. URLhttp://dx.doi.org/10.1561/0400000060

    doi:10.1561/0400000060 2014

  24. [24]

    Pengcheng Xie and Stefan M. Wild. ReMU: Regional minimal updating for model-based derivative-free optimization, 2025. URLhttps://arxiv.org/abs/2504.03606

    Pith/arXiv arXiv 2025

  25. [25]

    Least H2 norm updating of quadratic interpolation models for derivative-free trust-region algorithms.IMA Journal of Numerical Analysis, page drae106, 03 2025

    Pengcheng Xie and Ya xiang Yuan. Least H2 norm updating of quadratic interpolation models for derivative-free trust-region algorithms.IMA Journal of Numerical Analysis, page drae106, 03 2025. doi:10.1093/imanum/drae106. URLhttps://doi.org/10.1093/ imanum/drae106

    doi:10.1093/imanum/drae106 2025

  26. [26]

    A New Two-dimensional Model-based Subspace Method for Large-scale Unconstrained Derivative-free Optimization: 2D-MoSub

    Pengcheng Xie and Ya-xiang Yuan. A new two-dimensional model-based subspace method for large-scale unconstrained derivative-free optimization: 2D-MoSub, 2024. URLhttps: //arxiv.org/abs/2309.14855. 26

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  27. [27]

    Derivative-free optimization with transformed ob- jective functions (DFOTO) and the algorithm based on the least Frobenius norm up- dating quadratic model.J

    Pengcheng Xie and Ya-xiang Yuan. Derivative-free optimization with transformed ob- jective functions (DFOTO) and the algorithm based on the least Frobenius norm up- dating quadratic model.J. Oper. Res. Soc. China, 13:327–363, 2025. URLhttps: //link.springer.com/article/10.1007/s40305-023-00532-x

    doi:10.1007/s40305-023-00532-x 2025

  28. [28]

    A derivative-free method using a new under- determined quadratic interpolation model.SIAM J

    Pengcheng Xie and Ya-xiang Yuan. A derivative-free method using a new under- determined quadratic interpolation model.SIAM J. Optim., 35(2):1110–1133, 2025. doi:10.1137/23M1582023. URLhttps://doi.org/10.1137/23M1582023

    doi:10.1137/23m1582023 2025

  29. [29]

    Robinson, and Rene Vidal

    Chong You, Chun-Guang Li, Daniel P. Robinson, and Rene Vidal. Oracle based active set algorithm for scalable elastic net subspace clustering. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016

    work page 2016

  30. [30]

    On the truncated conjugate gradient method.Math

    Ya-xiang Yuan. On the truncated conjugate gradient method.Math. Program., 87(3): 561–573, 2000. doi:10.1007/s101079900126

    work page doi:10.1007/s101079900126 2000

  31. [31]

    Ya-xiang Yuan and J. Stoer. A subspace study on conjugate gradient algorithms.ZAMM Z. Angew. Math. Mech., 75:69–77, 1995

    work page 1995

  32. [32]

    DR- SOM: A dimension reduced second-order method, 2023

    Chuwen Zhang, Dongdong Ge, Chang He, Bo Jiang, Yuntian Jiang, and Yinyu Ye. DR- SOM: A dimension reduced second-order method, 2023. URLhttps://arxiv.org/abs/ 2208.00208

    Pith/arXiv arXiv 2023

  33. [33]

    Scalable derivative-free optimization algorithms with low-dimensional sub- space techniques, 2025

    Zaikun Zhang. Scalable derivative-free optimization algorithms with low-dimensional sub- space techniques, 2025. URLhttps://arxiv.org/abs/2501.04536. 27

    Pith/arXiv arXiv 2025

  34. [2009]

    URLhttp://epubs.siam.org/doi/book/10.1137/ 1.9780898718768

    doi:10.1137/1.9780898718768. URLhttp://epubs.siam.org/doi/book/10.1137/ 1.9780898718768

    doi:10.1137/1.9780898718768

  35. [2023]

    doi:10.1145/3618297

    ISSN 0098-3500. doi:10.1145/3618297. URLhttps://doi.org/10.1145/3618297

    doi:10.1145/3618297