Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

A Model-Based Derivative-Free Optimization Algorithm for Partially Separable Problems

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read UPOQA is a trust-region DFO method that models each element of a partially separable objective separately and thereby reaches target accuracy with far fewer function evaluations than treating the problem as a single black box.

desk verdict A real algorithmic contribution whose headline success rates depend on a benchmark target that is the best solver value, so read the absolute numbers with care. read the letter →

arxiv 2506.21948 v2 pith:XJHOKGX5 submitted 2025-06-27 math.OC

Yichuan Liu , Yingzhou Li This is my paper · ORCID
classification math.OC MSC 90C5690C3065K0590C26
keywords derivative-freeoptimizationpartiallyseparableproblemstrustregionmethodsquadraticinterpolationstructuredregionsSteinmetzprojectionspeed-upprofilequantumvariationalalgorithms
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

UPOQA is a derivative-free optimization algorithm aimed at unconstrained problems that decompose into a sum of element functions. The paper's central claim is that by building a separate quadratic interpolation model for each element and giving each its own trust-region radius, the algorithm reaches a target accuracy with far fewer evaluations of the expensive objective than methods that treat the whole problem as one black box. On an 85-problem test set, UPOQA is reported to solve 97.6%, 88.2%, and 84.7% of problems at tolerances $\varepsilon = 10^{-1}, 10^{-5}, 10^{-7}$, respectively, the highest success rates among the compared solvers, and to use the fewest evaluations on at least 80% of problems at every tolerance. The paper also introduces a speed-up profile showing that the advantage of exploiting partial separability is modest at low precision and grows when high precision is required, and it demonstrates the method on quantum variational problems where element functions are circuit expectation values.

What carries the argument

The load-bearing object is the structured trust region $\mathcal{S}(\mathbf{\Delta}_k)=\{s\in\mathbb{R}^n : \|s_{\mathcal{I}_i}\|_2\le \Delta_{k,i},\, i=1,\dots,q\}$, the intersection of one cylinder per element, together with the element models $m_{k,i}$ that it bounds. UPOQA maintains an underdetermined quadratic interpolation model per element, updated by the derivative-free symmetric Broyden update, so one full objective evaluation supplies every element value and refreshes all models. The structured subproblem is solved by a modified projected gradient method whose projection step is the Steinmetz projection, a fixed-iteration approximate projection onto this intersection of cylinders. Per-element radii are then adjusted by a combined separation score that separates global model agreement from element-level agreement and tolerates slight element deterioration when elements work against one another. These pieces together convert knowledge of which variables each element depends on into a lower per-iteration modeling cost and a larger per-step search space than a single spherical trust region would allow.

What would settle it

Re-run the 85 CUTEst problems with the reference value $f^*_p$ in Eq. (13) replaced by a certified or independently verified minimum for each problem, or with a much larger evaluation budget; if UPOQA's reported success rates fall below the competing solvers, or its evaluation counts approach those of the single-element variant, the claimed speed-up from partial separability is an artifact of the test reference.

Watch

Extended reading notes

Core claim

The central claim is that a derivative-free trust-region method can exploit coordinate partial separability—where the objective is a sum of element functions, each depending on a small subset of variables—to reach the same accuracy with substantially fewer function evaluations than methods that query only the full objective. UPOQA assigns each element its own interpolation model and its own trust-region radius, so one evaluation of the full objective refreshes every element model at once, and steps are constrained only in the low-dimensional subspaces where each element actually varies. On the paper's 85-problem CUTEst suite, UPOQA solves 97.6%, 91.8%, 88.2%, and 84.7% of problems at tolerances $10^{-1}$, $10^{-3}$, $10^{-5}$, and $10^{-7}$, respectively; at three of the four tolerances this is the highest success rate among UPOQA, UPOQA treating the objective as a single element, NEWUOA, and L-BFGS-B with finite-difference gradients, and on successfully solved problems UPOQA uses the fewest evaluations on at least 80% of problems at every tolerance. The paper also reports that acceleration from structure is modest at low precision and grows at high precision, and that UPOQA escapes plateau regions faster on two quantum variational problems.

Load-bearing premise

The reported success rates treat a problem as solved once a solver reaches a value within tolerance of the best value any solver in the test happened to find, rather than the true minimum of that problem; if all solvers stall at the same suboptimal plateau, UPOQA can be counted as converged without actually solving the problem.

Editorial extensions

If this is right

  • At tolerances $\varepsilon = 10^{-1}$, $10^{-5}$, and $10^{-7}$, UPOQA achieves success rates of 97.6%, 88.2%, and 84.7% on the 85-problem test set, the highest among the compared solvers, and at every tolerance it uses the fewest evaluations on at least 80% of solved problems.
  • The speed-up profile shows that exploiting partial separability pays off mainly when high precision is demanded: predicted and actual speed-up agree poorly at coarse tolerance and the advantage strengthens as the tolerance tightens.
  • On quantum variational problems with $p$ parameterized circuits, reformulating each energy term and overlap penalty as an element yields the predicted speed-up ratio of about $p/2$ and faster escape from plateau regions.
  • Because a single full-objective evaluation supplies every element value, evaluation counts should grow with the largest element dimension rather than with the full dimension $n$ on problems whose elements can be evaluated independently.
  • The optional restart and hybrid black-white-box modes extend the method to noisy objectives and to objectives with known smooth parts, where element models only need to cover the black-box residuals.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the element-wise evaluation assumption holds, the same modelling strategy should transfer to overlapping-element problems beyond the test suite, such as objectives from PDE discretizations where each element touches few neighbours; the paper itself notes its CUTEst selection mostly has many small elements, so this is a natural next stress test.
  • The Steinmetz projection is a geometric primitive independent of derivative-free optimization: any trust-region or proximal method whose feasible set is an intersection of cylinders could reuse it in place of slower alternating or Dykstra projections, at the price of the approximation the paper accepts.
  • The speed-up profile, which divides actual evaluation savings by the predicted low-dimensionality ratio, could serve as a general benchmark for how efficiently any structured solver exploits problem structure, beyond partial separability.
  • A clean ablation test for the soft restart would be to run UPOQA with and without restart on noisy versions of the CUTEst problems, isolating whether the documented robustness comes from the restart or from the structured trust regions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes UPOQA, a trust-region derivative-free optimization algorithm for unconstrained partially separable problems. It maintains underdetermined quadratic interpolation models for each element function, solves the resulting structured trust-region subproblem by a modified projected gradient method built on a new approximate Steinmetz projection, and adjusts element-wise trust-region radii through a scoring scheme based on Shahabuddin's combined separation criterion. The manuscript also describes a starting-point search, an optional restart mechanism, and hybrid black-white-box support. Numerical experiments on 85 CUTEst problems and two quantum variational problems report that UPOQA reduces the number of function evaluations relative to UPOQA without structure exploitation, NEWUOA, and finite-difference L-BFGS-B, and a newly introduced speed-up profile is used to quantify the acceleration.

Significance. If the numerical claims withstand scrutiny, UPOQA is a useful contribution to structured derivative-free optimization. The paper ships an open-source Python implementation, provides a correctness proof for the SHRINK subroutine in Theorem 2.1, and introduces a speed-up profile that is a potentially useful diagnostic for structure exploitation. The main caveat is that the reported success rates and evaluation counts rest on a convergence target defined from the solvers' own achieved values rather than an absolute or independently certified optimum; this needs to be fixed before the empirical contribution can be accepted at face value.

major comments (3)
  1. [Section 3.1, Eq. (13)] The convergence criterion defines f*_p as the lowest function value achieved by any solver in S on problem p. If all solvers stall at the same local minimum or plateau, then every solver is declared converged without being close to a true solution, and the reported success rates and evaluation counts are measured against a self-referential target. This is not only a theoretical concern: in Table 1, for HYDC20LS at epsilon = 1e-5 and 1e-7, only UPOQA has finite evaluation counts, so f*_p is essentially UPOQA's own best value, and the criterion can be satisfied by a small improvement of one solver over itself. The screening in Section 3.2.1 that removes problems where no solver converges further removes exactly the cases that would reveal an unreliable common plateau. Please rerun the comparison with known CUTEst minima or another independently certified target, and report final function values for the converged cases; the headline numbers in Section 3.2.2 should be recomputed against an absolute target.
  2. [Section 3.2.2] The experimental comparison does not include PSDFO [14], the existing model-based derivative-free algorithm designed specifically for partially separable problems. Because UPOQA's distinguishing features, namely element-wise quadratic models and trust-region management, are precisely the aspects that PSDFO also addresses, the reported advantage over NEWUOA and L-BFGS-B may reflect the benefit of partial separability in general rather than UPOQA's specific mechanisms. Please either include PSDFO in the comparison or explain why a direct comparison is not feasible, and interpret the numerical claims with that limitation stated.
  3. [Section 3.1, Eq. (14) and Section 3.3] The speed-up profile is presented as comparing the actual speed-up t_single/t_wst with a predicted speed-up ratio n/max_i n_i. The dimension ratio is motivated by a finite-difference gradient argument, but UPOQA uses interpolation models and trust regions, and no argument is given that n/max_i n_i is the relevant predicted speed-up for this algorithm. The quantity c_p is therefore a normalization of the measured speed-up rather than a tested prediction, so statements such as 'the results align well with theoretical predictions' in Section 3.3 are stronger than the derivation supports. Please state explicitly that c_p is a normalized speed-up and either justify the prediction language or weaken it.
minor comments (6)
  1. [Section 2.1, Algorithm 2] In Step 2 of Algorithm 2, the violation ratios are written using s_0, but they should be computed from the current iterate s_k; please fix this typo.
  2. [Section 2.1, Algorithm 2] The complexity statement says the algorithm iterates at most 'q - q' times; the overline notation is not defined in the text, and the reader cannot tell which quantity is being subtracted from q.
  3. [Section 3.2.2] The solver name 'NEWUOQA' appears once; it should be 'NEWUOA'.
  4. [Algorithm 1, line 11] The condition 'F_GI_1,,...,F_GI_q' contains an extra comma; this is a typographical error.
  5. [Figures 3-5] The axis labels in Figures 3-5 are truncated in the manuscript, for example 'Number f Evaluati ns' and 'log2 ( )'; please regenerate the figures with complete labels.
  6. [Section 4] The conclusion correctly acknowledges that theoretical analysis is incomplete and that the artificially constructed CUTEst structures may lack representativeness; these limitations should be stated in the abstract or introduction so that the empirical framing of the contribution is transparent from the outset.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor self-referential benchmark target in Eq. (13); the algorithmic contribution itself is not circular.

  1. other [Section 3.1, Eq. (13) and Section 3.2.2]
    "a solver s∈S is said to converge on problem p with tolerance ε if and only if its iterate x satisfies f_p(x) ≤ f*_p + ε[f_p(x0,p) − f*_p], where f*_p is the lowest function value achieved by any solver in S on problem p."

    The convergence threshold f*_p is not an independently verified optimum; it is the minimum over the same solver set S that includes UPOQA itself. Therefore, on any problem where UPOQA attains the lowest observed value, UPOQA is automatically counted as converged at every tolerance, even if that value is far from a true minimum. The headline success rates (97.6%, 88.2%, 84.7%) and 'fewest evaluations' claims are thus measured against a target that UPOQA itself helps define. This is a self-referential benchmark convention rather than a fitted parameter, and it does not affect the construction of the algorithm, but it makes the empirical superiority claim less independent than it appears.

full rationale

The paper's core contribution—an underdetermined quadratic interpolation model, Steinmetz projection for structured trust regions, per-element radius management, and selective model updates—is derived from Powell-style model-based DFO and is not fitted to the benchmark results. The speed-up profile in Eq. (14) normalizes the measured evaluation ratio by a simple cost-model denominator n_p/max_i n_{p,i}; this is an evaluation metric, not a prediction derived from UPOQA, and it does not reduce to the algorithm's own output. The self-citations to qOMM [4,5] provide quantum test-circuit designs and background structure, but the claimed acceleration is demonstrated by direct comparison against NEWUOA and L-BFGS-B (ffd) and by UPOQA-versus-UPOQA(single) ratios, so those citations are not load-bearing. The only notable issue is the convergence test in Eq. (13), which uses the best solver value as the reference; this is a standard but self-referential benchmarking choice that can inflate success rates when all solvers plateau. Because this affects the interpretation of the empirical claims rather than the algorithmic derivation, the overall circularity is minor.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

UPOQA's central claim rests on coordinate partial separability, independent element evaluability, smoothness for quadratic interpolation, and heuristic algorithm parameters. No new physical or mathematical entities beyond the new projection operator are postulated.

free parameters (8)
  • theta_1 = 0.5
    Radius shrink factor in Section 2.2, chosen by hand; affects how quickly element trust regions contract when scores are low.
  • theta_2 = 1/sqrt(2) approximately 0.7071
    Radius shrink factor in Section 2.2, chosen by hand.
  • theta_3 = sqrt(2) approximately 1.4142
    Radius growth factor in Section 2.2, chosen by hand.
  • theta_4 = 2
    Radius growth factor in Section 2.2, chosen by hand.
  • xi = 1e-5
    Threshold for accepting interpolation points in Section 2.3; hand-chosen and affects model update decisions.
  • k_avg start-up iterations = 0 and 4
    Number of averaged projection iterations used alternately in Algorithm 4, selected in Section 2.1 as a heuristic.
  • mu_1 and mu_2 scoring thresholds = not specified in the paper
    Constants used in Algorithm 5 to set global scores; the paper states only that 0 < mu_1 < mu_2 < 1, leaving implicit free parameters.
  • quantum penalty coefficients = lambda=12 for H2, lambda=30 for H4
    Empirically chosen in Section 3.3; these values shape the quantum variational test problems.
assumptions (5)
  • domain assumption Each element function f_i depends only on its elemental variables and can be evaluated independently.
    Stated in Section 1: 'We further assume that each element function can be evaluated independently.' This underpins the entire element-model strategy.
  • domain assumption Each element function is smooth enough to be locally approximated by a quadratic interpolation model over its trust region.
    Inherited from Powell's model-based DFO framework; the paper uses underdetermined quadratic models without proving sufficient smoothness for the test problems.
  • domain assumption The S2MPJ group partially separable decomposition of CUTEst SIF problems is a valid representation of the true elemental structure.
    Section 3.2.1: each objective-function group becomes an element; if this decomposition is artificial, the measured speed-ups may not transfer to real problems.
  • domain assumption The convergence benchmark uses f*_p as the best value found by any solver, and this is treated as a valid reference.
    Section 3.1 Eq. (13). This benchmarking convention can overstate success when all solvers stagnate at a common suboptimal value.
  • ad hoc to paper The modified projected gradient method with approximate Steinmetz projection reliably solves the structured trust-region subproblem.
    Algorithm 4 is presented without a convergence proof; Theorem 2.1 only proves a property of the SHRINK subroutine, not the accuracy of the projection or the quality of the subproblem solution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Model-Based Derivative-Free Optimization Algorithm for Partially Separable Problems." pith.science (2026). https://pith.science/paper/XJHOKGX5

@misc{pith2026250621948,
  author       = {Pith},
  title        = {Pith review of: A Model-Based Derivative-Free Optimization Algorithm for Partially Separable Problems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XJHOKGX5}},
  note         = {Machine review of arXiv:2506.21948}
}
read the original abstract

We propose UPOQA, a derivative-free optimization algorithm for partially separable unconstrained problems, leveraging quadratic interpolation and a structured trust-region framework. By decomposing the objective into element functions, UPOQA constructs underdetermined element models and solves subproblems efficiently via a modified projected gradient method. Innovations include an approximate projection operator for structured trust regions, improved management of elemental radii and models, a starting point search mechanism, and support for hybrid black-white-box optimization, etc. Numerical experiments on 85 CUTEst problems demonstrate that \texttt{UPOQA} can significantly reduce the number of function evaluations. To quantify the impact of exploiting partial separability, we introduce the speed-up profile to further evaluate the acceleration effect. Results show that the speed-up of UPOQA over baselines is less significant in low-precision scenarios but becomes more pronounced in high-precision scenarios. Applications to quantum variational problems further validate its practical utility.

Figures

Figures reproduced from arXiv: 2506.21948 by the authors.

Figure 1
Figure 1. Generation of the surface of a Steinmetz solid. [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. The combined separation criterion proposed by Shahabuddin [ [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Performance profiles of UPOQA, UPOQA (single), NEWUOA, and L-BFGS-B (ffd) on the test problems from [PITH_FULL_IMAGE:figures/full_fig_p020_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Data profiles of UPOQA, UPOQA (single), NEWUOA, and L-BFGS-B (ffd) on the test problems from [PITH_FULL_IMAGE:figures/full_fig_p021_4.png]
Figure 5
Figure 5. Figure 5: Speed-up profiles computed by UPOQA and UPOQA (single) on the test problems from [PITH_FULL_IMAGE:figures/full_fig_p021_5.png]
Figure 6
Figure 6. Figure 6: Objective function value descent curves when optimizing the noiseless quantum variational problem (17) for [PITH_FULL_IMAGE:figures/full_fig_p023_6.png]
Figure 7
Figure 7. Figure 7: Performance profiles of UPOQA and its unstructured-trust-region variant UPOQA (non-struct) on the test problems from [PITH_FULL_IMAGE:figures/full_fig_p027_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Adaptive direct search algorithms with relaxable and quantifiable constraints

    math.OC 2026-07 conditional novelty 6.0 of 10

    ADS-PB extends mesh-free adaptive direct search to constrained blackbox optimization by incorporating a progressive barrier mechanism, with convergence guarantees and improved practical performance over mesh-based methods.

Reference graph

Works this paper leans on

66 extracted references · 65 canonical work pages · cited by 1 Pith paper

  1. [14]

    Benoît Colson and Philippe L. Toint. 2005. Optimizing partially separable functions without derivatives. Optim. Methods Softw. 20, 4-5 (2005), 493–508

    work page 2005

  2. [1]

    Bauschke and Jonathan M

    Heinz H. Bauschke and Jonathan M. Borwein. 1993. On the convergence of von Neumann’s alternating projection algorithm for two sets. Set-Valued Var. Anal. 1, 2 (1993), 185–212

    work page 1993

  3. [2]

    Berahas, Liyuan Cao, Krzysztof Choromanski, and Katya Scheinberg

    Albert S. Berahas, Liyuan Cao, Krzysztof Choromanski, and Katya Scheinberg. 2022. A theoretical and empirical comparison of gradient approxima- tions in derivative-free optimization. Found. Comut. Math. 22, 2 (2022), 507–560

    work page 2022

  4. [3]

    Bertsekas et al

    Dimitri P. Bertsekas et al. 2011. Incremental gradient, subgradient, and proximal methods for convex optimization: A survey. Optimization for Machine Learning 2010, 1-38 (2011), 3

    work page 2011

  5. [4]

    Joel Bierman, Yingzhou Li, and Jianfeng Lu. 2022. Quantum orbital minimization method for excited states calculation on a quantum computer. J. Chem. Theory Comput. 18, 8 (2022), 4674–4689

    work page 2022

  6. [5]

    Joel Bierman, Yingzhou Li, and Jianfeng Lu. 2023. Improving the Accuracy of Variational Quantum Eigensolvers with Fewer Qubits Using Orbital Optimization. J. Chem. Theory Comput. 19, 3 (2023), 790–798

    work page 2023

  7. [6]

    Zyed Bouzarkouna, Anne Auger, and Didier Y. Ding. 2011. Local-meta-model CMA-ES for partially separable functions. In Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation (Dublin, Ireland) (GECCO ’11). Association for Computing Machinery, New York, NY, USA, 869–876

    work page 2011

  8. [7]

    Boyle and Richard L

    James P. Boyle and Richard L. Dykstra. 1986. A Method for Finding Projections onto the Intersection of Convex Sets in Hilbert Spaces. In Advances in Order Restricted Statistical Inference . Springer New York, New York, NY, 28–47. Manuscript submitted to ACM 24 Y. Liu et al

    work page 1986

Show all 66 references
  1. [8]

    Huiping Cao and Lan Yao. 2016. A partitioned PSB method for partially separable unconstrained optimization problems. Appl. Math. Comput. 290 (2016), 164–177

    2016

  2. [9]

    Coralia Cartis, Jan Fiala, Benjamin Marteau, and Lindon Roberts. 2019. Improving the flexibility and robustness of model-based derivative-free optimization solvers. ACM Trans. Math. Softw. 45, 3 (2019), 1–41

    2019

  3. [10]

    Coralia Cartis, Lindon Roberts, and Oliver Sheridan-Methven. 2022. Escaping local minima with local derivative-free methods: a numerical investigation. Optimization 71, 8 (2022), 2343–2373

    2022

  4. [11]

    Benjamin, Suguru Endo, Keisuke Fujii, Jarrod R

    Marco Cerezo, Andrew Arrasmith, Ryan Babbush, Simon C. Benjamin, Suguru Endo, Keisuke Fujii, Jarrod R. McClean, Kosuke Mitarai, Xiao Yuan, Lukasz Cincio, et al. 2021. Variational quantum algorithms. Nat. Rev. Phys. 3, 9 (2021), 625–644

    2021

  5. [12]

    Benoît Colson and Philippe L. Toint. 2001. Exploiting band structure in unconstrained optimization without derivatives. Optim. Eng. 2 (2001), 399–412

    2001

  6. [13]

    Benoît Colson and Philippe L. Toint. 2002. A derivative-free algorithm for sparse unconstrained optimization problems. In Trends in Industrial and Applied Mathematics: Proceedings of the 1st International Conference on Industrial and Applied Mathematics of the Indian Subcontin...

    2002

  7. [15]

    Conn, Nicholas I

    Andrew R. Conn, Nicholas I. M. Gould, Annick Sartenaer, and Philippe L. Toint. 1996. Convergence properties of minimization algorithms for convex constraints using a structured trust region. SIAM J. Optim. 6, 4 (1996), 1059–1086

    1996

  8. [16]

    Conn, Nicholas I

    Andrew R. Conn, Nicholas I. M. Gould, and Philippe L. Toint. 1994. Improving the Decomposition of Partially Separable Functions in the Context of Large-Scale Optimization: a First Approach . Springer US, Boston, MA, 82–94

    1994

  9. [17]

    Conn, Nicholas I

    Andrew R. Conn, Nicholas I. M. Gould, and Philippe L. Toint. 2010. LANCELOT: A Fortran Package for Large-Scale Nonlinear Optimization (Release A) (1st ed.). Vol. 17. Springer Publishing Company, Incorporated

    2010

  10. [18]

    Conn, Katya Scheinberg, and Luis N

    Andrew R. Conn, Katya Scheinberg, and Luis N. Vicente. 2009. Introduction to Derivative-Free Optimization . SIAM, USA

    2009

  11. [19]

    Conn and Philippe L

    Andrew R. Conn and Philippe L. Toint. 1996. An Algorithm using Quadratic Interpolation for Unconstrained Derivative Free Optimization . Springer US, Boston, MA, 27–47

    1996

  12. [20]

    da Silva Santos, Marcos S

    Carlos H. da Silva Santos, Marcos S. Goncalves, and Hugo E. Hernandez-Figueroa. 2010. Designing novel photonic devices by bio-inspired computing. IEEE Photonics Technol. Lett. 22, 15 (2010), 1177–1179

    2010

  13. [21]

    Daydé, Jean-Yves L’Excellent, and Nicholas I

    Michel J. Daydé, Jean-Yves L’Excellent, and Nicholas I. M. Gould. 1997. Element-by-element preconditioners for large partially separable optimization problems. SIAM J. Sci. Comput. 18, 6 (1997), 1767–1787

    1997

  14. [22]

    Frank Deutsch and Hein Hundal. 1994. The rate of convergence of dykstra’s cyclic projections algorithm: The polyhedral case. Numerical Functional Analysis and Optimization 15, 5-6 (1994), 537–565

    1994

  15. [23]

    Ding and Christopher F

    Daisy Y. Ding and Christopher F. McKee. 2011. Using partial separability of the objective function for gradient-based optimizations in History Matching. In SPE Reservoir Simulation Conference , Vol. SPE Reservoir Simulation Symposium. SPE, The Woodlands, Texas, USA, SPE–140811–MS

    2011

  16. [24]

    Nicholas I. M. Gould, Dominique Orban, and Philippe L. Toint. 2015. CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput. Optim. Appl. 60 (2015), 545–557

    2015

  17. [25]

    Serge Gratton and Philippe L. Toint. 2025. S2MPJ and CUTEst optimization problems for Matlab, Python and Julia. Optimization Methods and Software (2025), 1–33

    2025

  18. [26]

    Andreas Griewank and Philippe L. Toint. 1982. On the Unconstrained Optimization of Partially Separable Functions . Academic Press, London, 301–312

    1982

  19. [27]

    Andreas Griewank and Philippe L. Toint. 1982. Partitioned variable metric updates for large structured optimization problems. Numer. Math. 39, 1 (1982), 119–137

    1982

  20. [28]

    Andreas Griewank and Philippe L. Toint. 1984. On the existence of convex decompositions of partially separable functions. Math. Program. 28 (1984), 25–49

    1984

  21. [29]

    Gross and Geoffrey T

    James C. Gross and Geoffrey T. Parks. 2022. Optimization by moving ridge functions: derivative-free optimization for computationally intensive functions. Eng. Optim. 54, 4 (2022), 553–575

    2022

  22. [30]

    Nikolaus Hansen. 2006. The CMA Evolution Strategy: A Comparing Review . Springer Berlin Heidelberg, Berlin, Heidelberg, 75–102

    2006

  23. [31]

    Willi Hock and Klaus Schittkowski. 1980. Test examples for nonlinear programming codes. J. Optim. Theory Appl. 30 (1980), 127–129

    1980

  24. [32]

    Matthew Hough and Lindon Roberts. 2022. Model-based derivative-free methods for convex-constrained optimization. SIAM J. Optim. 32, 4 (2022), 2552–2579

    2022

  25. [33]

    Bulent Karasözen. 2007. Survey of derivative free optimization methods based on interpolation. J. Ind. Manag. Optim. 3, 2 (2007), 321–334

    2007

  26. [34]

    İlker Birbil, A

    Kamer Kaya, Figen Öztoprak, Ş. İlker Birbil, A. Taylan Cemgil, Umut Şimşekli, Nurdan Kuru, Hazal Koptagel, and M. Kaan Öztürk. 2019. A framework for parallel second order incremental optimization algorithms for solving partially separable problems. Comput. Optim. Appl. 72 (201...

    2019

  27. [35]

    Stefan Kern, Nikolaus Hansen, and Petros Koumoutsakos. 2006. Local meta-models for optimization using evolution strategies. In Parallel Problem Solving from Nature - PPSN IX . Springer Berlin Heidelberg, Berlin, Heidelberg, 939–948

    2006

  28. [36]

    Gelatt Jr., and Mario P

    Scott Kirkpatrick, Charles D. Gelatt Jr., and Mario P. Vecchi. 1983. Optimization by simulated annealing. Science 220, 4598 (1983), 671–680

    1983

  29. [37]

    Lescrenier

    M. Lescrenier. 1988. Partially separable optimization and parallel computing. Ann. Oper. Res. 14 (1988), 213–224. Manuscript submitted to ACM A Model-Based Derivative-Free Optimization Algorithm for Partially Separable Problems 25

    1988

  30. [38]

    Lewis, Russell Luke, and Jérôme Malick

    Adrian S. Lewis, Russell Luke, and Jérôme Malick. 2009. Local linear convergence for alternating and averaged nonconvex projections. Found. Comut. Math. 9, 4 (2009), 485–513

    2009

  31. [39]

    Lewis and Virginia Torczon

    Robert M. Lewis and Virginia Torczon. 1999. Pattern search algorithms for bound constrained minimization. SIAM J. Optim. 9, 4 (1999), 1082–1099

    1999

  32. [40]

    Lewis, Virginia Torczon, and Michael W

    Robert M. Lewis, Virginia Torczon, and Michael W. Trosset. 2000. Direct search methods: then and now. J. Comput. Appl. Math. 124, 1-2 (2000), 191–207

    2000

  33. [41]

    Omidvar, Zhenyu Yang, and Kai Qin

    Xiaodong Li, Ke Tang, Mohammad N. Omidvar, Zhenyu Yang, and Kai Qin. 2013. Benchmark functions for the CEC 2013 special session and competition on large-scale global optimization. Gene 7, 33 (2013), 8

    2013

  34. [42]

    Zhi-Quan Luo and Paul Tseng. 1993. Error bounds and convergence analysis of feasible descent methods: a general approach. Ann. Oper. Res. 46, 1 (1993), 157–178

    1993

  35. [43]

    Moré and Stefan M

    Jorge J. Moré and Stefan M. Wild. 2009. Benchmarking derivative-free optimization algorithms. SIAM J. Optim. 20, 1 (2009), 172–191

    2009

  36. [44]

    Rodrigue Oeuvray. 2005. Trust-Region Methods Based on Radial Basis Functions with Application to Biomedical Imaging . Ph. D. Dissertation. École polytechnique fédérale de Lausanne, Lausanne

    2005

  37. [45]

    Margherita Porcelli and Philippe L. Toint. 2022. Exploiting problem structure in derivative free optimization. ACM Trans. Math. Softw. 48, 1 (2022), 1–25

    2022

  38. [46]

    Michael J. D. Powell. 1964. An efficient method for finding the minimum of a function of several variables without calculating derivatives. Comput. J. 7, 2 (1964), 155–162

    1964

  39. [47]

    Michael J. D. Powell. 1965. A method for minimizing a sum of squares of non-linear functions without calculating derivatives. Comput. J. 7, 4 (1965), 303–307

    1965

  40. [48]

    Michael J. D. Powell. 1994. A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation . Springer Netherlands, Dordrecht. 51–67 pages

    1994

  41. [49]

    Michael J. D. Powell. 2004. Least Frobenius norm updating of quadratic models that satisfy interpolation conditions. Math. Program. 100, 1 (May 2004), 183–215

    2004

  42. [50]

    Michael J. D. Powell. 2004. On updating the inverse of a KKT matrix. Numerical Linear Algebra and Optimization (Science Press, Beijing) (2004), 56–78

    2004

  43. [51]

    Michael J. D. Powell. 2006. The NEWUOA Software for Unconstrained Optimization without Derivatives . Springer US, Boston, MA, 255–297

    2006

  44. [52]

    Michael J. D. Powell. 2008. Developments of NEWUOA for minimization without derivatives. IMA J. Numer. Anal. 28, 4 (02 2008), 649–664

    2008

  45. [53]

    Michael J. D. Powell. 2009. The BOBYQA algorithm for bound constrained optimization without derivatives. Cambridge NA Report NA2009/06, University of Cambridge, Cambridge 26 (2009), 26–46

    2009

  46. [54]

    Price and Philippe L

    Christopher J. Price and Philippe L. Toint. 2006. Exploiting problem structure in pattern search methods for unconstrained optimization. Optim. Methods Softw. 21, 3 (2006), 479–491

    2006

  47. [55]

    Ragonneau

    Tom M. Ragonneau. 2023. Model-based derivative-free optimization methods and software . Ph. D. Dissertation. Hong Kong Polytechnic University, Hong Kong, China

    2023

  48. [56]

    Ragonneau and Zaikun Zhang

    Tom M. Ragonneau and Zaikun Zhang. 2024. PDFO: a cross-platform package for Powell’s derivative-free optimization solvers. Math. Program. Comput. 16, 4 (2024), 535–559

    2024

  49. [57]

    McClean, Cornelius Hempel, Peter J

    Jonathan Romero, Ryan Babbush, Jarrod R. McClean, Cornelius Hempel, Peter J. Love, and Alán Aspuru-Guzik. 2018. Strategies for quantum computing molecular energies using the unitary coupled cluster ansatz. Quantum Sci. Technol. 4, 1 (2018), 014008

    2018

  50. [58]

    Raymond Ros and Nikolaus Hansen. 2008. A simple modification in CMA-ES achieving linear time and space complexity. In International Conference on Parallel Problem Solving from Nature . Springer, Springer Berlin Heidelberg, Berlin, Heidelberg, 296–305

    2008

  51. [59]

    Runarsson and Xin Yao

    Thomas P. Runarsson and Xin Yao. 2005. Search biases in constrained evolutionary optimization. IEEE Trans. Syst. Man. Cybern. Pt. C. Appl. Rev. 35, 2 (2005), 233–243

    2005

  52. [60]

    Shahabuddin

    Johara S. Shahabuddin. 1996. Structured Trust Region Algorithms for the Minimization of Nonlinear Functions . Ph. D. Dissertation. Cornell University, USA

    1996

  53. [61]

    Philippe L. Toint. 1983. Test Problems for Partially Separable Optimization and Results for the Routine PSPMIN . Technical Report. The University of Namur, Department of Mathematics

    1983

  54. [62]

    Philippe L. Toint. 1986. Global convergence of the partitioned BFGS algorithm for convex partially separable optimization. Math. Program. 36 (1986), 290–306

    1986

  55. [63]

    John von Neumann. 1950. Functional Operators. Vol. II. Princeton University Press, Princeton, NJ. Reprint of mimeographed lecture notes first distributed in 1933

    1950

  56. [64]

    Hongchao Zhang and Andrew R. Conn. 2012. On the local convergence of a derivative-free algorithm for least-squares minimization. Comput. Optim. Appl. 51, 2 (2012), 481–507

    2012

  57. [65]

    Zaikun Zhang. 2025. Scalable derivative-free optimization algorithms with low-dimensional subspace techniques. arXiv:2501.04536 [math.OC]

    2025 arXiv

  58. [66]

    Yu Zhu, Li Zhang, Rushi Lan, and Xiaonan Luo. 2019. Large-scale partially separable function optimization using cooperative coevolution and competition strategies. In 2019 Eleventh International Conference on Advanced Computational Intelligence (ICACI) . IEEE, Guilin, China, 1...

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.