pith. sign in

arxiv: 2606.20498 · v1 · pith:OD7EF3XInew · submitted 2026-06-18 · 🧮 math.OC

CLUSTER: Derivative-free optimization of smooth functions with parameter-change costs

Serena Landers , Sahil Pontula , Shiekh Zia Uddin , Sachin Vaidya , Marin Solja\v{c}i\'c , Steven G. Johnson This is my paper

Pith reviewed 2026-06-26 15:49 UTC · model grok-4.3

classification 🧮 math.OC
keywords derivative-free optimizationtrust-region methodsquadratic interpolationparameter change costslaboratory optimizationconvergence guaranteesCLUSTER algorithm
0
0 comments X

The pith

The CLUSTER algorithm improves derivative-free optimization performance by about 50% when there are costs to changing parameters.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces the CLUSTER algorithm for optimizing smooth functions where changing parameters incurs a cost, such as in robotic laboratory experiments. It modifies quadratic-interpolation trust-region methods to account for these costs at the coordinate level during step evaluation. This leads to substantial efficiency gains over standard approaches. The method also comes with a convergence guarantee for one of its variants.

Core claim

By using a coordinate-level update strategy for refining trust-region steps, the CLUSTER algorithm incorporates the cost of parameter changes into the optimization process for twice-differentiable objectives, resulting in improved performance on test problems and laboratory experiments while maintaining theoretical convergence properties.

What carries the argument

The CLUSTER (coordinate-level update strategy for trust-region step evaluation refinement) which adjusts how candidate steps are selected and refined in quadratic-interpolation based trust-region algorithms to minimize parameter cluster changes.

If this is right

  • CLUSTER variants achieve around 50% better performance on a variety of test problems including an optics laboratory experiment.
  • They greatly outperform Bayesian optimization and Nelder-Mead for laboratory optimization tasks.
  • CLUSTER-Conn has a convergence guarantee similar to the original Conn algorithm.
  • The approach is suitable for low-noise experiments with twice-differentiable objective functions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar cost-aware strategies could improve optimization in other domains like hyperparameter tuning where parameter changes have associated expenses.
  • Extending the method to handle noisy or non-smooth functions would increase its applicability to real-world experiments.
  • The framework might inspire cost models for other types of experimental constraints beyond parameter changes.

Load-bearing premise

The objective functions are twice-differentiable and the parameter-change cost model accurately reflects the real experimental setting.

What would settle it

A test on a twice-differentiable function where applying CLUSTER does not reduce the total optimization cost compared to the standard Powell-Conn method despite an accurate cost model.

read the original abstract

We introduce the CLUSTER algorithm (\textbf{c}oordinate-\textbf{l}evel \textbf{u}pdate \textbf{s}trategy for \textbf{t}rust-region step \textbf{e}valuation \textbf{r}efinement) for local derivative-free optimization problems where there is a cost to changing each parameter (or clusters of parameters). For example, this type of cost model is appropriate for optimizing robot-controlled laboratory experiments, in which a robot may incur a separate motion for each parameter cluster to be adjusted. We build off of a class of quadratic-interpolation optimization algorithms by Powell and Conn that are known to perform well for twice-differentiable objectives (e.g. low-noise experiments), and show that the CLUSTER variants improve performance on a variety of test problems (including an optics laboratory experiment) by around 50$\%$, and greatly outperform common competing algorithms for laboratory optimization (Bayesian optimization and Nelder--Mead). We also adapt the convergence proof of the Conn algorithm to obtain a similar convergence guarantee for CLUSTER-Conn.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 3 minor

Summary. The manuscript introduces the CLUSTER algorithm (coordinate-level update strategy for trust-region step evaluation refinement) for local derivative-free optimization of twice-differentiable functions subject to costs for changing individual parameters or clusters of parameters. It extends Powell-Conn quadratic-interpolation trust-region methods, reports empirical performance gains of approximately 50% on a suite of test problems including an optics laboratory experiment, shows outperformance relative to Bayesian optimization and Nelder-Mead, and adapts the convergence analysis of the Conn algorithm to obtain an analogous guarantee for the CLUSTER-Conn variant.

Significance. If the reported performance gains hold under the stated assumptions and the proof adaptation is valid, the work supplies a practically motivated extension of established DFO methods together with both empirical validation on laboratory data and a convergence result. This combination is relevant to experimental optimization settings where parameter adjustments carry explicit costs.

minor comments (3)
  1. [Experiments] The abstract states an improvement 'by around 50%'; the main experimental section should explicitly define the performance metric (e.g., total cost or number of evaluations) and report the precise values or ranges achieved by each CLUSTER variant versus the baselines.
  2. The description of the parameter-change cost model would benefit from an explicit mathematical formulation (e.g., an equation defining the cost function) early in the methods to make the objective and the algorithm's handling of clusters fully precise.
  3. [Convergence analysis] In the convergence section, a short paragraph outlining the principal modifications made to the Conn proof (particularly how the cost model enters the trust-region acceptance and radius-update arguments) would improve readability for readers already familiar with the original result.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their positive evaluation of the manuscript, accurate summary of the CLUSTER algorithm and its contributions, and recommendation for minor revision. We are pleased that the practical motivation, empirical results on test problems and laboratory data, and adapted convergence guarantee were viewed as relevant to experimental optimization settings.

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper's core contributions consist of an algorithmic extension (CLUSTER) of the established Powell-Conn quadratic-interpolation DFO framework, empirical performance comparisons on test problems and a laboratory experiment, and an adaptation of an external convergence proof from the Conn algorithm. No load-bearing derivation step reduces to a self-definition, a fitted parameter renamed as a prediction, or a self-citation chain; the twice-differentiability assumption and cost model are stated explicitly as prerequisites rather than derived internally. The adaptation of the Conn proof is presented as building on prior external work, not as an internal self-referential construction. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Ledger extracted from abstract only; full paper may introduce additional parameters or assumptions.

axioms (1)
  • domain assumption Objective functions are twice-differentiable.
    Method builds directly on Powell-Conn quadratic-interpolation algorithms known to perform well for twice-differentiable objectives.

pith-pipeline@v0.9.1-grok · 5736 in / 1131 out tokens · 22954 ms · 2026-06-26T15:49:13.744660+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 32 canonical work pages

  1. [1]

    , month = may, year =

    Larson, Jeffrey and Menickelly, Matt and Wild, Stefan M. , month = may, year =. Derivative-free optimization methods , volume =. doi:10.1017/S0962492919000060 , journal =

    work page doi:10.1017/s0962492919000060

  2. [2]

    Journal of Global Optimization , author =

    Derivative-free optimization: a review of algorithms and comparison of software implementations , volume =. Journal of Global Optimization , author =. 2013 , pages =. doi:10.1007/s10898-012-9951-y , number =

    work page doi:10.1007/s10898-012-9951-y 2013

  3. [3]

    and Scheinberg, Katya and Vicente, Luis N

    Conn, Andrew R. and Scheinberg, Katya and Vicente, Luis N. , month = jan, year =. Introduction to. doi:10.1137/1.9780898718768 , publisher =

    work page doi:10.1137/1.9780898718768

  4. [4]

    Online convex optimization with switching costs: algorithms and performance , copyright =

    Liu, Qingsong and Li, Zhuoran and. Online convex optimization with switching costs: algorithms and performance , copyright =. 2022 , pages =. doi:10.23919/WiOpt56218.2022.9930570 , booktitle =

    work page doi:10.23919/wiopt56218.2022.9930570 2022

  5. [5]

    Cambridge NA Report NA2009/06, University of Cambridge, Cambridge , author =

    The. Cambridge NA Report NA2009/06, University of Cambridge, Cambridge , author =. 2009 , pages =

    2009

  6. [6]

    Powell, M. J. D. , editor =. The. 2006 , pages =. doi:10.1007/0-387-30065-1_16 , booktitle =

    work page doi:10.1007/0-387-30065-1_16 2006

  7. [7]

    2002 , pages =

    Mathematical Programming , author =. 2002 , pages =. doi:10.1007/s101070100290 , number =

    work page doi:10.1007/s101070100290 2002

  8. [8]

    Mathematical Programming , author =

    Least. Mathematical Programming , author =. doi:10.1007/s10107-003-0490-7 , number =

    work page doi:10.1007/s10107-003-0490-7

  9. [9]

    and Toint, Philippe L

    Conn, Andrew R. and Toint, Philippe L. , editor =. An algorithm using quadratic interpolation for unconstrained derivative free optimization , isbn =. 1996 , pages =. doi:10.1007/978-1-4899-0289-4_3 , booktitle =

    work page doi:10.1007/978-1-4899-0289-4_3 1996

  10. [10]

    Proceedings of machine learning research , year=

    Bayesian optimization for modular black-box systems with switching costs , author=. Proceedings of machine learning research , year=

  11. [11]

    , howpublished =

    Zhang, Z. , howpublished =

  12. [12]

    ACM SIGMETRICS Performance Evaluation Review , author =

    Online optimization with switching cost , volume =. ACM SIGMETRICS Performance Evaluation Review , author =. 2012 , pages =. doi:10.1145/2425248.2425275 , number =

    work page doi:10.1145/2425248.2425275 2012

  13. [13]

    and Huang, Luke and Englund, Dirk R

    Uddin, Shiekh Zia and Vaidya, Sachin and Choudhary, Shrish and Salib, Raafat K. and Huang, Luke and Englund, Dirk R. and Soljačić, Marin , year =. Collaborative robotics for free-space optics , doi =

  14. [14]

    and Huang, Luke and Englund, Dirk R

    Uddin, Shiekh Zia and Vaidya, Sachin and Choudhary, Shrish and Chen, Zhuo and Salib, Raafat K. and Huang, Luke and Englund, Dirk R. and Soljačić, Marin , year =. doi:10.48550/ARXIV.2505.17985 , publisher =

    work page doi:10.48550/arxiv.2505.17985

  15. [15]

    Nature , author =

    Autonomous mobile robots for exploratory synthetic chemistry , volume =. Nature , author =. 2024 , pages =. doi:10.1038/s41586-024-08173-7 , number =

    work page doi:10.1038/s41586-024-08173-7 2024

  16. [16]

    Nature , author =

    A mobile robotic chemist , volume =. Nature , author =. 2020 , pages =. doi:10.1038/s41586-020-2442-2 , number =

    work page doi:10.1038/s41586-020-2442-2 2020

  17. [17]

    Nature , author =

    Cloud labs: where robots do the research , volume =. Nature , author =. 2022 , pages =. doi:10.1038/d41586-022-01618-x , number =

    work page doi:10.1038/d41586-022-01618-x 2022

  18. [18]

    Schmid, Sterling G

    Self-driving laboratories for chemistry and materials science , volume =. Chemical Reviews , author =. 2024 , pages =. doi:10.1021/acs.chemrev.4c00055 , number =

    work page doi:10.1021/acs.chemrev.4c00055 2024

  19. [19]

    Bandits with switching costs:

    Dekel, Ofer and Ding, Jian and Koren, Tomer and Peres, Yuval , month = may, year =. Bandits with switching costs:. doi:10.1145/2591796.2591868 , booktitle =

    work page doi:10.1145/2591796.2591868

  20. [20]

    Cost-aware

    Lee, Eric Hans and Perrone, Valerio and Archambeau, Cedric and Seeger, Matthias , month = mar, year =. Cost-aware. doi:10.48550/arXiv.2003.10870 , publisher =

    work page doi:10.48550/arxiv.2003.10870 2003

  21. [21]

    ACM Transactions on Mathematical Software , author =

    Testing unconstrained optimization software , volume =. ACM Transactions on Mathematical Software , author =. 1981 , pages =. doi:10.1145/355934.355936 , number =

    work page doi:10.1145/355934.355936 1981

  22. [22]

    SIAM Journal on Optimization , author =

    Global convergence of general derivative-free trust-region algorithms to first- and second-order critical points , volume =. SIAM Journal on Optimization , author =. 2009 , pages =. doi:10.1137/060673424 , number =

    work page doi:10.1137/060673424 2009

  23. [23]

    Nature Communications , author =

    In-situ physical adjoint computing in multiple-scattering electromagnetic environments for wave control , volume =. Nature Communications , author =. 2025 , pages =. doi:10.1038/s41467-025-66385-5 , number =

    work page doi:10.1038/s41467-025-66385-5 2025

  24. [24]

    , year =

    Ragonneau, Tom M. , year =. Model-based derivative-free optimization methods and software , copyright =. doi:10.48550/ARXIV.2210.12018 , publisher =

    work page doi:10.48550/arxiv.2210.12018

  25. [25]

    Multi-stage

    Torresi, Luca and Friederich, Pascal , month = dec, year =. Multi-stage. doi:10.48550/arXiv.2512.15483 , publisher =

    work page doi:10.48550/arxiv.2512.15483

  26. [26]

    Adaptation of the

    Deng, Geng and Ferris, Michael , month = dec, year =. Adaptation of the. doi:10.1109/WSC.2006.323088 , booktitle =

    work page doi:10.1109/wsc.2006.323088 2006

  27. [27]

    Johnson , year =

    Steven G. Johnson , year =. The

  28. [28]

    J. A. Nelder and R. Mead , title =. doi:10.1093/comjnl/7.4.308 , year =

    work page doi:10.1093/comjnl/7.4.308

  29. [29]

    Fernando Nogueira , title =

  30. [30]

    Analytica Chimica Acta , author =

    Sequential simplex optimization in a constrained simplex mixture space in liquid chromatography , volume =. Analytica Chimica Acta , author =. 1992 , pages =. doi:10.1016/0003-2670(92)80096-P , language =

    work page doi:10.1016/0003-2670(92)80096-p 1992

  31. [31]

    Bayesian reaction optimization as a tool for chemical synthesis

    Bayesian reaction optimization as a tool for chemical synthesis , volume =. Nature , author =. 2021 , pages =. doi:10.1038/s41586-021-03213-y , language =

    work page doi:10.1038/s41586-021-03213-y 2021

  32. [32]

    Derivative-free optimization for chemical product design , volume =

    Sun, Yijia and Sahinidis, Nikolaos V and Sundaram, Anantha and Cheon, Myun-Seok , month = mar, year =. Derivative-free optimization for chemical product design , volume =. doi:10.1016/j.coche.2019.11.006 , journal =

    work page doi:10.1016/j.coche.2019.11.006 2019

  33. [33]

    Microchemical Journal , author =

    Simplex optimization:. Microchemical Journal , author =. 2016 , pages =

    2016

  34. [34]

    Computational Optimization and Applications , author =

    On the convergence of trust region algorithms for unconstrained minimization without derivatives , volume =. Computational Optimization and Applications , author =. 2012 , pages =. doi:10.1007/s10589-012-9483-x , number =

    work page doi:10.1007/s10589-012-9483-x 2012

  35. [35]

    SIAM Journal on Optimization , author =

    Benchmarking derivative-free optimization algorithms , volume =. SIAM Journal on Optimization , author =. 2009 , pages =. doi:10.1137/080724083 , number =

    work page doi:10.1137/080724083 2009

  36. [36]

    ACM Transactions on Mathematical Software , author =

    Algorithm 856:. ACM Transactions on Mathematical Software , author =. 2006 , pages =. doi:10.1145/1163641.1163647 , language =

    work page doi:10.1145/1163641.1163647 2006

  37. [37]

    A framework for closed-loop robotic assembly, alignment and self-recovery of precision optical systems , doi =

    Choi, Seou and Vaidya, Sachin and Silva, Caio and Uddin, Shiekh Zia and Shuvo, Sajib Biswas and Choudhary, Shrish and Soljačić, Marin , month = mar, year =. A framework for closed-loop robotic assembly, alignment and self-recovery of precision optical systems , doi =

  38. [38]

    Journal of Synchrotron Radiation , author =

    A general. Journal of Synchrotron Radiation , author =. 2024 , pages =. doi:10.1107/S1600577524008993 , number =

    work page doi:10.1107/s1600577524008993 2024

  39. [39]

    ACM Transactions on Mathematical Software , author =

    Algorithm 1027:. ACM Transactions on Mathematical Software , author =. 2022 , pages =. doi:10.1145/3544489 , number =

    work page doi:10.1145/3544489 2022

  40. [40]

    Nonlinear

    Bertsekas, Dimitri , year =. Nonlinear

  41. [41]

    American Journal of Science , author =

    On the relative motion of the. American Journal of Science , author =. 1887 , pages =. doi:10.2475/ajs.s3-34.203.333 , language =

    work page doi:10.2475/ajs.s3-34.203.333