Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers
read the original abstract
We present DFO-LS, a software package for derivative-free optimization (DFO) for nonlinear Least-Squares (LS) problems, with optional bound constraints. Inspired by the Gauss-Newton method, DFO-LS constructs simplified linear regression models for the residuals. DFO-LS allows flexible initialization for expensive problems, whereby it can begin making progress from as few as two objective evaluations. Numerical results show DFO-LS can gain reasonable progress on some medium-scale problems with fewer objective evaluations than is needed for one gradient evaluation. DFO-LS has improved robustness to noise, allowing sample averaging, the construction of regression-based models, and multiple restart strategies together with an auto-detection mechanism. Our extensive numerical experimentation shows that restarting the solver when stagnation is detected is a cheap and effective mechanism for achieving robustness, with superior performance over both sampling and regression techniques. We also present our package Py-BOBYQA, a Python implementation of BOBYQA (Powell, 2009), which also implements robustness to noise strategies. Our numerical experiments show that Py-BOBYQA is comparable to or better than existing general DFO solvers for noisy problems. In our comparisons, we introduce a new adaptive measure of accuracy for the data profiles of noisy functions that strikes a balance between measuring the true and the noisy objective improvement.
This paper has not been read by Pith yet.
Forward citations
Cited by 7 Pith papers
-
Cobaya: Code for Bayesian Analysis of hierarchical physical models
Cobaya is a modular Bayesian analysis code that exploits model interdependencies via automatic caching and a novel parameter-blocking algorithm to minimize sampling cost.
-
Negative neutrino mass or negative dark energy?
A sign-switching dark energy model (Λ_s CDM) recovers positive effective neutrino masses (0.055 ± 0.050 eV) consistent with oscillation data, unlike ΛCDM which prefers negative values (-0.075 eV), for DESI DR2 + CMB +...
-
Revisiting constraints on primordial vector modes and implications for sourced magnetic fields and observed $EB$ power spectrum
Updated constraints on neutrino-sustained primordial vector modes imply magnetic fields too weak to seed observations and cannot reproduce the EB power spectrum while satisfying parity-even limits.
-
SPT-3G D1: Axion Early Dark Energy with CMB experiments and DESI
CMB-only data give f_EDE < 0.07 at 95% CL with no strong AEDE signal, while CMB+DESI yields f_EDE = 0.055^{+0.024}_{-0.047} at 68% CL and lowers Hubble tension to 2.6 sigma.
-
Isogeometric Shape Optimization of Multi-Tapered Coaxial Baluns Simulated by an Integral Equation Method
Spline-based shape optimization via isogeometric integral equation simulation yields a multi-tapered coaxial balun with reduced scattering parameter magnitude across frequencies.
-
The Status of Gravitational Vector Perturbations with Recent CMB Data
Recent CMB datasets tighten 95% CL upper bounds on vector-mode amplitude r_v to 1.3e-4 (neutrino isocurvature), 6.8 (octupole), and 4.2 (sourced) at k=0.05 Mpc^-1, with no significant detection.
-
Probing cosmic anisotropy with galaxy clusters and supernovae
Analysis of galaxy cluster and supernova data reveals a ~2σ directional variation in the Hubble constant, robust across calibration methods and aligned with the CMB dipole.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.