CatMADS: Mesh Adaptive Direct Search for constrained blackbox optimization with categorical variables

Charles Audet; Christophe Tribes; Edward Hall\'e-Hannan; S\'ebastien Le Digabel; Youssef Diouane

REVIEW 3 major objections 4 minor 48 references

CatMADS: Mesh Adaptive Direct Search for constrained blackbox optimization with categorical variables

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CatMADS extends MADS to constrained blackbox optimization with categorical variables and proves Clarke stationarity at refined points.

desk verdict A real extension of MADS to categorical variables with a useful hierarchy of local optimality, but the implemented direction generation does not satisfy the density assumption behind the main stationarity theorem, so the abstract overclaims. read the letter →

arxiv 2506.06937 v2 pith:FMXAY2T6 submitted 2025-06-07 math.OC

Charles Audet , Youssef Diouane , Edward Hallé-Hannan , Sébastien Le Digabel , Christophe Tribes This is my paper · ORCID

classification math.OC MSC 90C5690C30

keywords mixed-variableoptimizationcategoricalvariablesblackboxderivative-freemeshadaptivedirectsearchprogressivebarrierClarkestationaritydataprofiles

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces CatMADS, an extension of the Mesh Adaptive Direct Search algorithm to constrained blackbox problems whose variables are categorical, integer, and continuous. Its central claim is that a distance-induced categorical neighborhood, a granular mesh for quantitative variables, and the progressive barrier strategy can be combined while preserving MADS-style convergence guarantees. The main theoretical result is that, provided the poll directions are dense in the unit sphere, every refined point satisfies a first-order necessary optimality condition in the continuous variables: the generalized directional derivative is nonnegative in every hypertangent direction. The paper also defines four types of mixed-variable local minima of increasing strength and shows that a dedicated variant reaches the stronger types when the extended poll is active. A prototype solves more instances than three state-of-the-art comparison solvers on the paper's 32-problem Cat-Suite, so the method is presented as both theoretically grounded and practically effective.

What carries the argument

The load-bearing mechanism is the two-part poll. A categorical poll selects the $m+1$ nearest categorical components with respect to a distance-induced neighborhood $\mathcal{N}(x^{\mathrm{cat}}; m)$, and a quantitative poll generates candidate points on a granular mesh using direction sets scaled by variable-wise frame and step sizes, with Householder transformations producing random directions. Around this, the progressive barrier maintains feasible and infeasible incumbents, and the optional extended poll runs quantitative polls centered at promising categorical neighbors. In the convergence proof, the key object is the refining subsequence: unsuccessful iterations whose continuous step sizes go to zero, integer step sizes go to one, and distance-induced neighborhoods stabilize; the refined point together with dense refining directions is what lets the paper connect poll failure to nonnegativity of the generalized directional derivative along hypertangent directions.

What would settle it

Record every normalized quantitative poll direction on unsuccessful iterations for a smooth constrained problem with at least two continuous variables; if, in a refining subsequence, those directions all lie in a finite union of proper subspaces of $\mathbb{R}^{n_{\mathrm{cont}}}$, the density hypothesis fails. A Lipschitz objective could then be chosen so that some refined point has a hypertangent direction with negative generalized directional derivative, contradicting Theorem 3.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that categorical variables can be incorporated into MADS not through user-defined neighborhoods but through distances: for a number $m$ of neighbors, the categorical poll evaluates the $m$ closest categorical components to the incumbent under a learned or user-supplied distance $d_{\mathrm{cat}}$, while the quantitative poll keeps the granular-mesh machinery for integers and continuous variables. The convergence analysis shows that a refining subsequence exists under compactness, and Theorem 3 asserts that for a refined point $x_*$, if the refining directions are dense in the unit sphere of $\mathbb{R}^{n_{\mathrm{cont}}}$ and the objective is Lipschitz near $x_{*}^{\mathrm{cont}}$, then $f_*^\circ(x_{*}^{\mathrm{cont}}; v) \ge 0$ for every hypertangent direction $v$; for infeasible refined points the same statement holds for the constraint-violation function $h$. This is a Clarke-stationarity guarantee in the continuous variables for fixed categorical and integer components. A hierarchy of four mixed-variable local minima, from $[\mathrm{cat},\mathrm{int},\mathrm{cont}]$ to $[\mathrm{cat}{-}\mathrm{cont},\mathrm{cat}{-}\mathrm{int}]$, lets the user choose how much joint interaction between variable types is certified, at a price in evaluations controlled by the extended-poll trigger $\xi$.

Load-bearing premise

The proof of Theorem 3 assumes that the set of refining directions is dense in the unit sphere of the continuous variables, and it does not prove that the random direction generator delivers that density; if the directions fall into a fixed finite set, the Clarke-stationarity conclusion no longer follows.

Editorial extensions

If this is right

If Theorem 3 holds, CatMADS inherits the MADS guarantee that limit points are not improvable by infinitesimal continuous moves, even when categorical and integer variables are present and constraints are active.
The four local-minimum types give users a dial: setting the extended-poll trigger $\xi<0$ certifies the weakest separated minimum, while $\xi=\infty$ certifies the strongest joint categorical-continuous and categorical-integer minimum at higher evaluation cost.
Because the progressive barrier is integrated, the method can start from an infeasible point and still produce convergence statements for both feasible and infeasible refined points.
The experimental section reports that CatMADS reaches the tight reduction target on roughly 70% of unconstrained instances at tolerance $\tau=10^{-5}$, against about 30% for the strongest of the three comparison solvers, so the method is claimed to be competitive as well as convergent.
The distance construction by cross-validated inverse-distance weighting on the initial design turns the categorical poll into a problem-specific choice, which the paper argues is what makes neighbor selection meaningful for optimization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension is to prove whether the Householder-based random direction generator actually produces a dense set of refining directions almost surely; if it does, Theorem 3's hypothesis is generic rather than a separate assumption.
The one-shot distance learned from the initial design could be updated as evaluations accumulate; the paper does not analyze whether online updates would preserve the refining-subsequence argument, but the distance-induced-neighborhood definition would allow it.
The local-minimum hierarchy suggests a multi-stage procedure in which coarse categorical interactions are explored first and finer joint neighborhoods are certified later; this is not proposed in the paper.
The 32-problem Cat-Suite could be reused as a standard benchmark, and the data-profile methodology would transfer directly to future solvers; this follows from the paper's choice to make scripts and data public.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: CatMADS is a legitimate extension of MADS to categorical variables. The distance-induced neighborhood idea is simple and useful, and the four-level hierarchy of mixed-variable local minima is a nice way to make the convergence-strength/cost tradeoff explicit. The main thing you should know is that the abstract overclaims: the Clarke-stationarity result is conditional on a dense-refining-directions assumption that the paper never verifies for its own Householder-based prototype, and the stress-test note gives good reason to think the assumption actually fails when integer variables are present.

What is new and good: distance-induced neighborhoods generalize MV-MADS user-defined neighborhoods; the granular mesh treats integer variables properly instead of lumping them with categoricals; the progressive barrier gives constrained convergence guarantees; and the paper ships a public prototype, a 32-problem Cat-Suite, and data profiles. Theorems 1 and 4 are routine but correct, Theorem 2 is a clean adaptation, and Theorem 3 is fine as a conditional statement. Building on G-MADS and the progressive barrier is not a problem here, since those are established external results.

The soft spots are proportionate. First, the dense-refining-directions assumption. In the described instance, poll directions come from columns of I-2vv^T, and a refining direction requires d_int=0. For any continuous distribution on v, the set of v for which a Householder column has zeros in all integer coordinates is a measure-zero algebraic subset; over countably many iterations, almost surely no such direction appears. So for n_int>0 the set of refining directions is likely empty, not dense. The paper needs either a different direction-generation mechanism that provably includes directions leaving integer variables fixed, or a proof that the Householder set contains a dense refining subset. Without that, Theorem 3 does not cover the prototype as described, and the abstract's unconditional "ensures Clarke stationarity" is not supported.

Second, the experiments: five seeds with no uncertainty intervals, a self-authored benchmark, and no direct comparison to MV-MADS, the closest predecessor. The data profiles show a large gap and the code is public, so the empirical claim is plausible, but "strongly dominates" is stronger than the evidence supports. The cross-validated distance fitting is heuristic, which is fine, but it should be kept clearly separate from the theoretical claims.

Who this is for: anyone working on derivative-free or mixed-variable blackbox optimization. It is a real step forward and deserves a serious referee. I would send it out, but with a request to fix the direction-density gap and temper the abstract. If the authors can patch the density argument, this could be a solid journal paper.

Referee Report

3 major / 4 minor

Summary. The paper proposes CatMADS, an extension of Mesh Adaptive Direct Search for constrained blackbox optimization with categorical, integer, and continuous variables. Categorical variables are handled through distance-induced neighborhoods; integer and continuous variables are handled through the granular mesh of G-MADS. The algorithm integrates the progressive barrier for constraints, an optional extended poll, and several user-tunable mechanisms. The convergence analysis introduces four types of mixed-variable local minima, proves the existence of refining subsequences (Theorem 1), establishes nonnegativity of generalized directional derivatives along refining hypertangent directions (Theorem 2), derives Clarke stationarity of refined points under a density assumption on refining directions (Theorem 3), and gives an order relation for refined neighbor points (Theorem 4). A prototype instance using cross-validated one-hot encoded categorical distances is compared on a new 32-problem Cat-Suite against Pymoo, Optuna, and BoTorch, with data profiles reported as showing CatMADS performing best.

Significance. If the convergence claims are made fully supported, CatMADS is a valuable contribution: it is one of the few direct-search frameworks with convergence guarantees for mixed-variable problems containing categorical variables, and it extends MV-MADS with the granular mesh, the progressive barrier, and an explicit hierarchy of local-minima notions. The paper builds on peer-reviewed G-MADS and progressive-barrier results, and the public prototype and Cat-Suite benchmark are useful resources. However, the central stationarity guarantee currently depends on an unverified hypothesis about the implemented direction-generation mechanism, and the empirical claims rest on a small number of seeds without uncertainty quantification. The conditional theorems themselves appear to follow standard MADS-style arguments.

major comments (3)

[Section 2.2, Definition 15, Theorem 3] The dense-refining-directions hypothesis of Theorem 3 is not satisfied by the Householder direction generation described in Section 2.2 when n_int > 0. For a random vector v(k) in R^{n_qnt}, a column of I - 2 v(k) v(k)^T has zero entries in all integer coordinates only on a measure-zero event: either one continuous coordinate of v(k) is zero, or all integer coordinates of v(k) are zero (up to the diagonal condition for an integer column). Over countably many iterations, almost surely no poll direction with d_int(k) = 0 is ever generated, so the set of refining directions is empty rather than dense. Consequently the hypothesis of Theorem 3 fails for the implemented instance, and the abstract's unconditional statement that CatMADS 'ensures Clarke stationarity' is not supported. Please modify the direction generator so that directions with d_int = 0 occur and are dense in the continuous unit sphere (for example, generate Householder vectors in the continuous subspace or explicitly include continuous coordinate directions), prove that density, or state the stationarity result strictly as conditional and revise the abstract and conclusions accordingly.
[Section 4.3.2] The empirical claim that CatMADS 'strongly dominates' Pymoo, Optuna, and BoTorch is based on data profiles with five random seeds per instance and no confidence intervals, error bars, or statistical testing. The largest reported margin, roughly 70% versus 30% at tau = 10^-5 on unconstrained problems, could be substantially affected by seed variability. Since Cat-Suite is introduced by the same authors, the possibility of favorable problem construction cannot be excluded. Please report uncertainty estimates, increase the number of seeds, or provide a statistical comparison, and temper the wording to the specific benchmark used.
[Section 3.4] The final note of Section 3.4 states that, if the set of refining directions is dense at z_cont^*, then Theorem 3 'can be adapted' to the endpoint subsequence {z(k)}. This is an unproven assertion that extends the main stationarity result to the extended-poll setting, and it is load-bearing for the claimed strengthening to [cat-cont, int] and [cat-cont, cat-int] minima. Please provide a formal proof or clearly mark this as a conjecture with the precise assumptions needed.

minor comments (4)

[Algorithm 2] In Step 2 of Algorithm 2, the expression 'P^int_inf' appears to be a typo and should likely read 'P^qnt_inf' for consistency with the quantitative poll notation used throughout the paper.
[Section 4.1 and Section 4.3.1] There are several grammatical and punctuation issues, such as 'Although xi = infinity is associated to the stronger [cat-cont,cat-int] local minimum its costs degrades performance' in Section 4.3.1; please proofread these passages.
[Section 3.4] The informal phrasing 'This covers the [cat-cont, int] minimum' and 'the same conclusion is drawn' should be replaced with precise mathematical statements, including the explicit relationship y_int = x_int^* for neighbor points and the exact hypotheses on the extended-poll trigger parameter xi.
[Section 1.1] The notation 'R = R cup {+infinity}' is confusing because the same symbol R is used for the real line and the extended-value codomain; please use distinct notation, for example R_bar, for the extended real line.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the theoretical claims are conditional theorems built on published G-MADS and progressive-barrier results, and the fitted categorical distances enter only the empirical instance, not the convergence proofs.

full rationale

The paper's derivation chain does not reduce any result to its own inputs. The categorical distance is fitted by cross-validation in Algorithm 2, but the convergence theorems in Section 3 are stated for an arbitrary fixed distance d_cat and never use the fitted weights, so no fitted parameter is renamed as a prediction. The proofs of Theorems 1 and 3 inherit the mesh/step-size convergence from G-MADS ([13], Theorems 3.1 and 3.2) and the continuous progressive-barrier optimality conditions from [7]; these are prior peer-reviewed results by overlapping authors but they are external, parameter-free theorems with stated assumptions that do not include CatMADS's target stationarity claim, so the self-citations are not load-bearing in a circular sense. The new definitions of local minima are taxonomy and are not used to define the algorithm's output into satisfaction of the theorems. A genuine weakness exists but it is not circularity: Theorem 3 assumes that the set of refining directions is dense in the unit sphere, and Section 3.3 does not prove that the Householder direction generator in Section 2.2 produces an infinite subsequence of poll directions with d_int(k)=0 (as required by Definition 15) when integer variables are present. This missing verification means the abstract's unconditional wording 'ensures Clarke stationarity' is stronger than the established conditional theorem, but the assumption is an explicit hypothesis rather than a disguised restatement of the conclusion, so it does not raise the circularity score.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central convergence proofs import standard MADS machinery, namely the granular mesh, the progressive barrier, and the hypertangent cone, from the authors' prior work and from nonsmooth analysis. The only genuinely new postulate is the dense refining directions assumption, plus standard Lipschitz and compactness hypotheses. The categorical-distance weights and the algorithmic hyperparameters xi, m, and DoE size are fitted or chosen empirically for the experimental instance, but the theorems do not depend on their specific values.

free parameters (4)

xi (extended poll trigger) = 0.05
Selected in Section 4.3.1 by comparing data profiles on Cat-Suite; controls which local-minimum type the convergence condition targets.
m, number of categorical neighbors = max(2, sqrt(|X_cat|))
Formula for the constant number of categorical neighbors, set in Section 4.1 after preliminary testing of alternatives; balances poll cost and local optimality.
DoE budget fraction = 20% of 250n evaluations
Initial design-of-experiment size chosen empirically in Section 4.1; used to fit the categorical distance.
categorical distance weights theta = not reported
Per-category weights in the one-hot encoded weighted Euclidean distance, fit per problem by minimizing RMSE of an inverse-distance weighting interpolation under 3-fold cross-validation in Section 4.1. These weights define the categorical neighborhoods used in polling.

assumptions (5)

ad hoc to paper The set of refining directions is dense in the unit sphere of R^{n_cont}.
Assumed in Theorem 3 and used to turn the inequality in Theorem 2 into a full Clarke-stationarity condition; not proved for the Householder-based G-MADS directions used by the prototype.
domain assumption All iterates generated by CatMADS lie in a compact set and the initial point has finite objective value.
Stated in Section 3 and used in Theorem 1 to construct a refining subsequence; excludes unbounded blackbox domains and infinite objective at start.
domain assumption The objective and constraint restrictions f* and h* are locally Lipschitz in the continuous variables at refined points.
Required to define the Clarke generalized derivative in Definition 16 and to apply Theorems 2 and 3; blackbox responses in applications may be discontinuous or noisy.
standard math The G-MADS frame and mesh size parameters decrease on unsuccessful iterations, with step sizes tending to zero faster than frame sizes.
Inherited from Audet, Le Digabel and Tribes [13]; used in Definition 13 and the proof of Theorem 2 to ensure the poll displacement vectors tend to zero.
domain assumption The progressive barrier convergence results for continuous MADS from [7] extend to the mixed-variable setting with finite categorical and integer polls.
Used in the proof of Theorem 3 to handle feasible and infeasible refined points; the paper invokes Corollary 3.4, Theorem 3.5, and Corollary 3.6 of [7] without re-proving them.

how reviews work

0 comments

Cite this review

Pith. "Pith review of CatMADS: Mesh Adaptive Direct Search for constrained blackbox optimization with categorical variables." pith.science (2026). https://pith.science/paper/FMXAY2T6

@misc{pith2026250606937,
  author       = {Pith},
  title        = {Pith review of: CatMADS: Mesh Adaptive Direct Search for constrained blackbox optimization with categorical variables},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FMXAY2T6}},
  note         = {Machine review of arXiv:2506.06937}
}

read the original abstract

Solving optimization problems in which functions are blackboxes and variables involve different types poses significant theoretical and algorithmic challenges. Nevertheless, such settings frequently occur in simulation-based engineering design and machine learning. This paper extends the Mesh Adaptive Direct Search (MADS) algorithm to address mixed-variable problems with categorical, integer and continuous variables. MADS is a robust derivative-free optimization framework with a well-established convergence analysis for constrained quantitative problems. CatMADS generalizes MADS by incorporating categorical variables through distance-induced neighborhoods. A detailed convergence analysis of CatMADS is provided, with flexible choices balancing computational cost and local optimality strength. Four types of mixed-variable local minima are introduced, corresponding to progressively stronger notions of local optimality. CatMADS integrates the progressive barrier strategy for handling constraints, and ensures Clarke stationarity. An instance of \catmads employs cross-validation to construct problem-specific categorical distances. This instance is compared to state-of-the-art solvers on 32 mixed-variable problems, half of which are constrained. Data profiles show that CatMADS achieves the best results, demonstrating that the framework is empirically efficient in addition to having strong theoretical foundations.

Figures

Figures reproduced from arXiv: 2506.06937 by the authors.

**Figure 1.** Three consecutive polls of MADS converging to a local minimum x∗ ∈ R 2 . 1.4 Related work The recent literature on mixed-variable blackbox problems mostly comes from BO with GPs, focusing on similarity measures, referred to as kernels, that characterize GPs. For continuous and integer variables, squared-exponential kernels can be used straightforwardly [23]. However, categorical variables require additional work. … view at source ↗

**Figure 2.** The RG example for the extended poll with the mesh in gray. [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗

**Figure 3.** A local minimum ⋆ = (Red, 2, x cont ⋆ ) for the RBG example, where x cont ∈ X cont ⊂ R 2 , m = 1 and η = 1. The local minimum, as described in Definition 9, is not covered by CatMADS. In practice, the evaluations required for reaching optimality could be substantial. To balance computational costs and solution quality, three weaker types of local minima are proposed. The new types of local minima are named with a sp… view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Three new types of local minima at ⋆ = (Red, 2, x cont ⋆ ) for the RBG example, where x cont ∈ X cont ⊂ R 2 , m = 1 and η = 1. Definition 11 ([cat − cont, int] local minimum). A point x = [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

**Figure 5.** Figure 5: Illustrative examples for the convergence analysis. [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]

**Figure 6.** Figure 6: Data profiles for different values of ξ. more efficient strategy than the extreme barrier [6]. The convergence analysis shows that CatMADS has convergence guarantees under reasonable assumptions. A prototype implementation of CatMADS is compared to state-of-the-art sol…

**Figure 7.** Figure 7: Comparison results on the test problems. [PITH_FULL_IMAGE:figures/full_fig_p025_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 48 canonical work pages

[1]

Abramson, C

M.A. Abramson, C. Audet, J.W. Chrissis, and J.G. Walston. Mesh Adaptive Direct Search Algorithms for Mixed Variable Optimization.Optimization Letters, 3(1):35–47, 2009

work page 2009
[2]

Akiba, S

T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama. Optuna: A Next-generation Hyper- parameter Optimization Framework. InProceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery, 2019

work page 2019
[3]

Ament, S

S. Ament, S. Daulton, D. Eriksson, M. Balandat, and E. Bakshy. Unexpected Improvements to Expected Improvement for Bayesian Optimization.Advances in Neural Information Processing Systems, 36:20577–20612, 2023

work page 2023
[4]

Andrés-Thió, C

N. Andrés-Thió, C. Audet, M. Diago, A.E. Gheribi, S. Le Digabel, X. Lebeuf, M. Lemyre Gar- neau, and C. Tribes.solar: A solar thermal power plant simulator for blackbox optimization benchmarking. Technical Report G-2024-37, Les cahiers du GERAD, 2025. To appear in Optimization and Engineering

work page 2024
[5]

Audet and J.E

C. Audet and J.E. Dennis, Jr. Pattern Search Algorithms for Mixed Variable Programming. SIAM Journal on Optimization, 11(3):573–594, 2001

work page 2001
[6]

Audet and J.E

C. Audet and J.E. Dennis, Jr. Mesh Adaptive Direct Search Algorithms for Constrained Optimization.SIAM Journal on Optimization, 17(1):188–217, 2006

work page 2006
[7]

Audet and J.E

C. Audet and J.E. Dennis, Jr. A Progressive Barrier for Derivative-Free Nonlinear Program- ming.SIAM Journal on Optimization, 20(1):445–472, 2009

work page 2009
[8]

Audet, E

C. Audet, E. Hallé-Hannan, and S. Le Digabel. A General Mathematical Framework for Con- strained Mixed-variable Blackbox Optimization Problems with Meta and Categorical Variables. Operations Research Forum, 4(12), 2023

work page 2023

Show all 48 references

[9]

Audet and W

C. Audet and W. Hare.Derivative-Free and Blackbox Optimization. Springer Series in Opera- tions Research and Financial Engineering. Springer, Cham, Switzerland, 2017

2017
[10]

Audet, W

C. Audet, W. Hare, and C. Tribes. Benchmarking constrained, multi-objective and surrogate- assisted derivative-free optimization methods. Technical Report G-2025-36, Les cahiers du GERAD, 2025

2025
[11]

Audet, S

C. Audet, S. Le Digabel, V. Rochon Montplaisir, and C. Tribes. Algorithm 1027: NOMAD ver- sion 4: Nonlinear optimization with the MADS algorithm.ACM Transactions on Mathematical Software, 48(3):35:1–35:22, 2022

2022
[12]

Audet, S

C. Audet, S. Le Digabel, and C. Tribes. Dynamic scaling in the mesh adaptive direct search algorithm for blackbox optimization.Optimization and Engineering, 17(2):333–358, 2016

2016
[13]

Audet, S

C. Audet, S. Le Digabel, and C. Tribes. The Mesh Adaptive Direct Search Algorithm for Granular and Discrete Variables.SIAM Journal on Optimization, 29(2):1164–1189, 2019

2019
[14]

Balandat, B

M. Balandat, B. Karrer, D.R. Jiang, S. Daulton, B. Letham, A.G. Wilson, and E. Bakshy. BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization. InAdvances in Neural Information Processing Systems 33. Curran Associates Inc., 2020. 26

2020
[15]

Blank and K

J. Blank and K. Deb. Pymoo: Multi-Objective Optimization in Python.IEEE Access, 8:89497– 89509, 2020

2020
[16]

Bussemaker, P.D

J.H. Bussemaker, P.D. Ciampa, T. De Smedt, B. Nagel, and G. La Rocca. System Architecture Optimization: An Open Source Multidisciplinary Aircraft Jet Engine Architecting Problem. InAIAA AVIATION 2021 Forum. American Institute of Aeronautics and Astronautics, 2021

2021
[17]

Choi and C.T

T.D. Choi and C.T. Kelley. Superlinear convergence and implicit filtering.SIAM Journal on Optimization, 10(4):1149–1162, 2000

2000
[18]

Clarke.Optimization and Nonsmooth Analysis

F.H. Clarke.Optimization and Nonsmooth Analysis. John Wiley and Sons, New York, 1983. Reissued in 1990 by SIAM Publications, Philadelphia, as Vol. 5 in the series Classics in Applied Mathematics

1983
[19]

Conn and S

A.R. Conn and S. Le Digabel. Use of quadratic models with mesh-adaptive direct search for constrained black box optimization.Optimization Methods and Software, 28(1):139–158, 2013

2013
[20]

Crélot, C

A.-S. Crélot, C. Beauthier, D. Orban, C. Sainvitu, and A. Sartenaer. Combining Surrogate Strategies with MADS for Mixed-Variable Derivative-Free Optimization. Technical Report G-2017-70, Les cahiers du GERAD, 2017

2017
[21]

Cuesta-Ramirez, R

J. Cuesta-Ramirez, R. Le Riche, O. Roustant, G. Perrin, C. Durantin, and A. Gliere. A compar- ison of mixed-variables Bayesian optimization approaches.Advanced Modeling and Simulation in Engineering Sciences, 9(1):6, 2022

2022
[22]

K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: NSGA-II.IEEE Transactions on Evolutionary Computation, 6(2):182–197, 2002

2002
[23]

Garrido-Merchán and D

E.C. Garrido-Merchán and D. Hernández-Lobato. Dealing with categorical and integer-valued variables in Bayesian Optimization with Gaussian processes.Neurocomputing, 380:20–35, 2020

2020
[24]

Hallé-Hannan, C

E. Hallé-Hannan, C. Audet, Y. Diouane, S. Le Digabel, and P. Saves. A graph-structured distance for mixed-variable domains with meta variables. Technical Report G-2024-33, Les cahiers du GERAD, 2024

2024
[25]

Cat-Suite: Acollectionof optimization problems with categorical and quantitative variables for benchmarking

E.Hallé-Hannan, C.Audet, Y.Diouane, S.LeDigabel, andC.Tribes. Cat-Suite: Acollectionof optimization problems with categorical and quantitative variables for benchmarking. Technical Report G-2025-29, Les cahiers du GERAD, 2025

2025
[26]

Hastie, R

T. Hastie, R. Tibshirani, and J. Friedman.The Elements of Statistical Learning. Springer Series in Statistics. Springer New York Inc., New York, NY, USA, 2001

2001
[27]

Jahn.Introduction to the theory of nonlinear optimization

J. Jahn.Introduction to the theory of nonlinear optimization. Springer, third edition, 2007

2007
[28]

Jamil and X.-S

M. Jamil and X.-S. Yang. A literature survey of benchmark functions for global optimisa- tion problems.International Journal of Mathematical Modelling and Numerical Optimisation, 4(2):150–194, 2013

2013
[29]

Jina and P.V

Y. Jina and P.V. Kumar. Bayesian optimisation for efficient material discovery: a mini review. Nanoscale, 15(26):10975–10984, 2023

2023
[30]

Le Digabel

S. Le Digabel. Algorithm 909: NOMAD: Nonlinear Optimization with the MADS algorithm. ACM Transactions on Mathematical Software, 37(4):44:1–44:15, 2011. 27

2011
[31]

Le Digabel and S.M

S. Le Digabel and S.M. Wild. A taxonomy of constraints in black-box simulation-based opti- mization.Optimization and Engineering, 25(2):1125–1143, 2024

2024
[32]

Lukšan and J

L. Lukšan and J. Vlček. Test Problems for Nonsmooth Unconstrained and Linearly Constrained Optimization. Technical Report V-798, ICS AS CR, 2000

2000
[33]

J.R.R.A Martins and A.B. Lambe. Multidisciplinary design optimization: a survey of archi- tectures.AIAA journal, 51(9):2049–2075, 2013

2013
[34]

Moré and S.M

J.J. Moré and S.M. Wild. Benchmarking Derivative-Free Optimization Algorithms.SIAM Journal on Optimization, 20(1):172–191, 2009

2009
[35]

Munoz Zuniga and D

M. Munoz Zuniga and D. Sinoquet. Global optimization for mixed categorical-continuous variables based on Gaussian process models with a randomized categorical space exploration step.INFOR: Information Systems and Operational Research, 58(2):310–341, 2020

2020
[36]

Pelamatti, L

J. Pelamatti, L. Brevault, M. Balesdent, E.-G. Talbi, and Y. Guerin. Efficient global optimiza- tion of constrained mixed variable problems.Journal of Global Optimization, 73(3):583–613, 2019

2019
[37]

Prager and H

R.P. Prager and H. Trautmann. Exploratory Landscape Analysis for Mixed-Variable Problems. IEEE Transactions on Evolutionary Computation, 2024

2024
[38]

Qiand, H

P.Z.G. Qiand, H. Wu, and C.F.J. Wu. Gaussian process models for computer experiments with qualitative and quantitative factors.Technometrics, 50(3):383–396, 2008

2008
[39]

Rockafellar.Convex Analysis

R.T. Rockafellar.Convex Analysis. Princeton University Press, Princeton, NJ, 1970

1970
[40]

Roustant, E

O. Roustant, E. Padonou, Y. Deville, A. Clément, G. Perrin, J. Giorla, and H. Wynn. Group kernels for Gaussian process metamodels with categorical inputs.Uncertainty Quantification, 8(2):775–806, 2020

2020
[41]

Saves, N

P. Saves, N. Bartoli, Y. Diouane, T. Lefebvre, J. Morlier, C. David, E. Nguyen Van, and S. De- foort. Constrained Bayesian Optimization Over Mixed Categorical Variables, with Application to Aircraft Design. InAeroBest, 2021

2021
[42]

Saves, Y

P. Saves, Y. Diouane, N. Bartoli, T. Lefebvre, and J. Morlier. A mixed-categorical correlation kernel for Gaussian process.Neurocomputing, 550:126472, 2023

2023
[43]

Sridevi, Z

P. Sridevi, Z. Arefin, and S.I. Ahamed. An integrated machine learning and hyperparame- ter optimization framework for noninvasive creatinine estimation using photoplethysmography signals.Healthcare Analytics, 7:100395, 2025

2025
[44]

Stelmack, N

M. Stelmack, N. Nakashima, and S. Batill. Genetic algorithms for mixed discrete/continuous optimization in multidisciplinary design. In7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization. Aerospace Research Central, 1998

1998
[45]

E.-G. Talbi. Metaheuristics for variable-size mixed optimization problems: A unified taxonomy and survey.Swarm and Evolutionary Computation, 89:101642, 2024

2024
[46]

Torres, G

J.J. Torres, G. Nannicini, E. Traversi, and R.W. Calvo. A trust-region framework for derivative- free mixed-integer optimization.Mathematical Programming Computation, 16:369–422, 2024. 28

2024
[47]

F. Wang, H. Zhang, and A. Zhou. A particle swarm optimization algorithm for mixed-variable optimization problems.Swarm and Evolutionary Computation, 60:100808, 2021

2021
[48]

Zhang, S

Y. Zhang, S. Tao, W. Chen, and D.W. Apley. A Latent Variable Approach to Gaussian Process Modeling with Qualitative and Quantitative Factors.Technometrics, 62(3):291–302, 2020. 29

2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Abramson, C

M.A. Abramson, C. Audet, J.W. Chrissis, and J.G. Walston. Mesh Adaptive Direct Search Algorithms for Mixed Variable Optimization.Optimization Letters, 3(1):35–47, 2009

work page 2009

[2] [2]

Akiba, S

T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama. Optuna: A Next-generation Hyper- parameter Optimization Framework. InProceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery, 2019

work page 2019

[3] [3]

Ament, S

S. Ament, S. Daulton, D. Eriksson, M. Balandat, and E. Bakshy. Unexpected Improvements to Expected Improvement for Bayesian Optimization.Advances in Neural Information Processing Systems, 36:20577–20612, 2023

work page 2023

[4] [4]

Andrés-Thió, C

N. Andrés-Thió, C. Audet, M. Diago, A.E. Gheribi, S. Le Digabel, X. Lebeuf, M. Lemyre Gar- neau, and C. Tribes.solar: A solar thermal power plant simulator for blackbox optimization benchmarking. Technical Report G-2024-37, Les cahiers du GERAD, 2025. To appear in Optimization and Engineering

work page 2024

[5] [5]

Audet and J.E

C. Audet and J.E. Dennis, Jr. Pattern Search Algorithms for Mixed Variable Programming. SIAM Journal on Optimization, 11(3):573–594, 2001

work page 2001

[6] [6]

Audet and J.E

C. Audet and J.E. Dennis, Jr. Mesh Adaptive Direct Search Algorithms for Constrained Optimization.SIAM Journal on Optimization, 17(1):188–217, 2006

work page 2006

[7] [7]

Audet and J.E

C. Audet and J.E. Dennis, Jr. A Progressive Barrier for Derivative-Free Nonlinear Program- ming.SIAM Journal on Optimization, 20(1):445–472, 2009

work page 2009

[8] [8]

Audet, E

C. Audet, E. Hallé-Hannan, and S. Le Digabel. A General Mathematical Framework for Con- strained Mixed-variable Blackbox Optimization Problems with Meta and Categorical Variables. Operations Research Forum, 4(12), 2023

work page 2023