Constrained Multi-Objective Optimization for Automated Machine Learning

Brett Wujek; Joshua Griffin; Oleg Golovidov; Patrick Koch; Steven Gardner; Wayne Thompson; Yan Xu

REVIEW 4 major objections 6 minor 42 references

Constrained Multi-Objective Optimization for Automated Machine Learning

T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A constrained multi-objective search can capture Pareto fronts efficiently while focusing AutoML on the trade-offs users actually want.

desk verdict A competent AutoML engineering paper whose central claim about constraints is confounded by the benchmarks' constraints being simple box bounds on x1. read the letter →

arxiv 1908.04909 v1 pith:ELUVWA3M submitted 2019-08-14 cs.LG cs.DCcs.NEstat.ML

Steven Gardner , Oleg Golovidov , Joshua Griffin , Patrick Koch , Wayne Thompson , Brett Wujek , Yan Xu This is my paper · ORCID

classification cs.LGcs.DCcs.NEstat.ML MSC 90C2990C56

keywords automatedmachinelearningmulti-objectiveoptimizationconstrainedParetofronthyperparametertuningderivative-freedistributedcomputing

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Automated machine learning usually tunes for a single metric, but model selection in practice is a trade-off among accuracy, false positive rate, speed, size, and fairness. This paper proposes Autotune, a distributed AutoML framework that treats hyperparameter and architecture search as a constrained multi-objective optimization problem, so one run returns a Pareto front of models rather than a single model. The central claim is that the hybrid search captures known Pareto fronts efficiently on benchmark problems, and that adding constraints, such as a cap on misclassification rate, directs evaluations toward the region of the front the user cares about, producing a denser and more useful approximation under a limited budget. If true, this gives data scientists a practical way to enforce business constraints during model search instead of filtering after the fact.

What carries the argument

The load-bearing mechanism is the constraint-aware Pareto dominance rule combined with projection and penalization inside a hybrid derivative-free search. In Autotune, a point is compared by dominance with feasibility taking precedence: if one point has a smaller maximum constraint violation, it dominates regardless of objective values. Linear constraints are handled by projecting trial points back to the feasible region using tangent directions, and nonlinear constraints are folded into objective values through an L2 penalty merit function. Local refinement uses generating set search, a compass-direction method that halves its step when no improving point is found, while a genetic algorithm keeps the search global; the averaged Hausdorff distance measures convergence to the true front.

What would settle it

Give Autotune the same 5,000-evaluation budget on ZDT1, drop the constraint f1 >= 0.6, but restrict the search domain to f1 in [0.6, 1] so the feasible region is identical. If the Pareto coverage of that region matches the constrained run's coverage, the constraint-handling mechanism is not the cause of the denser front; if coverage is clearly worse, the paper's attribution is supported.

Watch

Extended reading notes

Core claim

Autotune's default strategy combines a Latin hypercube sample as an initial population, a genetic algorithm for global exploration, and generating set search for local refinement around nondominated points, all running in parallel and sharing every evaluated configuration. On ZDT1 and ZDT3 with 25,000 evaluations, the resulting Pareto front covers the true front; with 5,000 evaluations there are gaps, but adding a constraint on the first objective (f1 >= 0.6 on ZDT1, f1 <= 0.3 on ZDT3) makes the constrained portion of the front much denser. The paper reads this as evidence that constraints improve both efficiency and desirability of the search. In the case studies, a constrained Pareto run on a donation-screening problem found a model with misclassification down from 15% to 10% and false positives down 8% relative to the default, while a sales-leads study cut false negatives by 40%.

Load-bearing premise

The argument's load-bearing premise is that the observed improvement in the constrained benchmark runs comes from the constraint-aware search mechanism, and not merely from the fact that the constraint shrinks the region of solution space being explored; the paper does not compare against an unconstrained search run with the same budget on the same restricted region.

Editorial extensions

If this is right

A single AutoML run can present a user with a Pareto front of models, letting the user choose the trade-off point instead of rerunning a weighted objective many times.
Under a limited evaluation budget, adding a business constraint concentrates evaluations on the useful part of the front, yielding a denser approximation exactly where decisions are made.
Constraints that cannot be expressed as objectives, such as 'accuracy must remain above a threshold while optimizing specificity and sensitivity,' can be enforced during search rather than as a post-filter.
Because solvers share evaluated configurations, adding a new search method to the framework improves the overall search without a separate tuning run.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A controlled baseline would isolate the mechanism: if an unconstrained search with the same 5,000 evaluations but the domain box restricted to f1 in [0.6, 1] matches the constrained run's coverage, then the gain comes from the smaller search space, not the constraint machinery itself.
The same constrained Pareto machinery could be applied to fairness constraints, such as enforcing a minimum recall per demographic segment while jointly optimizing accuracy and model size, because constraints are treated as black-box functions of the trained model.
The Pareto front produced by one run could be reused as a warm-start landscape for later single-objective or preference-based searches, since it already maps the achievable trade-offs under the stated constraints.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's what you should know: this is a solid engineering paper that overclaims one result. The framework itself—hybrid GA + generating set search with constraints, parallelism, and AutoML integration—is sensible and clearly described. The unconstrained multi-objective results look fine. But the paper's distinctive claim, that constraints guide the search to better Pareto coverage, isn't supported by the benchmark experiments, because the constraints are equivalent to simple box bounds on x1.

In ZDT1 and ZDT3, f1 = x1. So the constraint f1 >= 0.6 is exactly x1 >= 0.6, and f1 <= 0.3 is exactly x1 <= 0.3. Figure 3c compares a constrained run over the region x1 in [0.6,1] against an unconstrained run over [0,1], both with 5000 evaluations. Of course coverage on that segment is denser: the search space in that dimension is half the size. The paper doesn't compare against an unconstrained run with the same box bounds, and doesn't compare against any standard multi-objective optimizer like NSGA-II under either setting. So the load-bearing claim that the constraint-handling mechanism, rather than the reduced domain, drives the improvement is untested.

What the paper does well: it's honestly written. The leads case study says outright that constraints produced no additional Pareto solutions. The related work is appropriate, and the authors build on their own prior derivative-free optimization work without hiding it. The two case studies are real and the confusion matrices are consistent with the framework working as an AutoML tool.

The soft spots beyond the confound: no quantitative metrics like hypervolume or IGD, no error bars, single runs, and no code or data released. That makes 'very efficient' impossible to verify. Also, the comparison against single-objective tuning in the case studies is not a direct comparison to an unconstrained multi-objective run at the same budget.

Who's this for: practitioners building AutoML systems who want a blueprint for a parallel constrained multi-objective search stack. It's not a theoretical contribution.

My recommendation: send it to peer review, but expect major revisions. The authors need to rerun the benchmark with the same box bounds but no constraint handling, add quantitative comparisons to established MO optimizers, and ideally release code and data. If they do that, the constraint claim might actually stand. As it stands, the evidence is suggestive but not convincing.

Referee Report

4 major / 6 minor

Summary. The paper presents Autotune, a distributed, parallel derivative-free optimization framework for automated machine learning that supports multiple objectives and general constraints. The authors describe a hybrid search strategy combining genetic algorithms, generating set search, and Latin hypercube sampling, with mechanisms for linear and nonlinear constraint handling. They evaluate the framework on two standard benchmark problems (ZDT1, ZDT3) with known Pareto fronts, showing qualitative scatter-plot evidence that Autotune captures the fronts at large budgets and that adding constraints appears to focus the search on desired regions at limited budgets. Two real-world case studies (Donors Choose and sales leads) demonstrate the application of constrained multi-objective tuning for gradient boosted trees, comparing against default models and single-objective tuning baselines. The paper's central claims are that Autotune efficiently captures Pareto fronts and that constraints guide the search to more useful regions, ultimately producing more desirable trade-offs.

Significance. If the central claims are substantiated, the framework would be a useful contribution to AutoML: it would provide a practical, extensible system for simultaneously optimizing multiple model-quality metrics while enforcing business constraints, with multi-level parallelism for scalability. The paper also stresses the no-free-lunch perspective, suggesting that a hybrid of global and local search methods can be more robust than any single method. The benchmark and case-study evidence, however, is currently qualitative and confounded for the constraint-handling claim, which is the paper's distinctive contribution. The strengths are the clear framing of the AutoML multi-objective problem, the practical algorithmic details (e.g., reference cache-tree, linear constraint projection), and the honest reporting of a case study where constraints produced no additional Pareto solutions. The main weakness is the lack of quantitative performance metrics and controlled baselines to separate the effect of the constraint mechanism from the reduced search space.

major comments (4)

[Section IV, Figures 3c and 4c] The constraint experiments are confounded with the search-space restriction. In ZDT1 and ZDT3, f1(x) = x1, so the constraint f1 ≥ 0.6 is exactly the box bound x1 ∈ [0.6, 1], and f1 ≤ 0.3 is exactly x1 ∈ [0, 0.3]. The paper does not compare against an unconstrained run on the same restricted decision-variable domain with the same 5,000-evaluation budget, nor against a standard multi-objective optimizer such as NSGA-II under either setting. Therefore, the observed denser Pareto coverage in Figures 3c and 4c could be fully explained by the smaller search space rather than by Autotune's constraint-handling mechanism, and the claim that 'adding constraints can guide the search to more promising regions' is not established.
[Section IV, Figures 3a, 3b, 4a, 4b] The efficiency claim that 'Autotune is very efficient in capturing Pareto fronts' is supported only by single-run scatter plots. No quantitative performance indicators (e.g., hypervolume, inverted generational distance, averaged Hausdorff distance, or coverage error) are reported, and no repeated runs or error bars are provided. Given the stochastic nature of the hybrid genetic/generating-set search, these qualitative plots are insufficient to assess the efficiency of the method relative to existing multi-objective optimizers.
[Section V-A, Donors Choose case study] The constrained tuning run (misclassification < 0.15) is compared only against the default model and against single-objective runs; there is no unconstrained multi-objective run at the same budget. As a result, the improvement in the 'Best' model cannot be attributed to the constraint mechanism; it may simply reflect the larger evaluation budget or the freedom to choose a different point on the multi-objective trade-off. A controlled comparison with an unconstrained multi-objective run at the same budget is needed to support the claim that constraints focus the search.
[Section V-B, Sales Leads case study] The paper honestly states that adding constraints produced no additional Pareto solutions in this case study. While this is good reporting practice, it means the central claim that constraints 'guide the search to more promising regions' rests almost entirely on the benchmark experiments, which are confounded as noted above. The manuscript should either provide a different source of evidence for the constraint-handling benefit or soften the claim.

minor comments (6)

[Algorithm 1 and Section III-A] The pseudocode lists inputs np, nc, Δ̂, and α, but α (the sufficient decrease criterion) is never used in the body of the algorithm. Please clarify how α is applied or remove it from the parameter list if it is not used in this version.
[References] References [39] and [41] are the same paper (Griffin, Kolda, and Lewis, 'Asynchronous parallel generating set search for linearly constrained optimization'). This duplication should be fixed.
[Introduction, MCC formula] The Matthews Correlation Coefficient formula is typeset incorrectly in the plain text: the denominator is not fully under the square root. Please correct the equation.
[Introduction, Theorem 1] Theorem 1 is paraphrased from Zitzler et al. but the exact source result is not cited with a theorem number or page. Please add a precise reference so the reader can verify the statement.
[Section IV] The text says the authors present 'a sampling of the results' from benchmark problems, but it is not stated how many benchmarks were run in total or how ZDT1 and ZDT3 were selected. Please clarify the selection criteria and whether the omitted results are consistent with the reported ones.
[Figures 5-8] In the case-study figures, the markers labeled F1, MCE, KS, and AUC refer to results from single-objective tuning runs, but it is not explained whether these are the best model from each run or the entire set of nondominated solutions. Please clarify the plotting convention.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: evaluation relies on external ZDT benchmarks and real-world case studies; self-citations are background methodology, not load-bearing derivation.

full rationale

This paper is an experimental systems description, not a derivation from first principles. The central claims—that Autotune captures Pareto fronts efficiently and that constraints focus search—are tested against ZDT1/ZDT3 with known Pareto fronts and on two real datasets. The benchmark constraints (f1≥0.6, f1≤0.3) coincide with bounds on x1 because f1=x1 in ZDT1/ZDT3, but the paper uses these as test problems rather than as inputs to a fitted prediction; no parameter is fitted to a subset and then renamed a prediction. The authors' prior derivative-free optimization work is cited as the methodological basis of Autotune, but the experimental evidence is external and falsifiable, so these citations are not load-bearing in the circularity sense. Concerns that the constrained-vs-unconstrained comparison is confounded by the reduced feasible region are an experimental-design/correctness issue, not equation-level circularity. The leads case study even reports that constraints added no new Pareto solutions, which is inconsistent with a forced favorable result.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a small set of background assumptions: derivative-free black-box optimization is appropriate for the mixed-variable AutoML setting, concurrent global and local search with shared evaluations is beneficial, ZDT benchmarks transfer to real hyperparameter landscapes, and the described constraint-handling recipes converge to useful constrained fronts. The experimental results are parameterized by unstated or arbitrary search-control settings and evaluation budgets, reducing the strength of the efficiency claim.

free parameters (2)

Autotune search control parameters (population size, number of centers, initial step size, sufficient decrease… = Unknown; population size 50 and 20 iterations in case studies
Algorithm 1 requires these settings, but their values and sensitivity are not reported for most experiments, so the reported front quality is tied to unstated choices.
Evaluation budget = 5,000 and 25,000 evaluations for ZDT; case study budgets not stated
The efficiency claim is relative to budget; at 5,000 evaluations the unconstrained fronts have obvious gaps, so the budget choice drives the qualitative conclusion.

assumptions (4)

domain assumption Black-box derivative-free optimization is effective for problems with fewer than about 100 variables, making it suitable for AutoML hyperparameter tuning.
Used in Section III-A to justify the whole approach; cited to Gray and Fowler but not validated on the mixed-integer, categorical AutoML search space here.
domain assumption Running global and local searches concurrently with shared evaluations is at least as robust as choosing one strategy, without extra overhead.
Section III-A states this as a design claim; no ablation compares hybrid sharing versus sequential global-then-local search.
domain assumption ZDT1 and ZDT3 are representative of the difficulty of real AutoML model-selection problems.
Section IV uses them as the only optimization benchmarks; transfer to hyperparameter tuning landscapes is assumed, not demonstrated.
domain assumption Constraint handling via projection for linear constraints and L2 penalty for nonlinear constraints preserves convergence to the constrained Pareto front.
Section III-C describes the methods but provides no convergence theorem and no empirical constraint-violation analysis.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Constrained Multi-Objective Optimization for Automated Machine Learning." pith.science (2026). https://pith.science/paper/ELUVWA3M

@misc{pith2026190804909,
  author       = {Pith},
  title        = {Pith review of: Constrained Multi-Objective Optimization for Automated Machine Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ELUVWA3M}},
  note         = {Machine review of arXiv:1908.04909}
}

read the original abstract

Automated machine learning has gained a lot of attention recently. Building and selecting the right machine learning models is often a multi-objective optimization problem. General purpose machine learning software that simultaneously supports multiple objectives and constraints is scant, though the potential benefits are great. In this work, we present a framework called Autotune that effectively handles multiple objectives and constraints that arise in machine learning problems. Autotune is built on a suite of derivative-free optimization methods, and utilizes multi-level parallelism in a distributed computing environment for automatically training, scoring, and selecting good models. Incorporation of multiple objectives and constraints in the model exploration and selection process provides the flexibility needed to satisfy trade-offs necessary in practical machine learning applications. Experimental results from standard multi-objective optimization benchmark problems show that Autotune is very efficient in capturing Pareto fronts. These benchmark results also show how adding constraints can guide the search to more promising regions of the solution space, ultimately producing more desirable Pareto fronts. Results from two real-world case studies demonstrate the effectiveness of the constrained multi-objective optimization capability offered by Autotune.

Figures

Figures reproduced from arXiv: 1908.04909 by the authors.

**Figure 2.** Example Pareto Front. f1(x) and f2(x) are two functions to be minimized. Points a, b, c and d consist of the Pareto frontier found. A Pareto front contains only nondominated solutions. In [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** ZDT1 Benchmark Test Problem [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: ZDT3 Benchmark Test Problem of 5000 evaluations. This plot clearly shows how adding the constraint has focused the optimization to that lower-right section of the solution space. This has allowed Autotune to capture a much better representation of the true Pareto front…

**Figure 6.** Figure 6: Donors Choose - Misclassification <0.15 While the near zero FPR values are appealing, the increase in the misclassification makes these configurations undesirable. It is more beneficial to look at models with both objectives reduced compared to the default model. Beca…

**Figure 5.** Figure 5: Donors Choose - All Evaluations The multi-objective tuning results for the Donors Choose data set are shown in Figures 5 and 6. In [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 7.** Figure 7: Leads Data Results - All Evaluations The Pareto front is shown in more detail in [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

**Figure 8.** Figure 8: Leads Data - Pareto Front and Single Objective [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 40 canonical work pages

[1]

Cloud automl BETA,

Google, “Cloud automl BETA,” 2018. [Online]. Available: https: //cloud.google.com/automl

work page 2018
[2]

[Online]

Microsoft, “Automl,” 2018. [Online]. Available: https://www.microsoft. com/en-us/research/project/automl/

work page 2018
[3]

Open sourcing transmogrifai,

Salesforce, “Open sourcing transmogrifai,” 2018. [Online]. Available: https://engineering.salesforce.com/ open-sourcing-transmogrifai-4e5d0e098da2

work page 2018
[4]

Efﬁcient and robust automated machine learning,

M. Feurer, A. Klein, K. Eggensperger, J. Springenberg, M. Blum, and F. Hutter, “Efﬁcient and robust automated machine learning,” in Advances in Neural Information Processing Systems 28, C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, Eds. Curran Associates, Inc., 2015, pp. 2962–2970

work page 2015
[5]

Auto-weka 2.0: Automatic model selection and hyperparameter opti- mization in weka,

L. Kotthoff, C. Thornton, H. Hoos, F. Hutter, and K. Leyton-Brown., “Auto-weka 2.0: Automatic model selection and hyperparameter opti- mization in weka,” Journal of Machine Learning Research , vol. 18, no. 25, pp. 1–5, 2017

work page 2017
[6]

Automl: Automatic machine learning,

h2o, “Automl: Automatic machine learning,” 2018. [Online]. Available: http://docs.h2o.ai/h2o/latest-stable/h2o-docs/automl.html

work page 2018
[7]

H. Jin, Q. Song, and X. Hu. (2018) Auto-keras: Efﬁcient neural architecture search with network morphism

work page 2018
[8]

An experimental comparison of performance measures for classiﬁcation,

C. Ferri, J. Hern ´andez-Orallo, and R. Modroiu, “An experimental comparison of performance measures for classiﬁcation,” Pattern Recogn. Lett., vol. 30, no. 1, pp. 27–38, 2009

work page 2009

Show all 42 references

[9]

Evaluation: From precision, recall and f-factor to roc, informedness, markedness & correlation,

D. Powers, “Evaluation: From precision, recall and f-factor to roc, informedness, markedness & correlation,” Mach. Learn. Technol., vol. 2, 01 2008

2008
[10]

Why quality assessment of multiobjective optimizers is difﬁcult,

E. Zitzler, M. Laumanns, L. Thiele, C. M. Fonseca, and V . G. da Fonseca, “Why quality assessment of multiobjective optimizers is difﬁcult,” in Proceedings of the 4th Annual Conference on Genetic and Evolutionary Computation, ser. GECCO’02. San Francisco, CA, USA: Morgan Kaufm...

2002
[11]

Comparison of the predicted and observed secondary structure of t4 phage lysozyme,

B. W Matthews, “Comparison of the predicted and observed secondary structure of t4 phage lysozyme,” Biochimica et Biophysica Acta , vol. 405, pp. 442–51, 11 1975

1975
[12]

The limitations of fraud detection today, and its future with bolt,

Bolt, “The limitations of fraud detection today, and its future with bolt,” 2018. [Online]. Available: https://medium.com/@bolt.com/ the-limitations-of-fraud-detection-today-and-its-future-with-bolt-5cdac0114a2f

2018
[13]

Pareto-based multiobjective machine learning: An overview and case studies,

Y . Jin and B. Sendhoff, “Pareto-based multiobjective machine learning: An overview and case studies,” Trans. Sys. Man Cyber Part C , vol. 38, no. 3, pp. 397–415, 2008

2008
[14]

Comparison of multiobjective evolutionary algorithms: Empirical results,

E. Zitzler, K. Deb, and L. Thiele, “Comparison of multiobjective evolutionary algorithms: Empirical results,” Evolutionary Computation, vol. 8, no. 2, pp. 173–195, 2000

2000
[15]

Jin, Ed., Multi-Objective Machine Learning , ser

Y . Jin, Ed., Multi-Objective Machine Learning , ser. Studies in Compu- tational Intelligence. Springer, 2006, vol. 16

2006
[16]

ADC: automated deep compression and acceleration with reinforcement learning,

Y . He and S. Han, “ADC: automated deep compression and acceleration with reinforcement learning,” CoRR, vol. abs/1802.03494, 2018

2018 arXiv
[17]

Pareto- optimal multi-objective dimensionality reduction deep auto-encoder for mammography classiﬁcation,

S. A. Taghanaki, J. Kawahara, B. Miles, and G. Hamarneh, “Pareto- optimal multi-objective dimensionality reduction deep auto-encoder for mammography classiﬁcation,” Computer methods and programs in biomedicine, vol. 145, pp. 85–93, 2017

2017
[18]

Beyond mitchell: Multi-objective machine learning – min- imal entropy, energy and error,

J. Loeckx, “Beyond mitchell: Multi-objective machine learning – min- imal entropy, energy and error,” in 11th Metaheuristics International Conference (MIC), Agadir, Morocco, 6 2015

2015
[19]

Big models for big data using multi objective averaged one dependence estimators,

M. Panda, “Big models for big data using multi objective averaged one dependence estimators,” CoRR, vol. abs/1610.07752, 2016

2016 arXiv
[20]

Multi-objective evolution of artiﬁcial neural networks in multi-class medical diagnosis problems with class imbalance,

A. Shenﬁeld and S. Rostami, “Multi-objective evolution of artiﬁcial neural networks in multi-class medical diagnosis problems with class imbalance,” in CIBCB. IEEE, 2017, pp. 1–8

2017
[21]

Rapidminer documentation,

RapidMiner, “Rapidminer documentation,” 2019. [Online]. Available: https://docs.rapidminer.com

2019
[22]

Nemo : Neuro-evolution with multiobjective optimization of deep neural network for speed and accuracy,

Y .-H. Kim, B. Reddy, S. Yun, and C. Seo, “Nemo : Neuro-evolution with multiobjective optimization of deep neural network for speed and accuracy,” 2017

2017
[23]

Elsken, J

T. Elsken, J. H. Metzen, and F. Hutter. (2018) Efﬁcient multi-objective neural architecture search via lamarckian evolution. [Online]. Available: https://arxiv.org/pdf/1804.09081.pdf

2018 arXiv
[24]

Ppp-net: Platform-aware progressive search for pareto-optimal neural architec- tures,

J.-D. Dong, A.-C. Cheng, D.-C. Juan, W. Wei, and M. Sun, “Ppp-net: Platform-aware progressive search for pareto-optimal neural architec- tures,” 2018

2018
[25]

DVOLVER: Efﬁcient pareto-optimal neural network architecture search,

G. Michel, M. A. Alaoui, A. Lebois, A. Feriani, and M. Felhi, “DVOLVER: Efﬁcient pareto-optimal neural network architecture search,” 2019

2019
[26]

A fast elitist non- dominated sorting genetic algorithm for multi-objective optimization: Nsga-ii,

K. Deb, S. Agrawal, A. Pratap, and T. Meyarivan, “A fast elitist non- dominated sorting genetic algorithm for multi-objective optimization: Nsga-ii,” in Parallel Problem Solving from Nature PPSN VI , M. Schoe- nauer, K. Deb, G. Rudolph, X. Yao, E. Lutton, J. J. Merelo, and H.-...

2000
[27]

Multiobjective optimization through a series of single-objective formulations,

C. Audet, G. Savard, and W. Zghal, “Multiobjective optimization through a series of single-objective formulations,” SIAM J. on Opti- mization, vol. 19, no. 1, pp. 188–210, Feb. 2008

2008
[28]

A multi-objective DIRECT algorithm towards structural damage identiﬁcation with limited dynamic response information,

P. Cao, Q. Shuai, and J. Tang, “A multi-objective DIRECT algorithm towards structural damage identiﬁcation with limited dynamic response information,” CoRR, 2017

2017
[29]

Multiglods: global and local multiobjective optimization using direct search,

A. L. Cust ´odio and J. F. A. Madeira, “Multiglods: global and local multiobjective optimization using direct search,” Journal of Global Optimization, vol. 72, no. 2, pp. 323–345, Oct 2018

2018
[30]

Direct multisearch for multiobjective optimization

A. L. Cust ´odio, J. F. A. Madeira, A. I. F. Vaz, and L. N. Vicente, “Direct multisearch for multiobjective optimization.” SIAM Journal on Optimization, vol. 21, no. 3, pp. 1109–1140, 2011

2011
[31]

Reference point based multi-objective optimiza- tion using evolutionary algorithms,

K. Deb and J. Sundar, “Reference point based multi-objective optimiza- tion using evolutionary algorithms,” in Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, ser. GECCO ’06. New York, NY , USA: ACM, 2006, pp. 635–642

2006
[32]

Bayesian guided pattern search for robust local optimization,

M. A. Taddy, H. K. H. Lee, G. A. Gray, and J. D. Grifﬁn, “Bayesian guided pattern search for robust local optimization,” Technometrics, vol. 51, pp. 389–401, 2009

2009
[33]

Hopspack 2.0 user manual (v 2.0.2),

T. Plantenga, “Hopspack 2.0 user manual (v 2.0.2),” Sandia National Laboratories, Tech. Rep., 2009

2009
[34]

Hybrid optimization schemes for simulation-based problems,

G. A. Gray, K. R. Fowler, and J. D. Grifﬁn, “Hybrid optimization schemes for simulation-based problems,” Procedia Computer Science , vol. 1, pp. 1349–1357, 2010

2010
[35]

Asynchronous parallel hybrid optimiza- tion combining direct and gss,

J. D. Grifﬁn and T. G. Kolda, “Asynchronous parallel hybrid optimiza- tion combining direct and gss,” Optimization Methods and Software , vol. 25, pp. 797–817, 2010

2010
[36]

The effectiveness of derivative-free hybrid methods for black-box optimization,

G. A. Gray and K. R. Fowler, “The effectiveness of derivative-free hybrid methods for black-box optimization,” International Journal of Mathematical Modeling and Numerical Optimization , vol. 2, pp. 112– 133, 2011

2011
[37]

Derivative-free optimization via evolutionary algorithms guiding local search (eagls) for minlp,

J. D. Grifﬁn, K. R. Fowler, G. A. Gray, and T. Hemker, “Derivative-free optimization via evolutionary algorithms guiding local search (eagls) for minlp,” Paciﬁc Journal of Optimization , vol. 7, pp. 425–443, 2011

2011
[38]

D. E. Goldberg, Genetic Algorithms in Search, Optimization and Ma- chine Learning, 1st ed. Addison-Wesley Longman Publishing Co., Inc., 1989

1989
[39]

Asynchronous parallel gen- erating set search for linearly constrained optimization,

J. D. Grifﬁn, T. G. Kolda, and R. M. Lewis, “Asynchronous parallel gen- erating set search for linearly constrained optimization,” SIAM Journal on Scientiﬁc Computing , vol. 30, pp. 1892–1924., 2008

1924
[40]

Using the averaged hausdorff distance as a performance measure in evolutionary multiobjective optimization,

O. Sch ¨utze, X. Esquivel, A. Lara, and C. A. Coello Coello, “Using the averaged hausdorff distance as a performance measure in evolutionary multiobjective optimization,” IEEE Transactions on Evolutionary Com- putation, vol. 16, pp. 504–522, 2012

2012
[41]

Asynchronous parallel gen- erating set search for linearly constrained optimization,

J. D. Grifﬁn, T. G. Kolda, and R. M. Lewis, “Asynchronous parallel gen- erating set search for linearly constrained optimization,” SIAM Journal on Scientiﬁc Computing , vol. 30, pp. 1892–1924, 2008

1924
[42]

Nonlinearly constrained optimization using heuristic penalty methods and asynchronous parallel generating set search,

J. D. Grifﬁn and T. G. Kolda, “Nonlinearly constrained optimization using heuristic penalty methods and asynchronous parallel generating set search,” Applied Mathematics Research Express , vol. 2010, pp. 36– 62, 2010

2010

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Cloud automl BETA,

Google, “Cloud automl BETA,” 2018. [Online]. Available: https: //cloud.google.com/automl

work page 2018

[2] [2]

[Online]

Microsoft, “Automl,” 2018. [Online]. Available: https://www.microsoft. com/en-us/research/project/automl/

work page 2018

[3] [3]

Open sourcing transmogrifai,

Salesforce, “Open sourcing transmogrifai,” 2018. [Online]. Available: https://engineering.salesforce.com/ open-sourcing-transmogrifai-4e5d0e098da2

work page 2018

[4] [4]

Efﬁcient and robust automated machine learning,

M. Feurer, A. Klein, K. Eggensperger, J. Springenberg, M. Blum, and F. Hutter, “Efﬁcient and robust automated machine learning,” in Advances in Neural Information Processing Systems 28, C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, Eds. Curran Associates, Inc., 2015, pp. 2962–2970

work page 2015

[5] [5]

Auto-weka 2.0: Automatic model selection and hyperparameter opti- mization in weka,

L. Kotthoff, C. Thornton, H. Hoos, F. Hutter, and K. Leyton-Brown., “Auto-weka 2.0: Automatic model selection and hyperparameter opti- mization in weka,” Journal of Machine Learning Research , vol. 18, no. 25, pp. 1–5, 2017

work page 2017

[6] [6]

Automl: Automatic machine learning,

h2o, “Automl: Automatic machine learning,” 2018. [Online]. Available: http://docs.h2o.ai/h2o/latest-stable/h2o-docs/automl.html

work page 2018

[7] [7]

H. Jin, Q. Song, and X. Hu. (2018) Auto-keras: Efﬁcient neural architecture search with network morphism

work page 2018

[8] [8]

An experimental comparison of performance measures for classiﬁcation,

C. Ferri, J. Hern ´andez-Orallo, and R. Modroiu, “An experimental comparison of performance measures for classiﬁcation,” Pattern Recogn. Lett., vol. 30, no. 1, pp. 27–38, 2009

work page 2009