pith. sign in

arxiv: 2011.10760 · v1 · submitted 2020-11-21 · 💻 cs.NE · cs.LG· cs.PF· math.OC

Enhanced Innovized Repair Operator for Evolutionary Multi- and Many-objective Optimization

Sukrit Mittal , Dhish Kumar Saxena , Kalyanmoy Deb , Erik Goodman This is my paper

Pith reviewed 2026-05-24 14:11 UTC · model grok-4.3

classification 💻 cs.NE cs.LGcs.PFmath.OC
keywords innovizationrepair operatorrandom forestmulti-objective optimizationevolutionary algorithmsconvergence improvementmachine learningPareto-optimal solutions
0
0 comments X

The pith

A random forest trained on past non-dominated solutions can repair genetic offspring to accelerate convergence in evolutionary multi-objective optimization.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes training a random forest model on the sequence of improving non-dominated solutions produced during a run of an evolutionary algorithm. The model learns how design variables typically change to move solutions closer to the Pareto set. This learned model is then inserted as an extra repair step applied to new offspring generated by standard crossover and mutation operators. The repair step is tested inside NSGA-II, NSGA-III and MOEA/D on problems with two to five objectives and produces visibly faster convergence. No new function evaluations are required because the model re-uses information already collected inside the same run.

Core claim

We propose to use the resulting ML model as an additional innovized repair (IR2) operator to be applied on offspring solutions created by the usual genetic operators, as a novel mean of improving their convergence properties. In this paper, the well-known random forest (RF) method is used as the ML model and is integrated with various evolutionary multi- and many-objective optimization algorithms, including NSGA-II, NSGA-III, and MOEA/D. On several test problems ranging from two to five objectives, we demonstrate improvement in convergence behaviour using the proposed IR2-RF operator.

What carries the argument

The IR2-RF operator: a random forest regressor that maps the current state of a solution to the design-variable modifications observed in earlier non-dominated solutions, thereby predicting a repaired offspring expected to lie closer to the Pareto set.

If this is right

  • The repair operator can be dropped into existing evolutionary multi-objective frameworks without changing their selection or variation mechanisms.
  • Convergence improvement occurs on problems having between two and five objectives while using only information already generated inside the run.
  • Because the operator consumes no extra evaluations, the computational budget can be re-allocated to larger populations or more generations.
  • The same modelling idea can be re-used whenever an algorithm already maintains an archive of progressively better non-dominated points.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Replacing random forest with other supervised learners could produce different trade-offs between prediction accuracy and training cost inside the same repair framework.
  • The learned repair mapping might be stored and re-applied across multiple related problem instances to transfer knowledge between runs.
  • If the repair step proves reliable, it could reduce the need for hand-crafted variation operators that are currently tuned per problem class.

Load-bearing premise

The patterns that a random forest extracts from the chronological sequence of non-dominated solutions in one run will remain useful when applied to repair new offspring generated later in the same run.

What would settle it

Execute the IR2-RF versions of NSGA-II, NSGA-III and MOEA/D on the same test problems and measure whether the convergence curves are statistically indistinguishable from or worse than the baseline versions that lack the repair step.

Figures

Figures reproduced from arXiv: 2011.10760 by Dhish Kumar Saxena, Erik Goodman, Kalyanmoy Deb, Sukrit Mittal.

Figure 1
Figure 1. Figure 1: EMO/EMaO algorithm with IR2 operator. point. However, the case of a non-uniform distribution at any generation t has been discussed further. In order to associate the current population members Pt with the reference points, a normalization procedure is performed in the objective space. Given that F (i) k represents the k th objective value for the i th individual in Pt, F¯ (i) k represents the k th normali… view at source ↗
Figure 2
Figure 2. Figure 2: A schematic for identifying target-points using ASF in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: for two problems, L1 and L2 [9]. It can be observed that there is a consistent advantage at almost all generations with η = 1.1. Table III also shows the median HV values at the final generation and their respective p-values. However, even though η = 1.1 produces the best results, it is noteworthy that solutions with η > 1 are better than no enhancement (η = 1) at all. To conclude, this preliminary paramet… view at source ↗
Figure 4
Figure 4. Figure 4: Median IGD plot for DTLZ3 and median HV plot for [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Median HV plots for WFG4 (with standard and difficult [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 8
Figure 8. Figure 8: Median HV plots for L1 and L2 problems with NSGA [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
Figure 6
Figure 6. Figure 6: Median HV plots for biased problems: ZDT6 and [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Median HV plots for WFG7 (with standard and difficult [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 9
Figure 9. Figure 9: Median HV plots for DTLZ1 (M = 4) problem solved by NSGA-III and NSGA-III-IR2-RF using reference points generated by Das-Dennis method and layer-wise energy Method. boundary points and interior points on quality of learning in future studies. E. Limited Use of Learning In this subsection, the effect of terminating the RF-based learning process with a threshold gth and its importance are discussed [PITH_FU… view at source ↗
Figure 11
Figure 11. Figure 11: Analysis on constrained imbalance problem CIBN2. [PITH_FULL_IMAGE:figures/full_fig_p013_11.png] view at source ↗
Figure 10
Figure 10. Figure 10: Limited learning on two WFG problems. There can be some problems where the idea of termination becomes vital, where the algorithm first converges to only a portion of the PO front and then spreads along it. For example, the constrained imbalance problem CIBN2 [33], for which the PO front is shown with a solid black line in Figure 11a is such a problem. The green dots represent the premature front where th… view at source ↗
read the original abstract

"Innovization" is a task of learning common relationships among some or all of the Pareto-optimal (PO) solutions in multi- and many-objective optimization problems. Recent studies have shown that a chronological sequence of non-dominated solutions obtained in consecutive iterations during an optimization run also possess salient patterns that can be used to learn problem features to help create new and improved solutions. In this paper, we propose a machine-learning- (ML-) assisted modelling approach that learns the modifications in design variables needed to advance population members towards the Pareto-optimal set. We then propose to use the resulting ML model as an additional innovized repair (IR2) operator to be applied on offspring solutions created by the usual genetic operators, as a novel mean of improving their convergence properties. In this paper, the well-known random forest (RF) method is used as the ML model and is integrated with various evolutionary multi- and many-objective optimization algorithms, including NSGA-II, NSGA-III, and MOEA/D. On several test problems ranging from two to five objectives, we demonstrate improvement in convergence behaviour using the proposed IR2-RF operator. Since the operator does not demand any additional solution evaluations, instead using the history of gradual and progressive improvements in solutions over generations, the proposed ML-based optimization opens up a new direction of optimization algorithm development with advances in AI and ML approaches.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes an ML-assisted innovized repair (IR2) operator that trains a random forest on the chronological sequence of non-dominated solutions generated during an EMO run. The model predicts design-variable modifications that advance solutions toward the Pareto set and is applied as an additional repair step to offspring produced by standard genetic operators. The operator is integrated into NSGA-II, NSGA-III, and MOEA/D and is claimed to improve convergence on test problems with 2–5 objectives without requiring extra function evaluations.

Significance. If the reported gains are reproducible and the learned mappings transfer reliably, the work offers a low-cost mechanism for injecting data-driven repair into existing EMO frameworks by exploiting solution-history patterns. The absence of additional evaluations and the compatibility with multiple established algorithms are concrete strengths that could encourage further hybrid AI–evolutionary research.

major comments (2)
  1. [Experimental Results] The central performance claim rests on the transfer of the RF-learned repair mapping from the training sequence of non-dominated solutions to later offspring. The experimental section provides no within-run hold-out validation, cross-generation testing, or ablation that isolates whether the observed convergence improvement is attributable to the IR2 operator rather than the base algorithm. This omission is load-bearing for the attribution of benefit.
  2. [Proposed IR2-RF Operator] The method description does not specify whether the random forest is trained once on early generations and then held fixed or retrained periodically as the non-dominated set evolves. Without this detail or corresponding experiments, it is unclear whether the reported improvements survive non-stationarity in the population trajectory.
minor comments (2)
  1. [Abstract] The abstract states that improvement is demonstrated but supplies no quantitative metrics or statistical test references; adding one sentence summarizing effect size or win rates would improve clarity.
  2. [Method] Notation for the random-forest input features (design-variable deltas versus absolute values) should be defined explicitly before the first use in the method section.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address each major comment below with clarifications and indicate planned revisions where appropriate.

read point-by-point responses

  1. Referee: [Experimental Results] The central performance claim rests on the transfer of the RF-learned repair mapping from the training sequence of non-dominated solutions to later offspring. The experimental section provides no within-run hold-out validation, cross-generation testing, or ablation that isolates whether the observed convergence improvement is attributable to the IR2 operator rather than the base algorithm. This omission is load-bearing for the attribution of benefit.

    Authors: Our experiments compare each base algorithm (NSGA-II, NSGA-III, MOEA/D) directly against the identical algorithm augmented only by the IR2-RF operator across multiple 2- to 5-objective test problems, using standard convergence metrics. Because the sole difference between paired runs is the application of the repair operator, the consistent gains isolate its contribution. We nevertheless agree that explicit ablation or hold-out tests would strengthen attribution and will add such studies in the revision. revision: partial

  2. Referee: [Proposed IR2-RF Operator] The method description does not specify whether the random forest is trained once on early generations and then held fixed or retrained periodically as the non-dominated set evolves. Without this detail or corresponding experiments, it is unclear whether the reported improvements survive non-stationarity in the population trajectory.

    Authors: We acknowledge the description is insufficiently precise on training frequency. In the revised manuscript we will explicitly state that the random forest is retrained periodically on the most recent non-dominated solutions and will add analysis of retraining intervals to address non-stationarity. revision: yes

Circularity Check

0 steps flagged

No significant circularity; method is empirical and data-driven

full rationale

The paper proposes training a random forest on pairs from the chronological sequence of non-dominated solutions to predict design-variable modifications, then applying the model as an IR2 repair operator on new offspring. This construction does not reduce any claimed result to its inputs by definition or by fitting a parameter to the target quantity and relabeling it a prediction. No self-citation chain is invoked as a uniqueness theorem or load-bearing premise; the central performance claim rests on reported experiments across test problems rather than an algebraic identity. The generalization assumption is an empirical hypothesis, not a circular step. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available; no explicit free parameters, axioms, or invented entities are stated. The approach relies on standard random-forest training and the assumption that historical improvement patterns transfer to new offspring.

pith-pipeline@v0.9.0 · 5790 in / 1077 out tokens · 23424 ms · 2026-05-24T14:11:11.835485+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 34 canonical work pages

  1. [1]

    Deb, Multi-objective optimization using evolutionary algorithms

    K. Deb, Multi-objective optimization using evolutionary algorithms. Chichester, UK: Wiley, 2001

    work page 2001

  2. [2]

    C. A. C. Coello, D. A. VanVeldhuizen, and G. Lam- ont, Evolutionary Algorithms for Solving Multi-Objective Problems. Boston, MA: Kluwer, 2002

    work page 2002

  3. [3]

    A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: Nsga-ii,

    K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, “A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: Nsga-ii,”IEEE transactions on evolutionary computation, vol. 6, no. 2, pp. 182–197, 2002

    work page 2002

  4. [4]

    An evolutionary many-objective optimization algorithm using reference-point based non- dominated sorting approach, Part I: Solving problems with box constraints,

    K. Deb and H. Jain, “An evolutionary many-objective optimization algorithm using reference-point based non- dominated sorting approach, Part I: Solving problems with box constraints,” IEEE Transactions on Evolution- ary Computation, vol. 18, no. 4, pp. 577–601, 2014

    work page 2014

  5. [5]

    Jain and K

    H. Jain and K. Deb, “An evolutionary many-objective optimization algorithm using reference-point based non- dominated sorting approach, Part II: Handling constraints and extending to an adaptive approach,” IEEE Transac- tions on Evolutionary Computation , vol. 18, no. 4, pp. 602–622, 2014

    work page 2014

  6. [6]

    MOEA/D: A multiobjective evolu- tionary algorithm based on decomposition,

    Q. Zhang and H. Li, “MOEA/D: A multiobjective evolu- tionary algorithm based on decomposition,” Evolutionary Computation, IEEE Transactions on , vol. 11, no. 6, pp. 712–731, 2007

    work page 2007

  7. [7]

    The bayesian optimization algorithm,

    M. Pelikan, D. E. Goldberg, and E. Cant ˜Aº-Paz, “The bayesian optimization algorithm,” in GECCO. Morgan Kaufmann Publishers Inc., 1999, pp. 525–532

    work page 1999

  8. [8]

    In- troducing moneda: scalable multiobjective optimization with a neural estimation of distribution algorithm,

    M. Luis, G. Jesus, B. Antonio, and M. J. Manuel, “In- troducing moneda: scalable multiobjective optimization with a neural estimation of distribution algorithm,” in Proceedings of Genetic and Evolutionary Computation conference (GECCO-2008), 2008, pp. 689–696

    work page 2008

  9. [9]

    An ANN–assisted repair operator for evolutionary multiobjective optimization,

    S. Mittal, D. Saxena, K. Deb, and E. Goodman, “An ANN–assisted repair operator for evolutionary multiobjective optimization,” Department of Electrical and Computer Engineering, Michigan State University, East Lansing, USA, Tech. Rep. COIN Report No. 2020005, 2020. [Online]. Available: https://www.egr. msu.edu/∼kdeb/papers/c2020005.pdf

    work page 2020

  10. [10]

    Effect of size and order of vari- ables in rules for multi-objective repair-based innoviza- tion procedure,

    A. Gaur and K. Deb, “Effect of size and order of vari- ables in rules for multi-objective repair-based innoviza- tion procedure,” in Proceedings of Congress on Evolu- tionary Computation (CEC-2017) Conference . Piscat- way, NJ: IEEE Press, 2017

    work page 2017

  11. [11]

    Hybrid evolutionary multi- objective optimization and analysis of machining opera- tions,

    K. Deb and R. Datta, “Hybrid evolutionary multi- objective optimization and analysis of machining opera- tions,” Engineering Optimization, vol. 44, no. 6, pp. 685– 706, 2012

    work page 2012

  12. [12]

    A large-scale bi-objective optimization of solid rocket motors using innovization,

    A. Ghosh, E. Goodman, K. Deb, R. Averill, and A. Diaz, “A large-scale bi-objective optimization of solid rocket motors using innovization,” in 2020 IEEE Congress on Evolutionary Computation (CEC) , 2020, pp. 1–8

    work page 2020

  13. [13]

    Towards automating the dis- covery of certain innovative design principles through a clustering based optimization technique,

    S. Bandaru and K. Deb, “Towards automating the dis- covery of certain innovative design principles through a clustering based optimization technique,” Engineering optimization, vol. 43, no. 9, pp. 911–941, 2011

    work page 2011

  14. [14]

    In- tegration of data mining and multi-objective optimisation for decision support in production systems development,

    C. Dudas, A. H. Ng, L. Pehrsson, and H. Bostr ¨om, “In- tegration of data mining and multi-objective optimisation for decision support in production systems development,” International Journal of Computer Integrated Manufac- turing, no. ahead-of-print, pp. 1–16, 2013

    work page 2013

  15. [15]

    A dimensionally-aware genetic programming architecture for automated innovization,

    S. Bandaru and K. Deb, “A dimensionally-aware genetic programming architecture for automated innovization,” in Proceedings of the Seventh International Conference on Evolutionary Multi-Criterion Optimization (EMO-13), LNCS 7811. Heidelberg: Springer, 2013, pp. 513–527

    work page 2013

  16. [16]

    Learning-based multi-objective optimization through ANN–assisted online innovization,

    S. Mittal, D. K. Saxena, and K. Deb, “Learning-based multi-objective optimization through ANN–assisted online innovization,” in Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion , ser. GECCO 20. New York, NY , USA: Association for Computing Machinery, 2020, pp. 171–172. [Online]. Available: https://doi.org/10.1145/3377929.3389925

    work page doi:10.1145/3377929.3389925 2020

  17. [17]

    A survey of optimization by building and using probabilistic models,

    M. Pelikan, D. Goldberg, and F. Lobo, “A survey of optimization by building and using probabilistic models,” Computational Optimization and Applications , vol. 21, no. 1, pp. 5–20, 2002

    work page 2002

  18. [18]

    A robust hybrid approach based on estimation of distribution algorithm and support vector machine for hunting candidate disease genes,

    L. Li, H. Chen, and C. L. et al., “A robust hybrid approach based on estimation of distribution algorithm and support vector machine for hunting candidate disease genes,” The Scientific World Journal , vol. 2013, no. 393570, p. 7, 2013

    work page 2013

  19. [19]

    A survey of repair methods used as constraint handling techniques in evolutionary algo- rithms,

    S. Salcedo-Sanz, “A survey of repair methods used as constraint handling techniques in evolutionary algo- rithms,” Computer Science Review, vol. 3, no. 3, pp. 175 – 192, 2009

    work page 2009

  20. [20]

    Distributed memetic differential evolution with the synergy of lamarckian and baldwinian learning,

    C. Zhang, J. Chen, and B. Xin, “Distributed memetic differential evolution with the synergy of lamarckian and baldwinian learning,” Applied Soft Computing , vol. 13, no. 5, pp. 2947 – 2959, 2013

    work page 2013

  21. [21]

    Multi- objective immune algorithm with baldwinian learning,

    Y . Qi, F. Liu, M. Liu, M. Gong, and L. Jiao, “Multi- objective immune algorithm with baldwinian learning,” 15 Applied Soft Computing, vol. 12, no. 8, pp. 2654 – 2674, 2012

    work page 2012

  22. [22]

    Unveiling innovative design principles by means of multiple conflicting objectives,

    K. Deb, “Unveiling innovative design principles by means of multiple conflicting objectives,” Engineering Optimization, vol. 35, no. 5, pp. 445–470, 2003

    work page 2003

  23. [23]

    Innovization: Innovating design principles through optimization

    K. Deb and A. Srinivasan, “Innovization: Innovating design principles through optimization.” in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2006), New York: ACM, 2006, pp. 1629–1636

    work page 2006

  24. [24]

    Normal-boundary intersection: A new method for generating the Pareto surface in nonlin- ear multicriteria optimization problems,

    I. Das and J. Dennis, “Normal-boundary intersection: A new method for generating the Pareto surface in nonlin- ear multicriteria optimization problems,” SIAM Journal of Optimization, vol. 8, no. 3, pp. 631–657, 1998

    work page 1998

  25. [25]

    Generating uni- formly distributed points on a unit simplex for evo- lutionary many-objective optimization,

    K. Deb, S. Bandaru, and H. Seada, “Generating uni- formly distributed points on a unit simplex for evo- lutionary many-objective optimization,” in Proceedings of the Tenth International Conference on Evolutionary Multi-Criterion Optimization (EMO-19), LNCS 11411 . Heidelberg: Springer, 2019, pp. 179–190

    work page 2019

  26. [26]

    The use of reference objectives in mul- tiobjective optimization,

    A. P. Wierzbicki, “The use of reference objectives in mul- tiobjective optimization,” in Multiple Criteria Decision Making Theory and Applications , G. Fandel and T. Gal, Eds. Berlin: Springer-Verlag, 1980, pp. 468–486

    work page 1980

  27. [27]

    Boundary handling approaches in particle swarm optimization,

    N. Padhye, K. Deb, and P. Mittal, “Boundary handling approaches in particle swarm optimization,” in Pro- ceedings of Seventh International Conference on Bio- Inspired Computing: Theories and Applications (BIC-TA 2012). Advances in Intelligent Systems and Computing , J. Bansal, P. Singh, K. Deep, M. Pant, and A. Nagar, Eds., vol. 201. Springer, India, 2013...

    work page 2012

  28. [28]

    Comparison of multiobjective evolutionary algorithms: Empirical results,

    E. Zitzler, K. Deb, and L. Thiele, “Comparison of multiobjective evolutionary algorithms: Empirical results,” Evolutionary Computation , vol. 8, no. 2, pp. 173–195, 2000. [Online]. Available: https://doi.org/10. 1162/106365600568202

    work page 2000

  29. [29]

    A review of multiobjective test problems and a scalable test problem toolkit,

    S. Huband, P. Hingston, L. Barone, and L. While, “A review of multiobjective test problems and a scalable test problem toolkit,” IEEE Transactions on Evolutionary Computation, vol. 10, no. 5, pp. 477–506, 2006

    work page 2006

  30. [30]

    K. Deb, L. Thiele, M. Laumanns, and E. Zitzler, Scalable Test Problems for Evolutionary Multiobjective Optimization. London: Springer London, 2005, pp. 105–145. [Online]. Available: https://doi.org/10.1007/ 1-84628-137-7 6

    work page 2005

  31. [31]

    Performance comparison of nsga-ii and nsga-iii on various many-objective test problems,

    H. Ishibuchi, R. Imada, Y . Setoguchi, and Y . Nojima, “Performance comparison of nsga-ii and nsga-iii on various many-objective test problems,” in 2016 IEEE Congress on Evolutionary Computation (CEC) , July 2016, pp. 3045–3052

    work page 2016

  32. [32]

    Generating well-spaced points on a unit simplex for evolutionary many-objective optimization,

    J. Blank, K. Deb, Y . Dhebar, S. Bandaru, and H. Seada, “Generating well-spaced points on a unit simplex for evolutionary many-objective optimization,” IEEE Trans- actions on Evolutionary Computation , pp. 1–1, 2020

    work page 2020

  33. [33]

    The effect of feasible region on imbalanced problem in constrained multi-objective optimization,

    J. Lin, H. Liu, and C. Peng, “The effect of feasible region on imbalanced problem in constrained multi-objective optimization,” in 2017 13th International Conference on Computational Intelligence and Security (CIS), 2017, pp. 82–86

    work page 2017

  34. [34]

    Pymoo: Multi-objective optimiza- tion in python,

    J. Blank and K. Deb, “Pymoo: Multi-objective optimiza- tion in python,” IEEE Access, vol. 8, pp. 89 497–89 509, 2020

    work page 2020