pith. sign in

arxiv: 2604.08173 · v2 · submitted 2026-04-09 · 💻 cs.NE

Exploration of Pareto-preserving Search Space Transformations in Multi-objective Test Functions

Diederick Vermetten , Jeroen Rook This is my paper

Pith reviewed 2026-05-10 17:49 UTC · model grok-4.3

classification 💻 cs.NE
keywords multi-objective optimizationbenchmark problemssearch space transformationsPareto front preservationbijective mappingsalgorithm performance evaluation
0
0 comments X

The pith

Parameterized bijective transformations applied to the search space of standard multi-objective test problems change how well different optimization algorithms perform without altering the objective space structure.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper demonstrates that search space transformations can be added to multi-objective benchmark problems while preserving the Pareto front and boundary constraints. It applies two specific parameterized bijective transformations to popular test functions and measures the resulting shifts in performance across multiple algorithms. These transformations address the relative lack of attention to search space structure in multi-objective problem design compared to objective space features. The work also applies similar transformations directly to the objective space to compare their effects. This approach mirrors fixes used in single-objective optimization to reduce unintended biases that algorithms might exploit.

Core claim

We utilized two parameterized, bijective transformations to create different instantiations of popular benchmark problems, and show how these changes impact the performance of various multi-objective optimization algorithms. In addition to the search space transformations, we show that such parameterized transformations can also be applied to the objective space, and compare their respective performance impacts.

What carries the argument

Parameterized bijective transformations that map the search space while preserving Pareto optimality and problem boundaries.

If this is right

  • Standard multi-objective benchmarks may produce algorithm rankings that do not generalize to other valid search space configurations with the same objective space structure.
  • Search space transformations offer a controlled way to generate families of related test problems that share the same Pareto front.
  • Performance differences between search space and objective space transformations can be measured directly on the same base problems.
  • Algorithms that maintain consistent behavior across transformed instantiations are less likely to rely on specific search space features.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Benchmark suites could incorporate multiple transformed variants of each problem to test whether reported performance holds under varied search spaces.
  • This method provides a template for creating more robust test sets in other optimization settings where search space biases have been an issue.
  • If an algorithm's performance changes sharply under these mappings, it suggests sensitivity to particular search space geometries that may not appear in real applications.

Load-bearing premise

The chosen bijective transformations alter search-space difficulty for algorithms without introducing new unintended biases or violating the boundary constraints of the original problems.

What would settle it

Running the tested algorithms on both the original and transformed versions of the benchmarks and observing that their relative performance rankings and success rates stay identical.

Figures

Figures reproduced from arXiv: 2604.08173 by Diederick Vermetten, Jeroen Rook.

Figure 1
Figure 1. Figure 1: The impact of the Beta-CDF based transformation [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The different steps of the rotation-based transforma [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Wasserstein distance between pairwise distances [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Scatter plots of the non-dominated solutions found by RandomSearch on 2-dimensional DTLZ1 for 5 different [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Normalized hypervolume over time (calculated over the unbounded archive) for the 4 used algorithms (population [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Final hypervolume of the unbounded archive from [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: Final hypervolume of the archive from each of the [PITH_FULL_IMAGE:figures/full_fig_p007_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Average relative hypervolume (relative to the un [PITH_FULL_IMAGE:figures/full_fig_p007_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Averaged relative hypervolume (relative to the untransformed problems) for the 3 different transformation methods [PITH_FULL_IMAGE:figures/full_fig_p008_10.png] view at source ↗
read the original abstract

Benchmark problems are an important tool for gaining understanding of optimization algorithms. Since algorithms often aim to perform well on benchmarks, biases in benchmark design provide misleading insights. In single-objective optimization, for example, many problems used to have their optimum in the center of the search domain. To remedy these issues, search space transformations have been widely adopted by benchmark suites, preventing algorithms from exploiting unintended structure. In multi-objective optimization, problem design has focused primarily on the objective space structure. While this focus addresses important aspects of the multi-objective nature of the problems, the search space structures of these problems have received comparatively limited attention. In this work, we re-emphasize the importance of transformations in the search space, and address the challenges inherent in adding transformations to boundary constraints problems without impacting the structure of the objective space. We utilized two parameterized, bijective transformations to create different instantiations of popular benchmark problems, and show how these changes impact the performance of various multi-objective optimization algorithms. In addition to the search space transformations, we show that such parameterized transformations can also be applied to the objective space, and compare their respective performance impacts.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes two parameterized bijective transformations applicable to the search space (and separately to the objective space) of standard multi-objective benchmark problems. These transformations are designed to preserve the Pareto structure and boundary constraints while altering search-space difficulty; the central empirical claim is that the resulting problem instances produce measurable changes in the observed performance of several multi-objective optimization algorithms, thereby highlighting the need for greater attention to search-space structure in benchmark design.

Significance. If the transformations are verifiably bijective, Pareto-preserving, and boundary-compliant, and if the reported performance differences are statistically robust, the work would usefully extend single-objective benchmark practices to the multi-objective setting. It supplies a concrete mechanism for generating controlled variants of existing test suites without introducing new objective-space biases, which could improve the diagnostic value of algorithm comparisons.

major comments (2)
  1. The experimental section must specify the exact benchmark functions employed (e.g., ZDT, DTLZ, WFG), the precise mathematical definitions of the two transformations (including how bijectivity and boundary preservation are enforced), the number of independent runs, and the statistical tests used to establish that performance changes are significant rather than attributable to sampling variability.
  2. The claim that objective-space transformations produce distinct performance impacts relative to search-space transformations requires direct side-by-side tables or figures with effect-size measures; without these, the comparative conclusion remains under-supported.
minor comments (2)
  1. Notation for the transformation parameters should be introduced once and used consistently; the abstract refers to 'parameterized' transformations but does not name the parameters.
  2. Figure captions should explicitly state which transformation (search-space or objective-space) is illustrated and which algorithm is being evaluated.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments, which identify opportunities to improve the reproducibility and evidential support of our work. We address each major comment below and will incorporate the requested enhancements in the revised manuscript.

read point-by-point responses

  1. Referee: The experimental section must specify the exact benchmark functions employed (e.g., ZDT, DTLZ, WFG), the precise mathematical definitions of the two transformations (including how bijectivity and boundary preservation are enforced), the number of independent runs, and the statistical tests used to establish that performance changes are significant rather than attributable to sampling variability.

    Authors: We agree that these specifications are necessary for full reproducibility and will expand the experimental section accordingly. The revised manuscript will explicitly enumerate the benchmark functions (ZDT1–6, DTLZ1–7, and WFG1–9), supply the complete mathematical definitions of both parameterized transformations together with arguments establishing bijectivity (via explicit inverse mappings) and boundary preservation (via domain-consistent scaling), state that 30 independent runs were executed per configuration, and detail the statistical procedure (Wilcoxon rank-sum tests with Holm–Bonferroni correction at α = 0.05) used to verify that observed performance differences exceed sampling variability. revision: yes

  2. Referee: The claim that objective-space transformations produce distinct performance impacts relative to search-space transformations requires direct side-by-side tables or figures with effect-size measures; without these, the comparative conclusion remains under-supported.

    Authors: We accept that direct quantitative comparison with effect sizes would strengthen the distinction between the two transformation types. The revision will include new side-by-side tables and figures that juxtapose performance metrics (hypervolume and IGD) for search-space versus objective-space transformations on the same problem instances, augmented with effect-size statistics (Cohen’s d and Cliff’s delta) to quantify the magnitude and distinctness of the observed impacts. revision: yes

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The paper is an empirical study that applies two parameterized bijective transformations to the search and objective spaces of existing multi-objective benchmark problems, then measures resulting changes in algorithm performance. No load-bearing derivations, equations, or uniqueness claims are present that reduce by construction to fitted parameters, self-definitions, or self-citation chains; the transformations are introduced as explicit constructions that preserve Pareto fronts and boundaries, with all claims resting on experimental comparisons rather than internal redefinitions.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The central claim rests on experimental comparisons rather than derivations; the main assumptions are that the transformations remain bijective and boundary-respecting while only changing search-space structure.

free parameters (1)
  • transformation parameters
    Parameters that define the specific instantiations of the two bijective transformations applied to the benchmark problems.
axioms (1)
  • domain assumption The transformations are bijective and preserve the structure of the objective space (Pareto front) while respecting boundary constraints.
    Invoked to ensure changes affect only search-space difficulty without altering the multi-objective nature of the problems.

pith-pipeline@v0.9.0 · 5494 in / 1270 out tokens · 42848 ms · 2026-05-10T17:49:51.137750+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 35 canonical work pages

  1. [1]

    Nicola Beume, Boris Naujoks, and Michael Emmerich. 2007. SMS-EMOA: Mul- tiobjective selection based on dominated hypervolume.European journal of operational research181, 3 (2007), 1653–1669

    work page 2007

  2. [2]

    Julian Blank and Kalyanmoy Deb. 2020. Pymoo: Multi-objective optimization in python.Ieee access8 (2020), 89497–89509

    work page 2020

  3. [3]

    Dimo Brockhoff, Anne Auger, Nikolaus Hansen, and Tea Tušar. 2022. Using Well-Understood Single-Objective Functions in Multiobjective Black-Box Opti- mization Test Suites. (2022), 165–193. doi:10.1162/evco_a_00298

    work page doi:10.1162/evco_a_00298 2022

  4. [4]

    Arthur Gonçalves Carvalho and Aluizio F.R. Araujo. 2009. Improving NSGA-II with an Adaptive Mutation Operator. InProceedings of the 11th Annual Conference Companion on Genetic and Evolutionary Computation Conference: Late Breaking Papers(New York, NY, USA, 2009-07-08)(GECCO ’09). Association for Computing Machinery, 2697–2700. doi:10.1145/1570256.1570387

    work page doi:10.1145/1570256.1570387 2009

  5. [5]

    Ran Cheng, Miqing Li, Ye Tian, Xingyi Zhang, Shengxiang Yang, Yaochu Jin, and Xin Yao. 2017. A Benchmark Test Suite for Evolutionary Many-Objective Optimization. 3, 1 (2017), 67–81. doi:10.1007/s40747-017-0039-7 Pareto-preserving Search Space Transformations in MOO

    work page doi:10.1007/s40747-017-0039-7 2017

  6. [6]

    Jacob de Nobel, Furong Ye, Diederick Vermetten, Hao Wang, Carola Doerr, and Thomas Bäck. 2024. Iohexperimenter: Benchmarking platform for iterative optimization heuristics.Evolutionary Computation32, 3 (2024), 205–210

    work page 2024

  7. [7]

    Kalyanmoy Deb. 1999. Multi-objective Genetic Algorithms: Problem Difficulties and Construction of Test Problems. 7, 3 (1999), 205–230. doi:10.1162/evco.1999. 7.3.205

    work page doi:10.1162/evco.1999 1999

  8. [8]

    Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and TAMT Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II.IEEE transactions on evolutionary computation6, 2 (2002), 182–197

    work page 2002

  9. [9]

    Kalyanmoy Deb, Lothar Thiele, Marco Laumanns, and Eckart Zitzler. 2005. Scal- able Test Problems for Evolutionary Multiobjective Optimization. InEvolutionary Multiobjective Optimization. Springer-Verlag, 105–145. doi:10.1007/1-84628-137- 7_6

    work page doi:10.1007/1-84628-137- 2005

  10. [10]

    Benjamin Doerr and Weijie Zheng. 2021. Theoretical Analyses of Multi-Objective Evolutionary Algorithms on Multi-Modal Objectives. 35, 14 (2021), 12293–12301

    work page 2021

  11. [11]

    Coello Coello

    Jesus Guillermo Falcon-Cardona, Hisao Ishibuchi, and Carlos A. Coello Coello

    work page

  12. [12]

    A continuous information gain measure to find the most discriminatory problems for ai benchmarking

    Riesz S-Energy-Based Reference Sets for Multi-Objective Optimization. In2020 IEEE Congress on Evolutionary Computation (CEC)(Glasgow, United Kingdom, 2020-07). IEEE, 1–8. doi:10.1109/CEC48606.2020.9185833

    work page doi:10.1109/cec48606.2020.9185833 2020

  13. [13]

    Nikolaus Hansen, Anne Auger, Raymond Ros, Olaf Mersmann, Tea Tušar, and Dimo Brockhoff. 2021. COCO: A platform for comparing continuous optimizers in a black-box setting.Optimization Methods and Software36, 1 (2021), 114–144

    work page 2021

  14. [14]

    2009.Real- parameter black-box optimization benchmarking 2009: Noiseless functions defini- tions

    Nikolaus Hansen, Steffen Finck, Raymond Ros, and Anne Auger. 2009.Real- parameter black-box optimization benchmarking 2009: Noiseless functions defini- tions. Ph. D. Dissertation. INRIA

    work page 2009

  15. [15]

    Huband, P

    S. Huband, P. Hingston, L. Barone, and L. While. 2006. A review of multiobjective test problems and a scalable test problem toolkit. 10, 5 (2006), 477–506. doi:10. 1109/TEVC.2005.861417

    work page arXiv 2006

  16. [16]

    Christian Igel, Nikolaus Hansen, and Stefan Roth. 2007. Covariance matrix adaptation for multi-objective optimization.Evolutionary computation15, 1 (2007), 1–28

    work page 2007

  17. [17]

    Iorio and Xiaodong Li

    Antony W. Iorio and Xiaodong Li. 2006. Rotated Test Problems for Assessing the Performance of Multi-Objective Optimization Algorithms. InProceedings of the 8th Annual Conference on Genetic and Evolutionary Computation(New York, NY, USA, 2006-07-08)(GECCO ’06). Association for Computing Machinery, 683–690. doi:10.1145/1143997.1144118

    work page doi:10.1145/1143997.1144118 2006

  18. [18]

    Angus Kenny, Tapabrata Ray, and Hemant Singh. 2025. Multi-objective L-shaped Test Functions. InProceedings of the Genetic and Evolutionary Computation Con- ference. 22–29

    work page 2025

  19. [19]

    Jakub Kudela. 2022. A critical problem in benchmarking and analysis of evolution- ary computation methods.Nature Machine Intelligence4, 12 (2022), 1238–1245

    work page 2022

  20. [20]

    Miqing Li, Crina Grosan, Shengxiang Yang, Xiaohui Liu, and Xin Yao. 2018. Multiline Distance Minimization: A Visualized Many-Objective Test Problem Suite. 22, 1 (2018), 61–78. doi:10.1109/TEVC.2017.2655451

    work page doi:10.1109/tevc.2017.2655451 2018

  21. [21]

    Hiroyuki Masuda, Yusuke Nojima, and Hisao Ishibuchi. 2016. Common Properties of Scalable Multiobjective Problems and a New Framework of Test Problems. In 2016 IEEE Congress on Evolutionary Computation (CEC). 3011–3018. doi:10.1109/ CEC.2016.7744170

    work page arXiv 2016

  22. [22]

    Shuaiqun Pan, Diederick Vermetten, Manuel López-Ibáñez, Thomas Bäck, and Hao Wang. 2025. Transfer Learning of Surrogate Models: Integrating Domain Warping and Affine Transformations. InProceedings of the Genetic and Evolu- tionary Computation Conference Companion. 2481–2490

    work page 2025

  23. [23]

    2015.Multimodal Optimization by Means of Evolutionary Algorithms

    Mike Preuss. 2015.Multimodal Optimization by Means of Evolutionary Algorithms. Springer International Publishing. doi:10.1007/978-3-319-07407-8

    work page doi:10.1007/978-3-319-07407-8 2015

  24. [24]

    Jeroen Rook, Heike Trautmann, Jakob Bossek, and Christian Grimme. 2022. On the Potential of Automated Algorithm Configuration on Multi-Modal Multi- Objective Optimization Problems. InProceedings of the Genetic and Evolutionary Computation Conference Companion(Boston Massachusetts, 2022-07-09). ACM, 356–359. doi:10.1145/3520304.3528998

    work page doi:10.1145/3520304.3528998 2022

  25. [25]

    Lennart Schäpermeier, Pascal Kerschke, Christian Grimme, and Heike Traut- mann. 2023. Peak-A-Boo! Generating Multi-objective Multiple Peaks Benchmark Problems with Precise Pareto Sets. InEvolutionary Multi-Criterion Optimization. Vol. 13970. Springer Nature Switzerland, 291–304. doi:10.1007/978-3-031-27250- 9_21

    work page doi:10.1007/978-3-031-27250- 2023

  26. [26]

    Jasper Snoek, Kevin Swersky, Rich Zemel, and Ryan Adams. 2014. Input warping for Bayesian optimization of non-stationary functions. InInternational conference on machine learning. PMLR, 1674–1682

    work page 2014

  27. [27]

    Diederick Vermetten and Jeroen Rook. 2026. Reproducibility and additional files. Zenodo(2026). doi:10.5281/zenodo.18335350

    work page doi:10.5281/zenodo.18335350 2026

  28. [28]

    Diederick Vermetten, Jeroen Rook, Oliver L Preuß, Jacob de Nobel, Carola Do- err, Manuel López-Ibañez, Heike Trautmann, and Thomas Bäck. 2025. MO- IOHinspector: Anytime benchmarking of multi-objective algorithms using IOH- profiler. InInternational Conference on Evolutionary Multi-Criterion Optimization. Springer, 242–256

    work page 2025

  29. [29]

    Diederick Vermetten, Bas van Stein, Fabio Caraffini, Leandro L Minku, and Anna V Kononova. 2022. Bias: A toolbox for benchmarking structural bias in the continuous domain.IEEE Transactions on Evolutionary Computation26, 6 (2022), 1380–1393

    work page 2022

  30. [30]

    Pauli Virtanen, Ralf Gommers, Travis E Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, et al . 2020. SciPy 1.0: fundamental algorithms for scientific computing in Python.Nature methods17, 3 (2020), 261–272

    work page 2020

  31. [31]

    Vanessa Volz, Dani Irawan, Koen van der Blom, and Boris Naujoks. 2023. A short overview and common pitfalls of benchmarking evolutionary algorithms. In Many-Criteria Optimization and Decision Analysis. Springer

    work page 2023

  32. [32]

    Haoran Yin, Diederick Vermetten, Furong Ye, Thomas HW Bäck, and Anna V Kononova. 2024. Impact of spatial transformations on landscape features of cec2022 basic benchmark problems.arXiv preprint arXiv:2402.07654(2024)

    work page arXiv 2024

  33. [33]

    Caitong Yue, Boyang Qu, Kunjie Yu, Jing Liang, and Xiaodong Li. 2019. A Novel Scalable Test Problem Suite for Multimodal Multiobjective Optimization. 48 (2019), 62–71. doi:10.1016/j.swevo.2019.03.011

    work page doi:10.1016/j.swevo.2019.03.011 2019

  34. [34]

    Qingfu Zhang and Hui Li. 2007. MOEA/D: A multiobjective evolutionary algo- rithm based on decomposition.IEEE Transactions on evolutionary computation 11, 6 (2007), 712–731

    work page 2007

  35. [35]

    Eckart Zitzler, Kalyanmoy Deb, and Lothar Thiele. 2000. Comparison of Mul- tiobjective Evolutionary Algorithms: Empirical Results. 8, 2 (2000), 173–195. doi:10.1162/106365600568202

    work page doi:10.1162/106365600568202 2000