pith. sign in

arxiv: 2605.25785 · v1 · pith:FC3QRJHGnew · submitted 2026-05-25 · 💻 cs.NE

A Scalable Benchmark Test Suite for Dynamic Multi-Objective Optimization with a Changing Number of Objectives

Ke Shang , Zhiyun Xiao , Yuxuan Liu , Jianguo Li , Shaojiang Wang , Wei Sun This is my paper

Pith reviewed 2026-06-29 19:25 UTC · model grok-4.3

classification 💻 cs.NE
keywords dynamic multi-objective optimizationbenchmark test suitechanging number of objectivesMinus-DTLZMinus-WFGdynamic subset selectionnon-stationary objective space
0
0 comments X

The pith

New benchmark suite keeps objective functions fixed while the number of active objectives changes over time.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Existing benchmark suites for dynamic multi-objective optimization alter the objective functions themselves when the number of objectives changes, which confounds efforts to test an algorithm's response to objective-count dynamics alone. The paper constructs a scalable alternative by starting from a fixed maximum-objective Minus-DTLZ or Minus-WFG problem and then dynamically selecting different subsets of those objectives as the active set. Because the underlying functions never change, any performance variation can be attributed to the changing objective count rather than to hidden function shifts. The Minus formulations ensure all objectives remain mutually conflicting, avoiding degeneracy that classical DTLZ and WFG problems can introduce. Representative algorithms evaluated on the new suite illustrate its flexibility for isolating this specific form of dynamism.

Core claim

By defining a maximum-objective problem and dynamically selecting subsets of objectives from fixed Minus-DTLZ and Minus-WFG formulations, the objective functions remain constant throughout the optimization process while only the number of active objectives varies, directly addressing the isolation failure in prior suites.

What carries the argument

Dynamic subset selection from a fixed maximum-objective Minus-DTLZ or Minus-WFG problem, which holds all objective functions constant while varying the active subset.

If this is right

  • Algorithms can be evaluated on their handling of objective-count dynamics in isolation from function changes.
  • The suite remains scalable because any number of active objectives up to the maximum can be obtained by subset selection.
  • Benchmark comparisons become cleaner because all algorithms face identical underlying functions at every time step.
  • Theoretical claims about handling non-stationary objective spaces no longer rest on violated assumptions about function constancy.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Designers of new dynamic algorithms may add explicit mechanisms to detect and respond to changes in objective cardinality without retraining on altered functions.
  • Re-running prior dynamic MOEAs on this suite could expose performance drops previously masked by the confounding function changes in older benchmarks.
  • The fixed-function construction could be extended to problems with time-varying objective correlations while still keeping the function definitions themselves stationary.

Load-bearing premise

Selecting subsets from the fixed maximum-objective problem preserves mutual conflict and non-degeneracy without introducing new implicit changes to the functions.

What would settle it

A demonstration that, under the new suite, switching the active objective subset produces Pareto-front or conflict changes that cannot be explained by the subset size alone.

Figures

Figures reproduced from arXiv: 2605.25785 by Jianguo Li, Ke Shang, Shaojiang Wang, Wei Sun, Yuxuan Liu, Zhiyun Xiao.

Figure 1
Figure 1. Figure 1: Illustration of normalized Pareto fronts of 3-objective DTLZ1 and Minus￾DTLZ1 and their projected Pareto fronts in f1-f2 space. 4.1 Problem Settings Based on the proposed benchmark construction, we consider three dynamic settings with different degrees of objective-number variation. All problems are constructed using the proposed Minus-DTLZ / Minus-WFG framework with a maximum number of objectives mmax. At… view at source ↗
Figure 2
Figure 2. Figure 2: Friedman ranking among MHV of obtained solutions by four algorithms under Setting I. Performance Under Moderate Objective Changes Under Setting II, where two objectives are added or removed at each change, the relative rank￾ing becomes more stable and the superiority of KTDMOEA and STA becomes more evident. Compared with Setting I, the MHV values of all algorithms de￾crease on many problems, especially on … view at source ↗
Figure 3
Figure 3. Figure 3: Friedman ranking among MHV of obtained solutions by four algorithms under Setting II [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Friedman ranking among MHV of obtained solutions by four algorithms under Setting III [PITH_FULL_IMAGE:figures/full_fig_p013_4.png] view at source ↗
read the original abstract

Dynamic multi-objective optimization with a changing number of objectives has recently attracted increasing attention due to its relevance to real-world problems whose evaluation criteria may evolve over time. However, existing benchmark test suites for this problem setting suffer from a fundamental limitation: when the number of objectives changes, the objective functions themselves also change implicitly. This makes it difficult to isolate and evaluate an algorithm's capability to handle dynamics in the number of objectives alone. In this paper, we analyze this issue in detail and show that several theoretical properties claimed in prior studies rely on an assumption that is violated by commonly used test suites. To address this problem, we propose a scalable benchmark test suite in which the objective functions are fixed throughout the optimization process, while the number of active objectives changes over time. Our benchmark is constructed by defining a maximum-objective problem and dynamically selecting subsets of objectives. To avoid degeneracy issues in classical DTLZ and WFG problems, we adopt Minus-DTLZ and Minus-WFG formulations, in which all objectives are mutually conflicting. Extensive benchmark studies using representative algorithms from the literature demonstrate the usefulness and flexibility of the proposed test suite.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 2 minor

Summary. The paper identifies a limitation in existing benchmark suites for dynamic multi-objective optimization with a changing number of objectives: altering the objective count implicitly changes the objective functions themselves, preventing isolation of the number-of-objectives dynamic. It analyzes how this violates assumptions underlying theoretical properties in prior work and proposes a new scalable suite constructed via dynamic subset selection from a fixed maximum-objective Minus-DTLZ or Minus-WFG problem (where all objectives are mutually conflicting). Only the active objective count varies while the underlying functions remain constant; extensive experiments with representative algorithms are used to demonstrate the suite's usefulness and flexibility.

Significance. If the subset-selection construction from Minus-DTLZ/Minus-WFG indeed preserves mutual conflict, non-degeneracy, and isolation of the dynamic without introducing new implicit changes, the benchmark supplies a controlled testbed that directly targets a recognized gap. This would enable more precise evaluation of algorithm behavior under objective-count dynamics, supporting better algorithm development for applications where evaluation criteria evolve.

minor comments (2)
  1. [Abstract] The abstract states that the new construction 'addresses the limitation of prior suites' and that 'several theoretical properties claimed in prior studies rely on an assumption that is violated'; a concrete example of one such violated property (with reference to a specific prior suite) would strengthen the motivation section.
  2. The description of the dynamic subset selection mechanism should include explicit pseudocode or a formal definition of how subsets are chosen at each time step to ensure reproducibility across implementations.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive summary, recognition of the significance of isolating objective-count dynamics, and recommendation for minor revision. No specific major comments were provided in the report.

Circularity Check

0 steps flagged

No significant circularity; constructive benchmark proposal is self-contained

full rationale

The paper's central contribution is an explicit construction: define a fixed maximum-objective Minus-DTLZ/Minus-WFG instance (all objectives mutually conflicting by design) and dynamically select subsets to vary only the active objective count. This directly targets the stated limitation of prior suites without any derivation reducing to fitted parameters, self-referential definitions, or load-bearing self-citations. The mutual-conflict and non-degeneracy properties are inherited from the Minus formulations as stated, and the proposal's validity is externally checkable via algorithm performance on the described problems. No steps match the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The proposal rests on standard domain assumptions of multi-objective optimization (Pareto dominance, mutual conflict of objectives) and the mathematical properties of the Minus-DTLZ and Minus-WFG formulations; no free parameters or invented entities are introduced in the abstract.

axioms (2)
  • domain assumption Minus-DTLZ and Minus-WFG formulations ensure all objectives are mutually conflicting and avoid degeneracy issues present in classical DTLZ/WFG.
    Invoked when the paper states it adopts these formulations 'to avoid degeneracy issues'.
  • domain assumption Dynamic subset selection from a fixed maximum-objective problem isolates changes in objective count without implicit changes to the objective functions.
    Central premise of the benchmark construction described in the abstract.

pith-pipeline@v0.9.1-grok · 5744 in / 1477 out tokens · 26640 ms · 2026-06-29T19:25:36.438629+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

18 extracted references · 18 canonical work pages

  1. [1]

    Surveys in Operations Research and Management Science20(2), 35–42 (2015)

    Chand, S., Wagner, M.: Evolutionary many-objective optimization: A quick-start guide. Surveys in Operations Research and Management Science20(2), 35–42 (2015). https://doi.org/10.1016/j.sorms.2015.08.001

    work page doi:10.1016/j.sorms.2015.08.001 2015

  2. [2]

    IEEE Transactions on Evolutionary Computation22(1), 157–171 (2018)

    Chen, R., Li, K., Yao, X.: Dynamic multiobjectives optimization with a changing number of objectives. IEEE Transactions on Evolutionary Computation22(1), 157–171 (2018). https://doi.org/10.1109/TEVC.2017.2669638

    work page doi:10.1109/tevc.2017.2669638 2018

  3. [3]

    In: Evolutionary Multiobjective Opti- mization: Theoretical Advances and Applications, pp

    Deb, K., Thiele, L., Laumanns, M., Zitzler, E.: Scalable test problems for evolutionary multiobjective optimization. In: Evolutionary Multiobjective Opti- mization: Theoretical Advances and Applications, pp. 105–145. Springer (2005). https://doi.org/10.1007/1-84628-137-7_6

    work page doi:10.1007/1-84628-137-7_6 2005

  4. [4]

    IEEE Transactions on Evolutionary Computation8(5), 425–442 (2004)

    Farina, M., Deb, K., Amato, P.: Dynamic multiobjective optimization problems: Test cases, approximations, and applications. IEEE Transactions on Evolutionary Computation8(5), 425–442 (2004). https://doi.org/10.1109/TEVC.2004.831456

    work page doi:10.1109/tevc.2004.831456 2004

  5. [5]

    In: Evolutionary Multi-Criterion Optimization

    Huband, S., Barone, L., While, L., Hingston, P.: A scalable multi-objective test problem toolkit. In: Evolutionary Multi-Criterion Optimization. Lec- ture Notes in Computer Science, vol. 3410, pp. 280–295. Springer (2005). https://doi.org/10.1007/978-3-540-31880-4_20

    work page doi:10.1007/978-3-540-31880-4_20 2005

  6. [6]

    IEEE Transactions on Evolutionary Computation21(2), 169–190 (2017)

    Ishibuchi, H., Setoguchi, Y., Masuda, H., Nojima, Y.: Performance of decomposition-based many-objective algorithms strongly depends on pareto front shapes. IEEE Transactions on Evolutionary Computation21(2), 169–190 (2017). https://doi.org/10.1109/TEVC.2016.2587749

    work page doi:10.1109/tevc.2016.2587749 2017

  7. [7]

    In: 2008 IEEE Congress on Evolutionary Computation

    Ishibuchi, H., Tsukamoto, N., Nojima, Y.: Evolutionary many-objective optimiza- tion: A short review. In: 2008 IEEE Congress on Evolutionary Computation. pp. 2419–2426. IEEE (2008). https://doi.org/10.1109/CEC.2008.4631121

    work page doi:10.1109/cec.2008.4631121 2008

  8. [8]

    IEEE Transactions on Cybernet- ics50(6), 2814–2826 (2020)

    Jiang, S., Kaiser, M., Yang, S., Kollias, S., Krasnogor, N.: A scalable test suite for continuous dynamic multiobjective optimization. IEEE Transactions on Cybernet- ics50(6), 2814–2826 (2020). https://doi.org/10.1109/TCYB.2019.2896021

    work page doi:10.1109/tcyb.2019.2896021 2020

  9. [9]

    ACM Computing Surveys55(4), 76:1–76:47 (2023)

    Jiang, S., Zou, J., Yang, S., Yao, X.: Evolutionary dynamic multi-objective optimisation: A survey. ACM Computing Surveys55(4), 76:1–76:47 (2023). https://doi.org/10.1145/3524495

    work page doi:10.1145/3524495 2023

  10. [10]

    In: 2007 IEEE Congress on Evolutionary Computation

    Li, X., Branke, J., Kirley, M.: On performance metrics and particle swarm methods for dynamic multiobjective optimization problems. In: 2007 IEEE Congress on Evolutionary Computation. pp. 576–583. IEEE (2007). https://doi.org/10.1109/CEC.2007.4424522

    work page doi:10.1109/cec.2007.4424522 2007

  11. [11]

    ZTE Communications15(3), 30–42 (2017)

    Liao, H., Wang, W., Huang, H., Zhang, L., Chen, Y., Wang, J.: Evolutionary algorithms in software defined networks: Techniques, applications, and issues. ZTE Communications15(3), 30–42 (2017). https://doi.org/10.3969/j.issn.1673- 5188.2017.03.004

    work page doi:10.3969/j.issn.1673- 2017

  12. [12]

    Swarm and Evolutionary Computation6, 1–24 (2012)

    Nguyen, T.T., Yang, S., Branke, J.: Evolutionary dynamic optimization: A survey of the state of the art. Swarm and Evolutionary Computation6, 1–24 (2012). https://doi.org/10.1016/j.swevo.2012.05.001

    work page doi:10.1016/j.swevo.2012.05.001 2012

  13. [13]

    IEEE Transactions on Evolutionary Com- putation29(5), 1531–1545 (2025)

    Ruan, G., Hou, Z., Yao, X.: Coping with a severely changing number of objectives in dynamic multiobjective optimization. IEEE Transactions on Evolutionary Com- putation29(5), 1531–1545 (2025). https://doi.org/10.1109/TEVC.2025.3558987

    work page doi:10.1109/tevc.2025.3558987 2025

  14. [14]

    IEEE Transactions on Emerging Topics in Computational Intelligence8(6), 4210–4224 (2024)

    Ruan, G., Minku, L.L., Menzel, S., Sendhoff, B., Yao, X.: Knowledge transfer for dynamic multiobjective optimization with a changing number of objectives. IEEE Transactions on Emerging Topics in Computational Intelligence8(6), 4210–4224 (2024). https://doi.org/10.1109/TETCI.2024.3389769 16 K. Shang et al

    work page doi:10.1109/tetci.2024.3389769 2024

  15. [15]

    IEEE Transactions on Evolutionary Computation29(4), 865–879 (2025)

    Ruan, G., Minku, L.L., Menzel, S., Sendhoff, B., Yao, X.: Learning to ex- pand/contract pareto sets in dynamic multiobjective optimization with a changing number of objectives. IEEE Transactions on Evolutionary Computation29(4), 865–879 (2025). https://doi.org/10.1109/TEVC.2024.3375751

    work page doi:10.1109/tevc.2024.3375751 2025

  16. [16]

    A fast way of calculating exact hy- pervolumes.IEEE Transactions on Evolutionary Computation, 16(1):86–95, 2012

    While, L., Bradstreet, L., Barone, L.: A fast way of calculating exact hyper- volumes. IEEE Transactions on Evolutionary Computation16(1), 86–95 (2012). https://doi.org/10.1109/TEVC.2010.2077298

    work page doi:10.1109/tevc.2010.2077298 2012

  17. [17]

    ZTE Communications19(4), 98–104 (2021)

    Zhang, C., Zhang, N., Cao, W., Tian, K., Yang, Z.: Ai-based optimization of han- dover strategy in non-terrestrial networks. ZTE Communications19(4), 98–104 (2021). https://doi.org/10.12142/ZTECOM.202104011

    work page doi:10.12142/ztecom.202104011 2021

  18. [18]

    Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach

    Zitzler, E., Thiele, L.: Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach. IEEE Transactions on Evolutionary Com- putation3(4), 257–271 (1999). https://doi.org/10.1109/4235.797969

    work page doi:10.1109/4235.797969 1999