pith. sign in

arxiv: 2509.16478 · v2 · pith:TFY6WASPnew · submitted 2025-09-20 · 💻 cs.SE

Constrained Co-evolutionary Metamorphic Differential Testing for Autonomous Systems with an Interpretability Approach

Hossein Yousefizadeh , Shenghui Gu , Lionel C. Briand , Ali Nasr This is my paper

Pith reviewed 2026-05-21 22:29 UTC · model grok-4.3

classification 💻 cs.SE
keywords autonomous systems testingmetamorphic relationsco-evolutionary algorithmsdifferential testingsearch-based software engineeringinterpretabilityregression testingCarla simulator
0
0 comments X

The pith

CoCoMagic detects 287% more high-severity changes across ADS versions

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

CoCoMagic is a test generation method that combines metamorphic testing with constrained co-evolutionary search to find behavioral differences between successive versions of autonomous systems. It evolves source scenarios and perturbations jointly to increase the number of high-severity divergences in metamorphic relations while enforcing realism constraints. This matters for developers who must verify that updates do not introduce safety degradations in complex systems where traditional testing oracles do not exist. The method further includes an interpretability component that helps explain the sources of the detected differences.

Core claim

CoCoMagic formulates test generation as a constrained cooperative co-evolutionary search, evolving both source scenarios and metamorphic perturbations to maximize differences in violations of predefined metamorphic relations across versions of autonomous systems. Constraints and population initialization strategies guide the search toward realistic, relevant scenarios. An integrated interpretability approach aids in diagnosing the root causes of divergences. Evaluation on an end-to-end ADS, InterFuser, within the Carla virtual simulator shows significant improvements, identifying up to 287% more distinct high-severity behavioral differences while maintaining scenario realism.

What carries the argument

Constrained cooperative co-evolutionary search that jointly evolves source scenarios and metamorphic perturbations to maximize cross-version differences in metamorphic-relation violations.

If this is right

  • More distinct high-severity behavioral differences are identified than with baseline search methods.
  • Scenario realism is preserved during the search process.
  • Interpretability provides actionable insights for debugging version changes.
  • The approach supports efficient differential testing of evolving autonomous systems.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar co-evolutionary techniques might improve testing in other domains lacking oracles, such as reinforcement learning agents.
  • Effectiveness depends on the quality and completeness of the initial metamorphic relations chosen by the user.
  • Results from the Carla simulator could be validated against physical vehicle tests to confirm transferability.

Load-bearing premise

The predefined metamorphic relations and severity definitions are assumed to capture all important safety-relevant differences between versions.

What would settle it

A follow-up experiment that applies CoCoMagic and the baselines to a new version update with independently documented high-severity faults; if the method fails to surface a significantly larger fraction of those faults, its advantage is disproved.

Figures

Figures reproduced from arXiv: 2509.16478 by Ali Nasr, Hossein Yousefizadeh, Lionel C. Briand, Shenghui Gu.

Figure 1
Figure 1. Figure 1: Overview of CoCoMagic. Algorithm 2: assessFitness Input: Population of scenarios Ps Population of perturbations Pq Archive of scenarios Xs Archive of perturbations Xq Archive of complete solutions X Dissimilarity threshold θ Execution scenarios S Output: Updated population of scenarios Ps Updated population of perturbations Pq Updated archive of complete solutions X 1 Set of complete solutions CS ← collabo… view at source ↗
Figure 2
Figure 2. Figure 2: Distinct Solutions (DS) vs. distance threshold (θd) across different fitness thresholds (θf ). A higher DS value indicates more distinct test cases discovered by the method. Each curve plots the average DS value at different θd settings, under a specific fitness threshold θf . This figure reveals how each method balances the quantity and diversity of test cases as θf varies. values. Specifically, we analyz… view at source ↗
Figure 3
Figure 3. Figure 3: Fitness distribution of test cases generated by [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Distinct Solutions (DS) progression across simulation budget levels. Each subplot illustrates how the DS value grows as more simulation resources are consumed, under specific combinations of fitness thresholds (θf ) and distance thresholds (θd). The curves represent different methods, allowing a direct comparison of how quickly and effectively each approach uncovers severe and diverse behavioral discrepanc… view at source ↗
Figure 5
Figure 5. Figure 5: Distribution of execution times (in hours) for [PITH_FULL_IMAGE:figures/full_fig_p016_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Distinct Solutions (DS) vs. distance threshold (θd) across different fitness thresholds (θf ) for different configurations of CoCoMagic. A higher DS value indicates more distinct test cases discovered by the method. Each curve plots the average DS value at different θd settings, under a specific fitness threshold θf . This figure reveals how each method balances the quantity and diversity of test cases as … view at source ↗
Figure 8
Figure 8. Figure 8: Average fitness of test cases generated by different configurations of [PITH_FULL_IMAGE:figures/full_fig_p018_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Distribution of Mean Absolute Error (MAE) for RuleFit in CoCoMagic and baseline methods. A lower MAE value indicates higher accuracy of the model in capturing behavioral differences between the two system versions. Individual points beside each box represent the MAE values for each identified test case. 2) Results [PITH_FULL_IMAGE:figures/full_fig_p019_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Distribution of the support of the rules extracted by RuleFit in CoCoMagic. Higher support values indicate that the rules cover more complete solutions. The left subplot shows the distribution across all 500 runs, with horizontal jitter added to avoid overlapping points and improve visibility. The right subplot shows the same data as a histogram with a logarithmic scale for better visualization [PITH_FUL… view at source ↗
Figure 11
Figure 11. Figure 11: An example scenario satisfying a representative rule extracted by [PITH_FULL_IMAGE:figures/full_fig_p020_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Distinct Solutions (DS) vs. distance threshold (θd) across different fitness thresholds (θf ). A higher DS value indicates that more distinct test cases have been discovered by the method. Each curve plots the average DS value at different θd settings, under a specific fitness threshold θf . This figure reveals how each method balances the quantity and diversity of test cases as θf varies [PITH_FULL_IMAG… view at source ↗
Figure 13
Figure 13. Figure 13: illustrates the progression of DS as the simulation budget increases for GP2 . Each subplot corresponds to a specific combination of θf and θd , with distinct curves for CoCoMagic, SGA, and RS. Similar to GP1, the plots suggest differences in efficiency as methods uncover severe and diverse behavioral discrepancies over time. 10 20 30 40 50 60 70 80 90 100 Simulation budget (%) 1 2 A v e r a g e D S Fitne… view at source ↗
Figure 14
Figure 14. Figure 14: Distribution of execution times (in hours) for [PITH_FULL_IMAGE:figures/full_fig_p027_14.png] view at source ↗
read the original abstract

Autonomous systems, such as autonomous driving systems, evolve rapidly through frequent updates, risking unintended behavioral degradations. Effective system-level testing is challenging due to the vast scenario space, the absence of reliable test oracles, and the need for practically applicable and interpretable test cases. We present CoCoMagic, a novel automated test case generation method that combines metamorphic testing, differential testing, and advanced search-based techniques to identify behavioral divergences between versions of autonomous systems. CoCoMagic formulates test generation as a constrained cooperative co-evolutionary search, evolving both source scenarios and metamorphic perturbations to maximize differences in violations of predefined metamorphic relations across versions. Constraints and population initialization strategies guide the search toward realistic, relevant scenarios. An integrated interpretability approach aids in diagnosing the root causes of divergences. We evaluate CoCoMagic on an end-to-end ADS, InterFuser, within the Carla virtual simulator. Results show significant improvements over baseline search methods, identifying up to 287\% more distinct high-severity behavioral differences while maintaining scenario realism. The interpretability approach provides actionable insights for developers, supporting targeted debugging and safety assessment. CoCoMagic offers an efficient, effective, and interpretable way for the differential testing of evolving autonomous systems across versions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper presents CoCoMagic, a constrained cooperative co-evolutionary approach that combines metamorphic testing and differential testing to generate scenarios revealing behavioral divergences between versions of autonomous driving systems. It evolves source scenarios and metamorphic perturbations to maximize violations of predefined metamorphic relations, incorporates constraints for realism, and includes an interpretability module for diagnosing divergences. Evaluation on the InterFuser end-to-end ADS in the Carla simulator reports up to 287% more distinct high-severity cases than baseline search methods while preserving scenario realism.

Significance. If the quantitative gains and realism claims hold under rigorous validation, the work could meaningfully advance regression testing practices for rapidly evolving autonomous systems by offering an interpretable, search-driven alternative to manual or random scenario generation. The co-evolutionary formulation and interpretability component are constructive contributions that address practical needs in safety-critical software engineering.

major comments (3)
  1. [Evaluation] Evaluation section: the central claim of identifying up to 287% more distinct high-severity behavioral differences is reported without error bars, statistical significance tests, number of independent runs, or random seeds. This single-point percentage undermines confidence that the improvement is robust rather than an artifact of a particular execution.
  2. [Method and Evaluation] Method and Evaluation: the metamorphic relations and severity thresholds used to guide search and label divergences lack any described external validation against public ADS incident data, NHTSA reports, or expert-elicited failure scenarios. Because the 287% gain is produced by counting violations of these fixed relations, the absence of such grounding makes the safety-assessment benefit of the extra cases difficult to assess.
  3. [Experimental Setup] Experimental setup: comparisons are made against unspecified baseline search methods on only a single ADS (InterFuser) and simulator (Carla). The generalizability of the constrained co-evolutionary advantage therefore rests on an extremely narrow empirical base.
minor comments (2)
  1. [Abstract] Abstract and results: clarify whether the 287% figure represents the maximum observed across all experiments or a specific configuration, and ensure all quantitative claims are accompanied by the corresponding raw counts or tables.
  2. [Notation] Notation: ensure consistent use of terms such as 'distinct high-severity behavioral differences' and 'metamorphic relation violations' throughout the method and evaluation sections to avoid ambiguity.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed comments, which highlight important aspects of rigor and generalizability. We address each major comment below, indicating where revisions will be made to the manuscript.

read point-by-point responses

  1. Referee: [Evaluation] Evaluation section: the central claim of identifying up to 287% more distinct high-severity behavioral differences is reported without error bars, statistical significance tests, number of independent runs, or random seeds. This single-point percentage undermines confidence that the improvement is robust rather than an artifact of a particular execution.

    Authors: We agree that the reported 287% figure would be more convincing with statistical support. Although the experiments underlying the results involved multiple independent runs with varied random seeds, these details were omitted from the manuscript. In the revision we will add the number of runs performed, the specific seeds, mean and standard deviation values, error bars on the relevant plots, and statistical significance tests (Wilcoxon rank-sum) between CoCoMagic and each baseline. These additions will appear in the Evaluation section and associated figures. revision: yes

  2. Referee: [Method and Evaluation] Method and Evaluation: the metamorphic relations and severity thresholds used to guide search and label divergences lack any described external validation against public ADS incident data, NHTSA reports, or expert-elicited failure scenarios. Because the 287% gain is produced by counting violations of these fixed relations, the absence of such grounding makes the safety-assessment benefit of the extra cases difficult to assess.

    Authors: The metamorphic relations and severity thresholds were chosen from properties commonly used in the metamorphic-testing literature for autonomous driving (lane-keeping, collision avoidance, speed consistency). Severity was quantified via deviation distance and time-to-collision thresholds that separate minor from high-severity divergences. We will revise the Method section to include explicit references to prior work justifying these choices and will add a short paragraph discussing their relation to safety-critical behaviors. Direct mapping to specific NHTSA reports or expert-elicited scenarios was not performed; we view this as a valuable direction for follow-on work rather than a requirement for the present methodological contribution. revision: partial

  3. Referee: [Experimental Setup] Experimental setup: comparisons are made against unspecified baseline search methods on only a single ADS (InterFuser) and simulator (Carla). The generalizability of the constrained co-evolutionary advantage therefore rests on an extremely narrow empirical base.

    Authors: Section 4.2 of the manuscript specifies the baselines (random search, standard genetic algorithm, and particle-swarm optimization, each adapted to the same constrained co-evolutionary framework). InterFuser and Carla were selected because both are open-source, publicly documented, and representative of end-to-end ADS evaluation platforms. To address the generalizability concern we will add a dedicated “Threats to Validity” subsection that explicitly discusses the single-system, single-simulator limitation and outlines planned extensions to additional ADS platforms. We maintain that the current empirical base is sufficient to demonstrate the advantage of the co-evolutionary formulation while remaining reproducible. revision: partial

Circularity Check

0 steps flagged

No significant circularity detected in derivation or evaluation chain

full rationale

The paper describes a constrained co-evolutionary search method that uses a fixed set of predefined metamorphic relations and severity thresholds as inputs to guide test generation and label results. The reported 287% improvement is an empirical count of distinct high-severity violations found versus baselines on the InterFuser system in Carla; this count does not reduce by construction to a parameter fitted on the same evaluation data or to a self-referential definition. No equations or method steps in the abstract or described approach equate the output metric to its own inputs, and no load-bearing self-citation chain is evident that would force the central claim. The derivation remains self-contained as an engineering method whose validity rests on external falsifiability through the reported experiments rather than internal redefinition.

Axiom & Free-Parameter Ledger

2 free parameters · 2 axioms · 0 invented entities

The method rests on the existence of well-defined metamorphic relations whose violations can be compared across versions, on the assumption that the chosen constraints produce realistic scenarios, and on the claim that the interpretability module correctly identifies root causes. No free parameters are explicitly named in the abstract, but the co-evolutionary search necessarily contains population sizes, mutation rates, and constraint weights that are not reported.

free parameters (2)
  • co-evolutionary population sizes and mutation rates
    Standard parameters of the search algorithm that control how scenarios and perturbations are evolved; their values are not stated in the abstract.
  • constraint weights and severity thresholds
    Values that determine which scenarios are considered realistic and which divergences count as high-severity; these directly affect the 287% improvement metric.
axioms (2)
  • domain assumption Metamorphic relations exist that capture meaningful safety properties for autonomous driving and can be evaluated automatically.
    Invoked when the method uses violations of these relations to drive the search and label behavioral differences.
  • domain assumption The Carla simulator and InterFuser model produce behavior sufficiently representative of real autonomous vehicles for the purpose of differential testing.
    Required for the claim that discovered divergences are relevant to deployed systems.

pith-pipeline@v0.9.0 · 5761 in / 1750 out tokens · 32366 ms · 2026-05-21T22:29:00.789901+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

  • IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear
    ?
    unclear

    Relation between the paper passage and the cited Recognition theorem.

    CoCoMagic formulates test generation as a constrained cooperative co-evolutionary search, evolving both source scenarios and metamorphic perturbations to maximize differences in violations of predefined metamorphic relations

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. From Research to Practice: An Interactive Rapid Review of Autonomous Driving System Testing in Industry

    cs.SE 2026-05 unverdicted novelty 5.0

    Industry practitioners identified 12 ADS testing challenges, prioritized two for end-to-end systems, and found that most of the 17 examined research studies lack direct applicability to real industrial contexts.

Reference graph

Works this paper leans on

103 extracted references · 103 canonical work pages · cited by 1 Pith paper

  1. [1]

    A survey on automated driving system testing: Landscapes and trends,

    S. Tang, Z. Zhang, Y . Zhang, J. Zhou, Y . Guo, S. Liu, S. Guo, Y .-F. Li, L. Ma, Y . Xue, and Y . Liu, “A survey on automated driving system testing: Landscapes and trends,”ACM Transactions on Software Engineering and Methodology, vol. 32, no. 5, pp. 1–62, Jul. 2023. [Online]. Available: http://dx.doi.org/10.1145/3579642

    work page doi:10.1145/3579642 2023

  2. [2]

    Safety testing of automated driving systems: A literature review,

    F. Khan, M. Falco, H. Anwar, and D. Pfahl, “Safety testing of automated driving systems: A literature review,”IEEE Access, vol. 11, pp. 120 049–120 072, Oct. 2023. [Online]. Available: http://dx.doi.org/10.1109/ACCESS.2023.3327918

    work page doi:10.1109/access.2023.3327918 2023

  3. [3]

    Testing of autonomous driving systems: where are we and where should we go?

    G. Lou, Y . Deng, X. Zheng, M. Zhang, and T. Zhang, “Testing of autonomous driving systems: where are we and where should we go?” inProceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ser. ESEC/FSE ’22. New York, NY , USA: Association for Computing Machinery, Nov. 2022, pp. 3...

    work page doi:10.1145/3540250.3549111 2022

  4. [4]

    Iterative learning of an unknown road path through cooperative driving of vehicles,

    L. Yang, Y . Li, D. Huang, and J. Xia, “Iterative learning of an unknown road path through cooperative driving of vehicles,”IET Intelligent Transport Systems, vol. 14, pp. 423–431, Mar. 2020. [Online]. Available: http://dx.doi.org/10.1049/iet-its.2019.0411

    work page doi:10.1049/iet-its.2019.0411 2020

  5. [5]

    Robust adaptive learning-based path tracking control of autonomous vehicles under uncertain driving environments,

    X. Li, C. Liu, B. Chen, and J. Jiang, “Robust adaptive learning-based path tracking control of autonomous vehicles under uncertain driving environments,”IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 11, pp. 20 798–20 809, Nov. 2022. [Online]. Available: http://dx.doi.org/10.1109/TITS.2022.3176970

    work page doi:10.1109/tits.2022.3176970 2022

  6. [6]

    Enhancing autonomous driving systems with deep learning and spatial channel attention mechanisms: an experimental study,

    Y . Yao, “Enhancing autonomous driving systems with deep learning and spatial channel attention mechanisms: an experimental study,” in Fourth International Conference on Machine Learning and Computer Application, ser. ICMLCA ’23, X. Yao and X. Kong, Eds., vol. 13176, International Society for Optics and Photonics. SPIE, May 2024, p. 131761L. [Online]. Ava...

    work page doi:10.1117/12.3029174 2024

  7. [7]

    Whole test suite generation,

    G. Fraser and A. Arcuri, “Whole test suite generation,”IEEE Transactions on Software Engineering, vol. 39, no. 2, pp. 276–291, Feb

    work page

  8. [8]

    Available: http://dx.doi.org/10.1109/TSE.2012.14

    [Online]. Available: http://dx.doi.org/10.1109/TSE.2012.14

    work page doi:10.1109/tse.2012.14 2012

  9. [9]

    Testing machine learning based systems: a systematic mapping,

    V . Riccio, G. Jahangirova, A. Stocco, N. Humbatova, M. Weiss, and P. Tonella, “Testing machine learning based systems: a systematic mapping,”Empirical Software Engineering, vol. 25, no. 6, pp. 5193–5254, Sep. 2020. [Online]. Available: https: //doi.org/10.1007/s10664-020-09881-0

    work page doi:10.1007/s10664-020-09881-0 2020

  10. [10]

    Metamorphic testing: A review of challenges and opportunities,

    T. Y . Chen, F.-C. Kuo, H. Liu, P.-L. Poon, D. Towey, T. H. Tse, and Z. Q. Zhou, “Metamorphic testing: A review of challenges and opportunities,”ACM Computing Surveys, vol. 51, no. 1, pp. 1–27, Jan

    work page

  11. [11]

    Available: https://doi.org/10.1145/3143561

    [Online]. Available: https://doi.org/10.1145/3143561

    work page doi:10.1145/3143561

  12. [12]

    Metamorphic testing of driverless cars,

    Z. Q. Zhou and L. Sun, “Metamorphic testing of driverless cars,” Communications of the ACM, vol. 62, no. 3, pp. 61–67, 2019

    work page 2019

  13. [13]

    Mind the gaps: Assuring the safety of autonomous systems from an engineering, ethical, and legal perspective,

    S. Burton, I. Habli, T. Lawton, J. McDermid, P. Morgan, and Z. Porter, “Mind the gaps: Assuring the safety of autonomous systems from an engineering, ethical, and legal perspective,”Artificial Intelligence, vol. 279, p. 103201, Feb. 2020. [Online]. Available: http://dx.doi.org/10.1016/j.artint.2019.103201

    work page doi:10.1016/j.artint.2019.103201 2020

  14. [14]

    CARLA: An open urban driving simulator,

    A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V . Koltun, “CARLA: An open urban driving simulator,” inProceedings of the 1st Annual Conference on Robot Learning, ser. Proceedings of Machine Learning Research, vol. 78. PMLR, Nov. 2017, pp. 1–16. [Online]. Available: https://proceedings.mlr.press/v78/dosovitskiy17a.html

    work page 2017

  15. [15]

    Safety-enhanced autonomous driving using interpretable sensor fusion transformer,

    H. Shao, L. Wang, R. Chen, H. Li, and Y . Liu, “Safety-enhanced autonomous driving using interpretable sensor fusion transformer,” in Proceedings of The 6th Conference on Robot Learning, ser. Proceedings of Machine Learning Research, K. Liu, D. Kulic, and J. Ichnowski, Eds., vol. 205. PMLR, Dec. 2023, pp. 726–737. [Online]. Available: https://proceedings....

    work page 2023

  16. [16]

    Using cooperative co- evolutionary search to generate metamorphic test cases for autonomous driving systems,

    H. Yousefizadeh, S. Gu, L. C. Briand, and A. Nasr, “Using cooperative co- evolutionary search to generate metamorphic test cases for autonomous driving systems,”IEEE Transactions on Software Engineering, pp. 1–30,

    work page

  17. [17]

    Available: http://dx.doi.org/10.1109/TSE.2025.3570897

    [Online]. Available: http://dx.doi.org/10.1109/TSE.2025.3570897

    work page doi:10.1109/tse.2025.3570897 2025

  18. [18]

    Testing advanced driver assistance systems using multi-objective search and neural networks,

    R. B. Abdessalem, S. Nejati, L. C. Briand, and T. Stifter, “Testing advanced driver assistance systems using multi-objective search and neural networks,” inProceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, ser. ASE ’16. New York, NY , USA: Association for Computing Machinery, Aug. 2016, pp. 63–74. [Online]. Avail...

    work page doi:10.1145/2970276.2970311 2016

  19. [19]

    Testing autonomous cars for feature interaction failures using many- objective search,

    R. B. Abdessalem, A. Panichella, S. Nejati, L. C. Briand, and T. Stifter, “Testing autonomous cars for feature interaction failures using many- objective search,” inProceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, ser. ASE ’18. New York, NY , USA: Association for Computing Machinery, Sep. 2018, pp. 143–154. [Onl...

    work page doi:10.1145/3238147.3238192 2018

  20. [20]

    Testing vision-based control systems using learnable evolutionary algorithms,

    R. B. Abdessalem, S. Nejati, L. C. Briand, and T. Stifter, “Testing vision-based control systems using learnable evolutionary algorithms,” inProceedings of the 40th International Conference on Software Engineering, ser. ICSE ’18. New York, NY , USA: Association for Computing Machinery, May 2018, pp. 1016–1026. [Online]. Available: http://dx.doi.org/10.114...

    work page doi:10.1145/3180155.3180160 2018

  21. [21]

    Compositional falsification of cyber-physical systems with machine learning components,

    T. Dreossi, A. Donz ´e, and S. A. Seshia, “Compositional falsification of cyber-physical systems with machine learning components,”Journal of Automated Reasoning, vol. 63, no. 4, pp. 1031–1053, Jan. 2019. [Online]. Available: http://dx.doi.org/10.1007/s10817-018-09509-5

    work page doi:10.1007/s10817-018-09509-5 2019

  22. [22]

    Simulation-based testing to improve safety of autonomous robots,

    L. V . Sartori, “Simulation-based testing to improve safety of autonomous robots,” in2019 IEEE International Symposium on Software Reliability Engineering Workshops, ser. ISSREW ’19. Institute of Electrical and Electronics Engineers (IEEE), Oct. 2019, pp. 104–107. [Online]. Available: http://dx.doi.org/10.1109/ISSREW.2019.00053

    work page doi:10.1109/issrew.2019.00053 2019

  23. [23]

    Efficient online testing for dnn-enabled systems using surrogate-assisted and many-objective optimization,

    F. U. Haq, D. Shin, and L. Briand, “Efficient online testing for dnn-enabled systems using surrogate-assisted and many-objective optimization,” inProceedings of the 44th International Conference on Software Engineering, ser. ICSE ’22. New York, NY , USA: Association for Computing Machinery, May 2022, pp. 811–822. [Online]. Available: http://dx.doi.org/10....

    work page doi:10.1145/3510003.3510188 2022

  24. [24]

    Fitness function templates for testing automated and autonomous driving systems in intersection scenarios,

    N. Kolb, F. Hauer, and A. Pretschner, “Fitness function templates for testing automated and autonomous driving systems in intersection scenarios,” in2021 IEEE International Intelligent Transportation Systems Conference, ser. ITSC ’21. Institute of Electrical and Electronics Engineers (IEEE), Sep. 2021, pp. 217–222. [Online]. Available: http://dx.doi.org/1...

    work page doi:10.1109/itsc48978.2021.9564591 2021

  25. [25]

    Fitness functions for testing automated and autonomous driving systems,

    F. Hauer, A. Pretschner, and B. Holzm ¨uller, “Fitness functions for testing automated and autonomous driving systems,” inComputer Safety, Reliability, and Security, ser. SAFECOMP ’19. Springer International Publishing, Aug. 2019, pp. 69–84. [Online]. Available: http://dx.doi.org/10.1007/978-3-030-26601-1 5

    work page doi:10.1007/978-3-030-26601-1 2019

  26. [26]

    Nguyen and Raymond Choo

    Y . Luo, X.-Y . Zhang, P. Arcaini, Z. Jin, H. Zhao, F. Ishikawa, R. Wu, and T. Xie, “Targeting requirements violations of autonomous driving systems by dynamic evolutionary search,” in2021 36th IEEE/ACM International Conference on Automated Software Engineering, ser. ASE ’21. Institute of Electrical and Electronics Engineers (IEEE), Nov. 2021, pp. 279–291...

    work page doi:10.1109/ase51524.2021.9678883 2021

  27. [27]

    Cost-effective simulation-based test selection in self-driving cars software,

    C. Birchler, N. Ganz, S. Khatiri, A. Gambi, and S. Panichella, “Cost-effective simulation-based test selection in self-driving cars software,”Science of Computer Programming, vol. 226, p. 102926, Mar

    work page

  28. [28]

    Available: http://dx.doi.org/10.1016/j.scico.2023.102926

    [Online]. Available: http://dx.doi.org/10.1016/j.scico.2023.102926

    work page doi:10.1016/j.scico.2023.102926 2023

  29. [29]

    Single and multi-objective test cases prioritization for self-driving cars in virtual environments,

    C. Birchler, S. Khatiri, P. Derakhshanfar, S. Panichella, and A. Panichella, “Single and multi-objective test cases prioritization for self-driving cars in virtual environments,”ACM Transactions on Software Engineering and Methodology, vol. 32, no. 2, pp. 1–30, Apr. 2023. [Online]. Available: http://dx.doi.org/10.1145/3533818

    work page doi:10.1145/3533818 2023

  30. [30]

    Simulation-based test case generation for unmanned aerial vehicles in the neighborhood of real flights,

    S. Khatiri, S. Panichella, and P. Tonella, “Simulation-based test case generation for unmanned aerial vehicles in the neighborhood of real flights,” in2023 IEEE Conference on Software Testing, Verification and Validation, ser. ICST ’23. Institute of Electrical and Electronics Engineers (IEEE), Apr. 2023, pp. 281–292. [Online]. Available: http://dx.doi.org...

    work page doi:10.1109/icst57152.2023.00034 2023

  31. [31]

    CORTEX- A VD: A framework for CORner case testing and EXploration in autonomous vehicle development,

    G. K. G. Shimanuki, A. M. Nascimento, L. F. Vismari, J. B. C. Junior, J. R. de Almeida Junior, and P. S. Cugnasca, “CORTEX- A VD: A framework for CORner case testing and EXploration in autonomous vehicle development,” 2025. [Online]. Available: https://arxiv.org/abs/2504.03989

    work page arXiv 2025

  32. [32]

    In: Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Im- plementation

    D. J. Fremont, T. Dreossi, S. Ghosh, X. Yue, A. L. Sangiovanni- Vincentelli, and S. A. Seshia, “Scenic: a language for scenario specification and scene generation,” inProceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, ser. PLDI ’19. New York, NY , USA: Association for Computing Machinery (ACM), Jun. 2019, pp....

    work page doi:10.1145/3314221.3314633 2019

  33. [33]

    A V-FUZZER: Finding safety violations in autonomous driving systems,

    G. Li, Y . Li, S. Jha, T. Tsai, M. Sullivan, S. K. S. Hari, Z. Kalbarczyk, and R. Iyer, “A V-FUZZER: Finding safety violations in autonomous driving systems,” in2020 IEEE 31st International Symposium on Software Reliability Engineering, ser. ISSRE ’20. Institute of Electrical and Electronics Engineers (IEEE), Oct. 2020, pp. 25–36. [Online]. Available: htt...

    work page doi:10.1109/issre5003.2020.00012 2020

  34. [34]

    Baidu. Apollo. [Online]. Available: https://en.apollo.auto/ apollo-self-driving

    work page

  35. [35]

    BehA VExplor: Behavior diversity guided testing for autonomous driving systems,

    M. Cheng, Y . Zhou, and X. Xie, “BehA VExplor: Behavior diversity guided testing for autonomous driving systems,” inProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, ser. ISSTA ’23. New York, NY , USA: Association for Computing Machinery, Jul. 2023, pp. 488–500. [Online]. Available: https://doi.org/10.1145/35979...

    work page doi:10.1145/3597926.3598072 2023

  36. [36]

    LGSVL simulator: A high fidelity simulator for autonomous driving,

    G. Rong, B. H. Shin, H. Tabatabaee, Q. Lu, S. Lemke, M. Mozeiko, E. Boise, G. Uhm, M. Gerow, S. Mehta, E. Agafonov, T. H. Kim, E. Sterner, K. Ushiroda, M. Reyes, D. Zelenkovsky, and S. Kim, “LGSVL simulator: A high fidelity simulator for autonomous driving,” in2020 IEEE 23rd International Conference on Intelligent Transportation Systems, ser. ITSC ’20. In...

    work page doi:10.1109/itsc45102.2020.9294422 2020

  37. [37]

    PAFOT: A position-based approach for finding optimal tests of autonomous vehicles,

    V . Crespo-Rodriguez, Neelofar, and A. Aleti, “PAFOT: A position-based approach for finding optimal tests of autonomous vehicles,” in Proceedings of the 5th ACM/IEEE International Conference on Automation of Software Test, ser. AST ’24. New York, NY , USA: Association for Computing Machinery (ACM), Apr. 2024, pp. 159–170. [Online]. Available: http://dx.do...

    work page doi:10.1145/3644032.3644457 2024

  38. [38]

    Autonomous driving system testing via diversity-oriented driving scenario exploration,

    X. Ji, L. Xue, Z. He, and X. Luo, “Autonomous driving system testing via diversity-oriented driving scenario exploration,”ACM Transactions on Software Engineering and Methodology, Apr. 2025. [Online]. Available: https://doi.org/10.1145/3727875

    work page doi:10.1145/3727875 2025

  39. [39]

    CARLA autonomous driving leaderboard

    CARLA. CARLA autonomous driving leaderboard. [Online]. Available: https://leaderboard.carla.org/leaderboard/

    work page

  40. [40]

    Borges Jr., and Andreas Zeller

    A. Gambi, M. Mueller, and G. Fraser, “Automatically testing self-driving cars with search-based procedural content generation,” inProceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, ser. ISSTA ’19, vol. 4. New York, NY , USA: Association for Computing Machinery, Jul. 2019, pp. 318–328. [Online]. Available: http://...

    work page doi:10.1145/3293882.3330566 2019

  41. [41]

    Eagle strategy with local search for scenario based validation of autonomous vehicles,

    Q. Goss and M. I. Akba s ¸, “Eagle strategy with local search for scenario based validation of autonomous vehicles,” in2022 International IEEE TRANSACTIONS ON SOFTW ARE ENGINEERING, VOL. 14, NO. 8, AUGUST 2021 24 Conference on Connected Vehicle and Expo, ser. ICCVE ’22. Institute of Electrical and Electronics Engineers (IEEE), Mar. 2022, pp. 1–6. [Online]...

    work page doi:10.1109/iccve52871.2022.9743067 2021

  42. [42]

    Rapid generation of challenging simulation scenarios for autonomous vehicles based on adversarial test,

    X. Zheng, H. Liang, B. Yu, B. Li, S. Wang, and Z. Chen, “Rapid generation of challenging simulation scenarios for autonomous vehicles based on adversarial test,” in2020 IEEE International Conference on Mechatronics and Automation, ser. ICMA ’20. Institute of Electrical and Electronics Engineers (IEEE), Oct. 2020, pp. 1166–1172. [Online]. Available: http:/...

    work page doi:10.1109/icma49215.2020.9233535 2020

  43. [43]

    AmbieGen: A search-based framework for autonomous systems testing,

    D. Humeniuk, F. Khomh, and G. Antoniol, “AmbieGen: A search-based framework for autonomous systems testing,”Science of Computer Programming, vol. 230, p. 102990, Aug. 2023. [Online]. Available: http://dx.doi.org/10.1016/j.scico.2023.102990

    work page doi:10.1016/j.scico.2023.102990 2023

  44. [44]

    Reinforcement learning informed evolutionary search for autonomous systems testing,

    ——, “Reinforcement learning informed evolutionary search for autonomous systems testing,”ACM Transactions on Software Engineering and Methodology, vol. 33, no. 8, Nov. 2024. [Online]. Available: http://dx.doi.org/10.1145/3680468

    work page doi:10.1145/3680468 2024

  45. [45]

    Two is better than one: digital siblings to improve autonomous driving testing,

    M. Biagiola, A. Stocco, V . Riccio, and P. Tonella, “Two is better than one: digital siblings to improve autonomous driving testing,”Empirical Software Engineering, vol. 29, no. 4, May 2024. [Online]. Available: https://doi.org/10.1007/s10664-024-10458-4

    work page doi:10.1007/s10664-024-10458-4 2024

  46. [46]

    Simulator ensembles for trustworthy autonomous driving testing,

    L. Sorokin, M. Biagiola, and A. Stocco, “Simulator ensembles for trustworthy autonomous driving testing,” 2025. [Online]. Available: https://arxiv.org/abs/2503.08936

    work page arXiv 2025

  47. [47]

    Simulation-based safety assessment of vehicle characteristics variations in autonomous driving systems,

    Q. Pan, T. Wang, J. Ma, P. Arcaini, and T. Yue, “Simulation-based safety assessment of vehicle characteristics variations in autonomous driving systems,”ACM Transactions on Software Engineering and Methodology, Jun. 2025. [Online]. Available: https://doi.org/10.1145/3743673

    work page doi:10.1145/3743673 2025

  48. [48]

    and Pratap, A

    K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, “A fast and elitist multiobjective genetic algorithm: NSGA-II,”IEEE Transactions on Evolutionary Computation, vol. 6, no. 2, pp. 182–197, Apr. 2002. [Online]. Available: http://dx.doi.org/10.1109/4235.996017

    work page doi:10.1109/4235.996017 2002

  49. [49]

    Metamorphic testing: A new approach for generating next test cases,

    T. Y . Chen, S. Cheung, and S. Yiu, “Metamorphic testing: A new approach for generating next test cases,”CoRR, vol. abs/2002.12543, Feb. 2020. [Online]. Available: https://arxiv.org/abs/2002.12543

    work page arXiv 2002

  50. [50]

    Metamorphic testing: Testing the untestable,

    S. Segura, D. Towey, Z. Q. Zhou, and T. Y . Chen, “Metamorphic testing: Testing the untestable,”IEEE Software, vol. 37, no. 3, pp. 46–53, May

    work page

  51. [51]

    Available: http://dx.doi.org/10.1109/MS.2018.2875968

    [Online]. Available: http://dx.doi.org/10.1109/MS.2018.2875968

    work page doi:10.1109/ms.2018.2875968 2018

  52. [52]

    MetaLiDAR: Automated metamorphic testing of lidar-based autonomous driving systems,

    Z. Yang, S. Huang, C. Zheng, X. Wang, Y . Wang, and C. Xia, “MetaLiDAR: Automated metamorphic testing of lidar-based autonomous driving systems,”Journal of Software: Evolution and Process, vol. 36, no. 7, p. e2644, Dec. 2023. [Online]. Available: http: //dx.doi.org/10.1002/smr.2644

    work page doi:10.1002/smr.2644 2023

  53. [53]

    MetaSem: metamorphic testing based on semantic information of autonomous driving scenes,

    Z. Yang, S. Huang, T. Bai, Y . Yao, Y . Wang, C. Zheng, and C. Xia, “MetaSem: metamorphic testing based on semantic information of autonomous driving scenes,”Software Testing, Verification and Reliability, vol. 34, no. 5, p. e1878, May 2024. [Online]. Available: http://dx.doi.org/10.1002/stvr.1878

    work page doi:10.1002/stvr.1878 2024

  54. [54]

    Metamorphic model-based testing of autonomous systems,

    M. Lindvall, A. Porter, G. Magnusson, and C. Schulze, “Metamorphic model-based testing of autonomous systems,” in2017 IEEE/ACM 2nd International Workshop on Metamorphic Testing, ser. MET ’17. Institute of Electrical and Electronics Engineers (IEEE), May 2017, pp. 35–41. [Online]. Available: http://dx.doi.org/10.1109/MET.2017.6

    work page doi:10.1109/met.2017.6 2017

  55. [55]

    DeepTest: automated testing of deep-neural-network-driven autonomous cars,

    Y . Tian, K. Pei, S. Jana, and B. Ray, “DeepTest: automated testing of deep-neural-network-driven autonomous cars,” inProceedings of the 40th International Conference on Software Engineering, ser. ICSE ’18. New York, NY , USA: Association for Computing Machinery, May 2018, pp. 303–314. [Online]. Available: http://dx.doi.org/10.1145/3180155.3180220

    work page doi:10.1145/3180155.3180220 2018

  56. [56]

    DeepRoad: Gan-based metamorphic testing and input validation framework for autonomous driving systems,

    M. Zhang, Y . Zhang, L. Zhang, C. Liu, and S. Khurshid, “DeepRoad: Gan-based metamorphic testing and input validation framework for autonomous driving systems,” inProceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, ser. ASE ’18. New York, NY , USA: Association for Computing Machinery, Sep. 2018, pp. 132–142. [Onlin...

    work page doi:10.1145/3238147.3238187 2018

  57. [57]

    Metamorphic testing for autonomous driving systems in fog based on quantitative measurement,

    Y . Pan, H. Ao, and Y . Fan, “Metamorphic testing for autonomous driving systems in fog based on quantitative measurement,” in2021 IEEE 21st International Conference on Software Quality, Reliability and Security Companion, ser. QRS-C ’21. Institute of Electrical and Electronics Engineers (IEEE), Dec. 2021, pp. 30–37. [Online]. Available: http://dx.doi.org...

    work page doi:10.1109/qrs-c55045.2021.00015 2021

  58. [58]

    Metamorphic fuzz testing of autonomous vehicles,

    J. C. Han and Z. Q. Zhou, “Metamorphic fuzz testing of autonomous vehicles,” inProceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops, ser. ICSEW ’20. New York, NY , USA: Association for Computing Machinery, Jun. 2020, pp. 380–385. [Online]. Available: http://dx.doi.org/10.1145/3387940.3392252

    work page doi:10.1145/3387940.3392252 2020

  59. [59]

    Evaluating decision optimality of autonomous driving via metamorphic testing,

    M. Cheng, Y . Zhou, X. Xie, J. Wang, G. Meng, and K. Yang, “Evaluating decision optimality of autonomous driving via metamorphic testing,” 2024. [Online]. Available: https://arxiv.org/abs/2402.18393

    work page arXiv 2024

  60. [60]

    Towards a metamorphic testing architecture for software-defined drone systems,

    E. M. Fredericks, M. Jacobs, and B. DeVries, “Towards a metamorphic testing architecture for software-defined drone systems,” in2024 11th International Conference on Software Defined Systems, ser. SDS ’24. Institute of Electrical and Electronics Engineers (IEEE), Dec. 2024, pp. 170–177. [Online]. Available: http://dx.doi.org/10.1109/SDS64317.2024. 10883896

    work page doi:10.1109/sds64317.2024 2024

  61. [61]

    Metamorphic relation generation: State of the art and research directions,

    R. Li, H. Liu, P.-L. Poon, D. Towey, C.-A. Sun, Z. Zheng, Z. Q. Zhou, and T. Y . Chen, “Metamorphic relation generation: State of the art and research directions,”ACM Transactions on Software Engineering and Methodology, vol. 34, no. 5, May 2025. [Online]. Available: https://doi.org/10.1145/3708521

    work page doi:10.1145/3708521 2025

  62. [62]

    A survey on cooperative co-evolutionary algorithms,

    X. Ma, X. Li, Q. Zhang, K. Tang, Z. Liang, W. Xie, and Z. Zhu, “A survey on cooperative co-evolutionary algorithms,”IEEE Transactions on Evolutionary Computation, vol. 23, no. 3, pp. 421–441, Jun. 2019. [Online]. Available: http://dx.doi.org/10.1109/TEVC.2018.2868770

    work page doi:10.1109/tevc.2018.2868770 2019

  63. [63]

    Large scale evolutionary optimization using cooperative coevolution,

    Z. Yang, K. Tang, and X. Yao, “Large scale evolutionary optimization using cooperative coevolution,”Information Sciences, vol. 178, no. 15, pp. 2985–2999, Aug. 2008. [Online]. Available: http://dx.doi.org/10.1016/j.ins.2008.02.017

    work page doi:10.1016/j.ins.2008.02.017 2008

  64. [64]

    Archive-based cooperative coevolutionary algorithms,

    L. Panait, S. Luke, and J. F. Harrison, “Archive-based cooperative coevolutionary algorithms,” inProceedings of the 8th annual conference on Genetic and evolutionary computation, ser. GECCO ’06. New York, NY , USA: Association for Computing Machinery, Jul. 2006, pp. 345–352. [Online]. Available: http://dx.doi.org/10.1145/1143997.1144060

    work page doi:10.1145/1143997.1144060 2006

  65. [65]

    A declarative metamorphic testing framework for autonomous driving,

    Y . Deng, X. Zheng, T. Zhang, H. Liu, G. Lou, M. Kim, and T. Y . Chen, “A declarative metamorphic testing framework for autonomous driving,”IEEE Transactions on Software Engineering, vol. 49, no. 4, pp. 1964–1982, Apr. 2023. [Online]. Available: http://dx.doi.org/10.1109/TSE.2022.3206427

    work page doi:10.1109/tse.2022.3206427 1964

  66. [66]

    Can offline testing of deep neural networks replace their online testing?: A case study of automated driving systems,

    F. U. Haq, D. Shin, S. Nejati, and L. Briand, “Can offline testing of deep neural networks replace their online testing?: A case study of automated driving systems,”Empirical Software Engineering, vol. 26, no. 90, Jul

    work page

  67. [67]

    Available: http://dx.doi.org/10.1007/s10664-021-09982-4

    [Online]. Available: http://dx.doi.org/10.1007/s10664-021-09982-4

    work page doi:10.1007/s10664-021-09982-4

  68. [68]

    Identifying the hazard boundary of ml-enabled autonomous systems using cooperative coevolutionary search,

    S. Sharifi, D. Shin, L. C. Briand, and N. Aschbacher, “Identifying the hazard boundary of ml-enabled autonomous systems using cooperative coevolutionary search,”IEEE Transactions on Software Engineering, vol. 49, no. 12, pp. 5120–5138, Dec. 2023. [Online]. Available: http://dx.doi.org/10.1109/TSE.2023.3327575

    work page doi:10.1109/tse.2023.3327575 2023

  69. [69]

    Evolutionary computation in multi-agent environments: Partners,

    L. Bull, “Evolutionary computation in multi-agent environments: Partners,” inProceedings of the 7th International Conference on Genetic Algorithms. Morgan Kaufmann, Jul. 1997, pp. 370–377

    work page 1997

  70. [70]

    Berlin, Heidelberg: Springer Berlin Heidelberg, 1998, pp

    ——,Evolutionary computing in multi-agent environments: Operators. Berlin, Heidelberg: Springer Berlin Heidelberg, 1998, pp. 43–52. [Online]. Available: http://dx.doi.org/10.1007/BFb0040758

    work page doi:10.1007/bfb0040758 1998

  71. [71]

    An empirical analysis of collaboration methods in cooperative coevolutionary algorithms,

    R. P. Wiegand, W. C. Liles, and K. A. D. Jong, “An empirical analysis of collaboration methods in cooperative coevolutionary algorithms,” in Proceedings of the genetic and evolutionary computation conference, ser. GECCO ’01, vol. 2611. Morgan Kaufmann, Jul. 2001, pp. 1235–1245

    work page 2001

  72. [72]

    Improved heterogeneous distance functions,

    D. R. Wilson and T. R. Martinez, “Improved heterogeneous distance functions,”Journal of Artificial Intelligence Research, vol. 6, pp. 1–34, Jan. 1997. [Online]. Available: http://dx.doi.org/10.1613/jair.346

    work page doi:10.1613/jair.346 1997

  73. [73]

    A clearing procedure as a niching method for genetic algorithms,

    A. Petrowski, “A clearing procedure as a niching method for genetic algorithms,” inProceedings of IEEE International Conference on Evolutionary Computation, ser. ICEC ’96. Institute of Electrical and Electronics Engineers (IEEE), May 1996, pp. 798–803. [Online]. Available: http://dx.doi.org/10.1109/ICEC.1996.542703

    work page doi:10.1109/icec.1996.542703 1996

  74. [74]

    Fitness sharing and niching methods revisited,

    B. Sareni and L. Krahenbuhl, “Fitness sharing and niching methods revisited,”IEEE Transactions on Evolutionary Computation, vol. 2, no. 3, pp. 97–106, Sep. 1998. [Online]. Available: http://dx.doi.org/10.1109/4235.735432

    work page doi:10.1109/4235.735432 1998

  75. [75]

    Selforganization of matter and the evolution of biological macromolecules,

    M. Eigen, “Selforganization of matter and the evolution of biological macromolecules,”Die Naturwissenschaften, vol. 58, no. 10, pp. 465–523, Oct. 1971. [Online]. Available: http://dx.doi.org/10.1007/BF00623322

    work page doi:10.1007/bf00623322 1971

  76. [76]

    Genetic algorithms with sharing for multimodal function optimization,

    D. E. Goldberg and J. Richardson, “Genetic algorithms with sharing for multimodal function optimization,” inGenetic algorithms and their applications: Proceedings of the Second International Conference on Genetic Algorithms, vol. 4149. Hillsdale, NJ: Lawrence Erlbaum, 1987

    work page 1987

  77. [77]

    Horn and D

    J. Horn and D. E. Goldberg,A timing analysis of convergence to fitness sharing equilibrium. Berlin, Heidelberg: Springer Science and Business Media LLC, 1998, pp. 23–33. [Online]. Available: http://dx.doi.org/10.1007/BFb0056846

    work page doi:10.1007/bfb0056846 1998

  78. [78]

    Population size and genetic drift in fitness sharing,

    S. W. Mahfoud, “Population size and genetic drift in fitness sharing,” in Foundations of Genetic Algorithms. Elsevier BV , 1995, vol. 3, pp. 185– IEEE TRANSACTIONS ON SOFTW ARE ENGINEERING, VOL. 14, NO. 8, AUGUST 2021 25

    work page 1995

  79. [79]

    Available: http://dx.doi.org/10.1016/B978-1-55860-356-1

    [Online]. Available: http://dx.doi.org/10.1016/B978-1-55860-356-1. 50014-5

    work page doi:10.1016/b978-1-55860-356-1

  80. [80]

    Luke,Essentials of Metaheuristics, 2nd ed

    S. Luke,Essentials of Metaheuristics, 2nd ed. Lulu, 2013, available for free at http://cs.gmu.edu/∼sean/book/metaheuristics/

    work page 2013

Showing first 80 references.