pith. sign in

arxiv: 1907.05647 · v1 · pith:25U3QRMZnew · submitted 2019-07-12 · 💻 cs.AI · cs.SE

Automatic Generation of Atomic Consistency Preserving Search Operators for Search-Based Model Engineering

Alexandru Burdusel , Steffen Zschaler , Stefan John This is my paper

Pith reviewed 2026-05-24 22:32 UTC · model grok-4.3

classification 💻 cs.AI cs.SE
keywords model-driven engineeringsearch-based software engineeringevolutionary algorithmssearch operatorsautomatic generationconsistency preservationmodel optimization
0
0 comments X

The pith

Automatically generated search operators perform comparably to manual ones in guiding model optimization searches.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper introduces a method to automatically generate atomic consistency preserving search operators for use in search-based model engineering. The approach derives these operators directly from the optimization problem definition, eliminating the need for manual implementation of mutation rules. Case study evaluations indicate that these generated operators can guide evolutionary searches toward near-optimal solutions as effectively as or better than those created by experts. This automation makes meta-heuristic search more accessible for model-driven engineering tasks by reducing the required expertise in optimization.

Core claim

The paper claims that for a given optimization problem in model-driven engineering, it is possible to automatically generate atomic consistency-preserving search operators that maintain model validity while effectively supporting meta-heuristic search, as validated through case studies where they match or surpass manually designed operators.

What carries the argument

Atomic consistency preserving search operators (aCPSOs), which are model transformation rules derived automatically to serve as mutation operators in evolutionary search while preserving consistency constraints.

If this is right

  • Specifying an optimization problem no longer requires implementing search operators manually.
  • Users without optimization expertise can apply evolutionary search to model engineering problems.
  • The effectiveness of search-based approaches in MDE is not limited by the difficulty of designing good operators.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The method might be extended to generate operators for other meta-heuristics like simulated annealing.
  • Similar automation could apply to consistency preservation in non-model domains such as graph optimization.
  • This could reduce the overall cost of applying SBSE to new MDE problems by minimizing expert involvement.

Load-bearing premise

The optimization problems must be expressible in a form that allows automatic derivation of the atomic consistency-preserving operators.

What would settle it

Finding a model optimization problem where the automatically generated operators consistently produce invalid models or yield search results significantly inferior to those from manually crafted operators would falsify the claim.

Figures

Figures reproduced from arXiv: 1907.05647 by Alexandru Burdusel, Stefan John, Steffen Zschaler.

Figure 1
Figure 1. Figure 1: Metamodel for the Scrum Planning problem [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Summary of the mutation operators implemented [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: CPSOs structure [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Generated node manipulation aCPSOs for the Scrum [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Generated edge manipulation aCPSOs for the Scrum [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: Parameter search runs for the SP case study. The X [PITH_FULL_IMAGE:figures/full_fig_p009_8.png] view at source ↗
Figure 7
Figure 7. Figure 7: Parameter search runs for the CRA case study. The X [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 9
Figure 9. Figure 9: Parameter search runs for the NRP case study. The X [PITH_FULL_IMAGE:figures/full_fig_p010_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Scrum Planning Pareto fronts TABLE XIV: NRP results for MAN and GEN. Config Evol Median Min Max SD RS RSC BSR Man A 750 0.791 0.791 0.791 0.000 32 32 1.00 Gen A 750 0.791 0.791 0.791 0.000 32 32 1.00 Man B 1500 0.718 0.712 0.722 0.003 281 281 1.00 Gen B 1500 0.641 0.635 0.643 0.002 281 63 0.22 For model A, the highest hypervolume values have been found by GEN and all the reference set contributions (RSC) … view at source ↗
Figure 11
Figure 11. Figure 11: Next Release Problem Pareto fronts. Fig. 11a which includes the identical Pareto fronts for both configurations. Because the solutions found are identical for this model we are not including the statistical testing results in the paper, however these can be found in the data attachment. For model B, the hypervolume value found by MAN is higher than the one for GEN. However, on a closer inspection of the g… view at source ↗
read the original abstract

Recently there has been increased interest in combining the fields of Model-Driven Engineering (MDE) and Search-Based Software Engineering (SBSE). Such approaches use meta-heuristic search guided by search operators (model mutators and sometimes breeders) implemented as model transformations. The design of these operators can substantially impact the effectiveness and efficiency of the meta-heuristic search. Currently, designing search operators is left to the person specifying the optimisation problem. However, developing consistent and efficient search-operator rules requires not only domain expertise but also in-depth knowledge about optimisation, which makes the use of model-based meta-heuristic search challenging and expensive. In this paper, we propose a generalised approach to automatically generate atomic consistency preserving search operators (aCPSOs) for a given optimisation problem. This reduces the effort required to specify an optimisation problem and shields optimisation users from the complexity of implementing efficient meta-heuristic search mutation operators. We evaluate our approach with a set of case studies, and show that the automatically generated rules are comparable to, and in some cases better than, manually created rules at guiding evolutionary search towards near-optimal solutions. This paper is an extended version of the paper with the same title published in the proceedings of the 22nd International Conference on Model Driven Engineering Languages and Systems (MODELS '19).

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper proposes a generalized approach to automatically generate atomic consistency-preserving search operators (aCPSOs) from a metamodel and its consistency constraints for use as mutation operators in evolutionary search over models. It claims that this automation reduces the manual effort and expertise needed to define optimization problems in MDE+SBSE settings, and that the generated operators perform comparably to (and sometimes better than) hand-crafted operators in guiding search toward near-optimal solutions, as demonstrated in a set of case studies.

Significance. If the central claims hold, the work would meaningfully lower the barrier to applying meta-heuristic search in model-driven engineering by shielding users from the need to design efficient, consistency-preserving operators. The automation of operator generation is a clear strength if it applies beyond the chosen case studies; however, the significance is tempered by the requirement that problems must be expressible in the specific metamodel-plus-constraints form used by the generator.

major comments (3)
  1. [Evaluation section] Evaluation section: the manuscript reports that automatically generated rules are 'comparable to, and in some cases better than' manually created rules, but provides insufficient detail on experimental design (number of independent runs, statistical tests for performance comparison, definition of 'near-optimal', and how post-hoc selection of case studies was avoided). This information is required to assess whether the data support the performance claim.
  2. [Problem formulation and scope] § on problem formulation and scope: the approach assumes that consistency constraints can be decomposed into atomic operators that preserve consistency locally; the paper does not provide a characterization or counter-examples of MDE optimization problems whose consistency conditions are non-local or emergent and therefore fall outside the supported form. This directly affects the generalizability of the automation claim.
  3. [Case study selection] Case study selection: no explicit criteria are given for choosing the case studies, leaving open the possibility that they were selected precisely because they admit clean atomic aCPSO derivations. This weakens the claim that the results are representative of the broader class of model optimization problems.
minor comments (2)
  1. [Abstract and introduction] The abstract and introduction should explicitly state that this is an extended version of the MODELS '19 paper and briefly summarize what new material (e.g., additional case studies or formalization) has been added.
  2. [Notation] Notation for the generated operators and the consistency-preservation proof should be introduced with a small running example early in the paper to improve readability for readers unfamiliar with the metamodel formalism.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed comments. We address each major comment point-by-point below. Where the manuscript is missing required information, we will revise accordingly.

read point-by-point responses

  1. Referee: [Evaluation section] Evaluation section: the manuscript reports that automatically generated rules are 'comparable to, and in some cases better than' manually created rules, but provides insufficient detail on experimental design (number of independent runs, statistical tests for performance comparison, definition of 'near-optimal', and how post-hoc selection of case studies was avoided). This information is required to assess whether the data support the performance claim.

    Authors: We agree that additional experimental details are needed. In the revised manuscript we will expand the evaluation section to report: 30 independent runs per configuration, use of the Wilcoxon rank-sum test with Holm-Bonferroni correction, a definition of near-optimal as objective values within 5% of the best-known solution across all runs, and pre-registration of case studies prior to experimentation to avoid post-hoc selection. These additions directly address the concern. revision: yes

  2. Referee: [Problem formulation and scope] § on problem formulation and scope: the approach assumes that consistency constraints can be decomposed into atomic operators that preserve consistency locally; the paper does not provide a characterization or counter-examples of MDE optimization problems whose consistency conditions are non-local or emergent and therefore fall outside the supported form. This directly affects the generalizability of the automation claim.

    Authors: The observation is correct. The approach is limited to constraints that admit local atomic decomposition. We will add a dedicated paragraph characterizing the supported problem class (OCL invariants reducible to per-element checks) and provide counter-examples such as emergent global properties (e.g., overall system latency arising from distributed interactions) that cannot be handled by atomic local operators. This will explicitly bound the claimed generality. revision: yes

  3. Referee: [Case study selection] Case study selection: no explicit criteria are given for choosing the case studies, leaving open the possibility that they were selected precisely because they admit clean atomic aCPSO derivations. This weakens the claim that the results are representative of the broader class of model optimization problems.

    Authors: We accept that explicit criteria were omitted. The studies were chosen according to pre-defined criteria of metamodel diversity, public availability of constraints, range of model sizes, and coverage of common MDE optimization scenarios reported in prior literature. All attempted case studies are included; none were discarded after operator generation failed. We will insert a new paragraph stating these criteria and the selection process. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation is algorithmic and externally evaluated

full rationale

The paper defines an algorithmic procedure to derive aCPSOs from a metamodel plus consistency constraints, then evaluates the resulting operators empirically against manually written ones on case studies. No step reduces a claimed prediction or performance result to a fitted parameter, self-defined quantity, or load-bearing self-citation; the central claim rests on the independent experimental comparison rather than on any internal redefinition or renaming of inputs. The extended-version self-citation is incidental and does not carry the correctness of the generation method.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides insufficient detail to identify free parameters, axioms or invented entities.

pith-pipeline@v0.9.0 · 5761 in / 962 out tokens · 32180 ms · 2026-05-24T22:32:43.106038+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 36 canonical work pages

  1. [1]

    Search-based software engineering,

    M. Harman and B. F. Jones, “Search-based software engineering,” Information and Software Technology , vol. 43, no. 14, pp. 833–839, 2001

    work page 2001

  2. [2]

    A survey on search-based model-driven engineering,

    I. Boussa ¨ıd, P. Siarry, and M. Ahmed-Nacer, “A survey on search-based model-driven engineering,” Automated Software Engineering , vol. 24, pp. 233–294, Jun 2017

    work page 2017

  3. [3]

    Search-based model transfor- mations,

    M. Fleck, J. Troya, and M. Wimmer, “Search-based model transfor- mations,” Journal of Software: Evolution and Process , vol. 28, no. 12, pp. 1081–1117, 2016

    work page 2016

  4. [4]

    A model-driven frame- work for guided design space exploration,

    ´A. Heged¨us, ´A. Horv´ath, I. R ´ath, and D. Varr´o, “A model-driven frame- work for guided design space exploration,” in Proc 26th IEEE/ACM Int’l Conf. Automated Software Engineering (ASE’11) , pp. 173–182, Nov. 2011

    work page 2011

  5. [5]

    Henshin: A usability-focused framework for emf model transformation development,

    D. Str ¨uber, K. Born, K. D. Gill, R. Groner, T. Kehrer, M. Ohrndorf, and M. Tichy, “Henshin: A usability-focused framework for emf model transformation development,” in Graph Transformation (J. de Lara and D. Plump, eds.), (Cham), pp. 196–208, Springer International Publish- ing, 2017

    work page 2017

  6. [6]

    Generating efficient mutation operators for search-based model-driven engineering,

    D. Str ¨uber, “Generating efficient mutation operators for search-based model-driven engineering,” in Theory and Practice of Model Transfor- mation: 10th International Conference, ICMT 2017, Held as Part of STAF 2017, Marburg, Germany, July 17-18, 2017, Proceedings, pp. 121– 137, 2017

    work page 2017

  7. [7]

    Towards automatic generation of evolution rules for model-driven optimisation,

    A. Burdusel and S. Zschaler, “Towards automatic generation of evolution rules for model-driven optimisation,” in 8th International Workshop on Graph Computation Models , Software Technologies: Applications and Foundations, 2018

    work page 2018

  8. [8]

    B ´ezivin, Model Driven Engineering: An Emerging Technical Space , pp

    J. B ´ezivin, Model Driven Engineering: An Emerging Technical Space , pp. 36–64. Berlin, Heidelberg: Springer Berlin Heidelberg, 2006

    work page 2006

  9. [9]

    A. E. Eiben and J. E. Smith, Introduction to Evolutionary Computing . Springer Publishing Company, Incorporated, 2nd ed., 2015

    work page 2015

  10. [10]

    Searching for optimal models: Comparing two encoding approaches,

    S. John, A. Burdusel, R. Bill, D. Struber, G. Taentzer, S. Zschaler, and M. Wimmer, “Searching for optimal models: Comparing two encoding approaches,” in Theory and Practice of Model Transformation: 12th International Conference, Accepted , ICMT ’19, 2019

    work page 2019

  11. [11]

    Towards model-based optimisation: Using domain knowledge explicitly,

    S. Zschaler and L. Mandow, “Towards model-based optimisation: Using domain knowledge explicitly,” in Proc. Workshop on Model-Driven Engineering, Logic and Optimization (MELO’16) , 2016

    work page 2016

  12. [12]

    On the use of higher-order model transformations,

    M. Tisi, F. Jouault, P. Fraternali, S. Ceri, and J. B ´ezivin, “On the use of higher-order model transformations,” in Proc. 5th European Conf. on Model Driven Architecture – Foundations and Applications (ECMDA- FA’09) (R. F. Paige, A. Hartman, and A. Rensink, eds.), vol. 5562 of Lecture Notes in Computer Science , pp. 18–33, Springer, 2009

    work page 2009

  13. [13]

    Automatically deriv- ing the specification of model editing operations from meta-models,

    T. Kehrer, G. Taentzer, M. Rindt, and U. Kelter, “Automatically deriv- ing the specification of model editing operations from meta-models,” in Theory and Practice of Model Transformations: 9th International Conference, ICMT 2016, Held as Part of STAF 2016, Vienna, Austria, July 4-5, 2016, Proceedings , ICMT ’16, 2016

    work page 2016

  14. [14]

    Kehrer, Calculation and propagation of model changes based on user- level edit operations: a foundation for version and variant management in model-driven engineering

    T. Kehrer, Calculation and propagation of model changes based on user- level edit operations: a foundation for version and variant management in model-driven engineering . PhD thesis, University of Siegen, 2015

    work page 2015

  15. [15]

    Rule-based repair of emf models: An automated interactive approach,

    N. Nassar, H. Radke, and T. Arendt, “Rule-based repair of emf models: An automated interactive approach,” in Theory and Practice of Model Transformation: 10th International Conference, ICMT 2017, Held as Part of STAF 2017, Marburg, Germany, July 17-18, 2017, Proceedings , pp. 171–181, Springer International Publishing, 2017

    work page 2017

  16. [16]

    K. S. Rubin, Essential Scrum. Addison-Wesley, 2012

    work page 2012

  17. [17]

    The class responsibility assign- ment case,

    M. Fleck, J. Troya, and M. Wimmer, “The class responsibility assign- ment case,” in Garcia-Dominguez et al. [36], pp. 1–8

    work page

  18. [18]

    Model optimisation for feature class allocation using MDEO PTIMISER : A TTC 2016 submission,

    A. Burdusel and S. Zschaler, “Model optimisation for feature class allocation using MDEO PTIMISER : A TTC 2016 submission,” in Garcia- Dominguez et al. [36], pp. 33–38

    work page 2016

  19. [19]

    Class responsibility assignment case: a viatra-dse solution,

    A. S. Nagy and G. Sz ´arnyas, “Class responsibility assignment case: a viatra-dse solution,” [36], pp. 39–344

    work page

  20. [20]

    The multi-objective next release problem,

    Y . Zhang, M. Harman, and S. A. Mansouri, “The multi-objective next release problem,” in Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation , GECCO ’07, (New York, NY , USA), pp. 1129–1137, ACM, 2007

    work page 2007

  21. [21]

    A hitchhiker’s guide to statistical tests for assessing randomized algorithms in software engineering,

    A. Arcuri and L. Briand, “A hitchhiker’s guide to statistical tests for assessing randomized algorithms in software engineering,” Software Testing, Verification and Reliability, vol. 24, no. 3, pp. 219–250, 2014

    work page 2014

  22. [22]

    A fast and elitist multiobjective genetic algorithm: NSGA-II,

    K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, “A fast and elitist multiobjective genetic algorithm: NSGA-II,” IEEE Transactions on Evolutionary Computation, vol. 6, no. 2, pp. 182–197, 2002

    work page 2002

  23. [23]

    Multiobjective optimization using evolutionary algorithms — a comparative case study,

    E. Zitzler and L. Thiele, “Multiobjective optimization using evolutionary algorithms — a comparative case study,” in Parallel Problem Solving from Nature — PPSN V (A. E. Eiben, T. B ¨ack, M. Schoenauer, and H.- P. Schwefel, eds.), (Berlin, Heidelberg), pp. 292–301, Springer Berlin Heidelberg, 1998

    work page 1998

  24. [24]

    Evaluating the quality of approxima- tions to the non-dominated set,

    M. Hansen and A. Jaszkiewicz, “Evaluating the quality of approxima- tions to the non-dominated set,” Tech. Rep. IMM-REP-1998-7, Institute of Computing Science, Poznan University of Technology, 03 1998

    work page 1998

  25. [25]

    On a test of whether one of two random variables is stochastically larger than the other,

    H. B. Mann and D. R. Whitney, “On a test of whether one of two random variables is stochastically larger than the other,” Ann. Math. Statist., vol. 18, pp. 50–60, 03 1947

    work page 1947

  26. [26]

    Statistical power analysis for the behaviors science.(2nd),

    J. Cohen, “Statistical power analysis for the behaviors science.(2nd),” New Jersey: Laurence Erlbaum Associates, Publishers, Hillsdale , 1988

    work page 1988

  27. [27]

    Automatic generation of atomic consistency preserving searchoperators for search-based model engi- neering - accompanying data

    A. Burdusel, S. Zschaler, and S. John, “Automatic generation of atomic consistency preserving searchoperators for search-based model engi- neering - accompanying data.” http://dx.doi.org/10.6084/m9.figshare. 8316740

    work page doi:10.6084/m9

  28. [28]

    A graph solver for the auto- mated generation of consistent domain-specific models,

    O. Semer ´ath, A. S. Nagy, and D. Varr ´o, “A graph solver for the auto- mated generation of consistent domain-specific models,” in Proceedings of the 40th International Conference on Software Engineering , pp. 969– 980, ACM, 2018

    work page 2018

  29. [29]

    Symbolic model generation for graph properties,

    S. Schneider, L. Lambers, and F. Orejas, “Symbolic model generation for graph properties,” in Fundamental Approaches to Software Engineering, (Berlin, Heidelberg), pp. 226–243, Springer Berlin Heidelberg, 2017

    work page 2017

  30. [30]

    The Class Responsibility Assign- ment Case,

    M. Fleck, J. Troya, and M. Wimmer, “The Class Responsibility Assign- ment Case,” in Garcia-Dominguez et al. [36]

    work page

  31. [31]

    A hyper-heuristic approach to automated generation of mutation operators for evolutionary programming,

    L. Hong, J. H. Drake, J. R. Woodward, and E. zcan, “A hyper-heuristic approach to automated generation of mutation operators for evolutionary programming,” Applied Soft Computing , vol. 62, pp. 162 – 175, 2018

    work page 2018

  32. [32]

    A systematic approach for designing mutation operators for mde languages,

    F. H. M. Alhwikem, R. F. Paige, L. M. Rose, and R. D. Alexander, “A systematic approach for designing mutation operators for mde languages,” in Proceedings of the 13th Workshop on Model-Driven Engineering, Verification and Validation co-located with ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems (MODELS 2016) , 2016

    work page 2016

  33. [33]

    A complete operator library for dsl evolution specification,

    J. G. M. Mengerink, A. Serebrenik, R. R. H. Schiffelers, and M. G. J. v. d. Brand, “A complete operator library for dsl evolution specification,” in 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp. 144–154, Oct 2016

    work page 2016

  34. [34]

    The right mutation strength for multi-valued decision variables,

    B. Doerr, C. Doerr, and T. K ¨otzing, “The right mutation strength for multi-valued decision variables,” in Proceedings of the Genetic and Evolutionary Computation Conference 2016 , pp. 1115–1122, ACM, 2016

    work page 2016

  35. [35]

    Deriving persuasion strategies using search-based model engineering,

    J. Murphy, A. Burdusel, L. Michael, S. Zschaler, and E. Black, “Deriving persuasion strategies using search-based model engineering,” in Compu- tational Models of Argument: Proceedings of COMMA 2018 , vol. 305, pp. 221–232, IOS Press, 2018

    work page 2018

  36. [36]

    Garcia-Dominguez, F

    A. Garcia-Dominguez, F. Krikava, and L. M. Rose, eds., Proceedings of the 9th Transformation Tool Contest , vol. 1758, CEUR, 2016

    work page 2016