Pith. sign in

REVIEW 2 major objections 6 minor 2 cited by

A Survey of Constrained Combinatorial Testing

T0 review · 2 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Only 32% of combinatorial testing studies handle constraints, a new survey of 129 papers finds.

desk verdict A genuinely useful survey of constrained combinatorial testing—the taxonomy is the contribution—but the motivating 32% statistic needs a clearer operational definition before the paper goes out. read the letter →

arxiv 1908.02480 v1 pith:WTUSDK6U submitted 2019-08-07 cs.SE

Huayao Wu , Changhai Nie , Justyna Petke , Yue Jia , Mark Harman This is my paper · ORCID
classification cs.SE
keywords combinatorialtestingconstrainedcoveringarrayconstrainthandlingidentificationmaintenancetestsuitegenerationsurvey
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This survey of 129 papers on constrained combinatorial testing organizes the field into three activities: identifying constraints, handling them during test suite generation, and maintaining them as test models evolve. It reports that only 32% of combinatorial test suite generation studies incorporate constraint-handling techniques, with the proportion staying near 30% even in the most recent years covered. The paper's central claim is that constraints remain a pressing open problem for combinatorial testing, because real-world programs are usually constrained and ignoring constraints produces invalid test cases and false confidence. The survey maps six constraint representations and four constraint-handling strategies, showing that handling dominates the literature while identification and maintenance are comparatively neglected.

What carries the argument

The organizing device is the constrained covering array, a covering array whose every row satisfies the constraints and whose coverage requirement applies only to valid $ au$-way combinations, together with the survey's three-part classification of constraint identification, handling, and maintenance. The classification does the argumentative work: grouping the 129 papers by research topic lets the authors compute distributions, such as 106 handling papers versus a handful on identification and maintenance, and thereby identify the under-studied areas. Within handling, the four-way division into Remodel, Avoid, Post-process, and Transfer, with Avoid and Transfer together accounting for 83% of handling studies, frames the claim that the field still has open problems and room for new techniques.

What would settle it

Re-run the same literature search with an independent team and check two things: whether the 32% proportion of generation studies with constraint handling still holds, and whether independent raters reproduce the identification/handling/maintenance classifications on a sample of the 129 papers. A substantially higher proportion of handling-aware generation studies, or low inter-rater agreement on the taxonomy, would undercut the survey's central characterisation.

Watch

Extended reading notes

Core claim

The central discovery is a map of constrained combinatorial testing: research in the area splits into constraint identification, constraint handling, and constraint maintenance, and handling techniques divide into Remodel (rewrite the test model), Avoid (keep generation constraint-free), Post-process (repair invalid tests after generation), and Transfer (reduce the problem to constraint satisfaction or graph problems). Alongside this taxonomy, the authors document that constraint handling is the dominant research line, accounting for 106 of the 129 surveyed papers, and that only 32% of test suite generation studies implement constraint handling at all. They also catalog six ways of representing constraints: forbidden tuples, implication relations, numeric relations, shielding constraints, counter and value properties, and embedded functions. The survey concludes that there is no agreed best representation or best handling technique, that identification and maintenance are under-studied, and that more powerful automated algorithms plus comparative evaluation are needed.

Load-bearing premise

The survey assumes that its literature search, based on six databases plus snowballing and manual filtering, captured all relevant work and that the manual assignment of each paper to a category is accurate; if relevant venues were missed or classifications are skewed, the reported percentages and the taxonomy itself could misrepresent the field.

Editorial extensions

If this is right

  • Practitioners working on constrained systems should treat unconstrained combinatorial testing tools as potentially generating invalid test cases, so tool selection and extension should explicitly account for constraint support.
  • Because only about a third of generation studies handle constraints, there is a clear opening for automated constraint-handling algorithms that are both efficient and applicable to large models; the authors call for exactly such algorithms and for comparative evaluation.
  • Constraint identification and maintenance are far less studied than handling, so advances in automatically inferring, validating, and repairing constraints could have an outsized effect on making constrained combinatorial testing practical.
  • Representation choices matter in practice: some constraint forms, such as numeric, shielding, counter/value, and embedded functions, cannot always be converted cheaply into forbidden tuples, so the choice of representation is itself a performance and usability decision.
  • The recent appearance of the Tolerate technique, alongside the long-standing dominance of Avoid and Transfer, indicates that the handling toolbox is still evolving and that no single approach has emerged as a standard.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the reported 32% figure generalizes beyond the surveyed literature, a large share of combinatorial testing practice likely runs on the unconstrained assumption; an industrial survey measuring how often constraints are modeled at all would be a direct test of this inference.
  • The taxonomy suggests a concrete research program: automated constraint identification from specifications, execution traces, or natural-language documents, combined with semantic differencing of test models, could remove the manual modeling bottleneck the survey identifies.
  • The top three handling techniques are Constraint Satisfaction Problem, Solver, and Verify, which suggests that SAT/SMT-based constraint solving has effectively become the de facto baseline for constrained test generation; future algorithms may do well to treat a solver as a standard component rather than an optional extension.
  • The survey's distinction between hard and soft constraints, and between system-wide and test-case-specific constraints, points toward a richer modeling language for combinatorial testing that could connect naturally to configurable software product lines.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. This paper presents a literature survey of constrained combinatorial testing (CT). The authors searched six digital libraries, applied inclusion/exclusion criteria, snowballed references in major venues, and assembled a public repository; from this they selected 129 constraint-related papers published between 1987 and 2018. The survey organizes these papers into three research topics—constraint identification, constraint handling, and constraint maintenance—and, within constraint handling, into four technique categories (Remodel, Avoid, Post-process, Transfer) with further sub-techniques. It also reviews constraint representations (forbidden, implication, numeric, shielding, counter/value property, embedded function) and the impact of constraints on CT. The paper's main motivating statistic is that only 32% of CT test suite generation studies incorporate constraint handling (30% in the last four years), which frames constraint handling as an open problem.

Significance. If the classification is accurate, this is a valuable reference for the CT community. Its strengths are a transparent search protocol, a public repository with a GitHub mirror, broad temporal coverage, and the first systematic organization of constraint identification and maintenance as distinct research areas. The taxonomy and the observation that identification and maintenance are under-studied can guide future research. The significance is moderated by the manual judgment underlying the topic assignments and by the lack of raw counts for some headline statistics, but these issues are addressable rather than fundamental.

major comments (2)
  1. [Section 1, Figure 1] The claim that 'only 32% of test suite generation studies have incorporated constraint handling techniques' is not reproducible from the manuscript as written. Figure 1 contains no raw counts, and the text does not specify the denominator (presumably the 'Generation' field in the repository described in Section 2.1) or the operational criterion for the numerator (the caption says 'papers on constraint support in CT' while the text says 'incorporated constraint handling techniques'). Since Section 2.1 assigns each paper to exactly one field based on its main contribution and Section 5 acknowledges that some classifications are uncertain, the 32% and 30% figures could shift under reasonable alternative readings. Please add a table with year-by-year raw counts and an explicit statement of which repository records and which criterion were used, or point to a specific queryable file in the GitHub mirror.
  2. [Section 5, Figure 2 and Figure 3] The paper's central quantitative observation is that constraint handling dominates the field (106 papers), but the reliability of the manual classification is not demonstrated. The text states that 'a few papers do not provide sufficient information to support a fully confident classification' yet provides no list of these papers and no sensitivity analysis. Because Figure 3's percentages (e.g., 83% of studies using Avoid/Transfer) and the 106-paper count are computed from this manual assignment, I ask the authors to (i) identify the uncertain papers, (ii) state the assignment rule used in each case, and (iii) show whether the main conclusions change if those papers are excluded or reassigned.
minor comments (6)
  1. [Section 3.1] The text 'all 32×23 = 72 test cases' is arithmetically incorrect; it should read 3^2 × 2^3 = 72.
  2. [Section 3.1, Definition 3] 'contains only element' should be 'contains only elements'.
  3. [Section 4] 'Nyguyen and Tonella' should be 'Nguyen and Tonella', matching reference [135].
  4. [Section 5.4.2] 'Multivalued Decision Digram' should be 'Multivalued Decision Diagram', and 'Gargntini' should be 'Gargantini'.
  5. [Table 4] In the Solver row, the reference list contains a duplicate '[65]'.
  6. [Figures 2 and 3] The figures would be easier to interpret if raw counts were printed on or below the bars, not only percentages.

Circularity Check

0 steps flagged · score 0.0 of 10

The survey's classifications and 32% observation are descriptive summaries of an external literature corpus, with no fitted input renamed as a prediction and no load-bearing self-citation chain; no circular step is exhibited.

full rationale

The paper is a literature survey, not a predictive derivation. Its central claims are a taxonomy of 129 constraint-related combinatorial testing papers into identification, handling, and maintenance, and an observed statistic that only 32% of test suite generation studies have incorporated constraint handling techniques. Both claims are summaries of a manually constructed repository and of external papers, not outputs that are fed back into the inputs by construction. The manuscript does use the authors' own prior combinatorial testing survey and repository ([3], Section 2.1) and one co-authored earlier survey ([24]) to organize the search and background. These self-citations are descriptive infrastructure rather than load-bearing evidence: the new three-category classification and the literature review are synthesized from the 129 selected external papers. The 32% figure is a count over the repository, with the inclusion criteria stated in Section 2.2 and the repository publicly archived; Section 5 candidly notes that 'a few papers do not provide sufficient information to support a fully confident classification,' which is a reproducibility and classification-judgment caveat, not circularity. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the authors' prior work, no ansatz is smuggled in via citation, and no known result is merely renamed. Therefore no circular step is identified and the appropriate score is 0.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The survey's conclusions rest on the completeness and classification of the literature, not on mathematical derivations. No free parameters or invented entities are present. The main assumptions are the coverage of the literature search and the reliability of manual topic classification, both acknowledged in the paper's methodology.

assumptions (2)
  • domain assumption The six selected digital libraries and search queries cover the relevant universe of constrained combinatorial testing literature.
    Section 2.1 lists IEEE Xplore, ACM DL, Elsevier, Springer, Wiley, and DBLP with queries including 'combinatorial testing' and 'covering array'; completeness is asserted but not proven.
  • domain assumption Manual filtering and classification of the 129 papers is sufficiently accurate to support the survey's conclusions.
    Sections 2.1 and 2.2 describe manual inspection of titles, abstracts, and full texts; the authors admit in Section 5 that some classifications are not fully confident, making this an accepted risk.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Survey of Constrained Combinatorial Testing." pith.science (2026). https://pith.science/paper/WTUSDK6U

@misc{pith2026190802480,
  author       = {Pith},
  title        = {Pith review of: A Survey of Constrained Combinatorial Testing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WTUSDK6U}},
  note         = {Machine review of arXiv:1908.02480}
}
read the original abstract

Combinatorial Testing (CT) is a potentially powerful testing technique, whereas its failure revealing ability might be dramatically reduced if it fails to handle constraints in an adequate and efficient manner. To ensure the wider applicability of CT in the presence of constrained problem domains, large and diverse efforts have been invested towards the techniques and applications of constrained combinatorial testing. In this paper, we provide a comprehensive survey of representations, influences, and techniques that pertain to constraints in CT, covering 129 papers published between 1987 and 2018. This survey not only categorises the various constraint handling techniques, but also reviews comparatively less well-studied, yet potentially important, constraint identification and maintenance techniques. Since real-world programs are usually constrained, this survey can be of interest to researchers and practitioners who are looking to use and study constrained combinatorial testing techniques.

Figures

Figures reproduced from arXiv: 1908.02480 by the authors.

Figure 1
Figure 1. Number of research papers on CT test suite generation, and the papers on constraint support in CT by year. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Distribution of research topics in constrained combinatorial test [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The distribution of constraint handling techniques used in the [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The chronological development of constraint handling techniques. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: An example of sub-model technique. 5.1.1 Sub-model The ‘sub-model’ technique removes constraints by con￾structing a set of conflict-free sub-models. Test suites are generated separately for each sub-model and then com￾bined later. This idea was initially sketched in 19…
Figure 6
Figure 6. Figure 6: An example of abstract parameter technique. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Optimal Combinatorial Testing with Constraints: The Balancing Act

    math.OC 2026-07 conditional novelty 6.0 of 10

    A new exact integer-programming method and structural bounds for pairwise covering arrays with forbidden binary assignments, plus a fast heuristic that often matches optimal solutions.

  2. How Low Can We Go? Minimizing Interaction Samples for Configurable Systems

    cs.SE 2025-01 conditional novelty 5.0 of 10

    SampLNS combines a mutually-exclusive-interactions lower bound with large neighborhood search to minimize pairwise interaction test samples and certify optimality.

Reference graph

Works this paper leans on

145 extracted references · 80 canonical work pages · cited by 2 Pith papers

  1. [65]

    An algorithm for generating t-wise covering arrays from large feature models,

    M. F. Johansen, O. Haugen, and F. Fleurey, “An algorithm for generating t-wise covering arrays from large feature models,” in International Software Product Line Conference, 2012, pp. 46–55

    2012

  2. [1]

    Orthogonal latin squares: An application of experi- mental design to compiler testing,

    R. Mandl, “Orthogonal latin squares: An application of experi- mental design to compiler testing,” Communications of the ACM , vol. 28, no. 10, pp. 1054–1058, 1985

    1985

  3. [2]

    Software fault interactions and implications for software testing,

    D. R. Kuhn and D. R. Wallace, “Software fault interactions and implications for software testing,” IEEE Transactions on Software Engineering, vol. 30, no. 6, pp. 418–421, 2004

    2004

  4. [3]

    A survey of combinatorial testing,

    C. Nie and H. Leung, “A survey of combinatorial testing,” ACM Computing Surveys, vol. 43, no. 2, pp. 11:1–11:29, 2011

    2011

  5. [4]

    Constraints: The future of combinatorial interaction testing,

    J. Petke, “Constraints: The future of combinatorial interaction testing,” in International Workshop on Search-Based Software Testing, 2015, pp. 17–18

    2015

  6. [5]

    Effective testing of factor combinations,

    G. Sherwood, “Effective testing of factor combinations,” in In- ternational Conference on Software Testing, Analysis & Review , 1994, pp. 1–16

    1994

  7. [6]

    Handling constraints in the input space when using combination strategies for software testing,

    M. Grindal, J. Offutt, and J. Mellin, “Handling constraints in the input space when using combination strategies for software testing,” School of Humanities and Informatics, Tech. Rep. HS- IKI-TR-06-01, 2006

    2006

  8. [7]

    Constructing interaction test suites for highly-configurable systems in the presence of constraints: A greedy approach,

    M. B. Cohen, M. B. Dwyer, and J. Shi, “Constructing interaction test suites for highly-configurable systems in the presence of constraints: A greedy approach,” IEEE Transactions on Software Engineering, vol. 34, no. 5, pp. 633–650, 2008

    2008

Show all 145 references
  1. [8]

    Evaluating im- provements to a meta-heuristic search for constrained interaction testing,

    B. J. Garvin, M. B. Cohen, and M. B. Dwyer, “Evaluating im- provements to a meta-heuristic search for constrained interaction testing,” Empirical Software Engineering, vol. 16, no. 1, pp. 61–102, 2010

    2010

  2. [9]

    Combina- torial test generation for software product lines using minimum invalid tuples,

    L. Yu, F. Duan, Y. Lei, R. N. Kacker, and D. R. Kuhn, “Combina- torial test generation for software product lines using minimum invalid tuples,” in International Symposium on High Assurance System Engineering, 2014, pp. 65–72

    2014

  3. [10]

    Constraint handling in combinatorial test generation using forbidden tuples,

    ——, “Constraint handling in combinatorial test generation using forbidden tuples,” in 4th International Workshop on Combinatorial Testing, 2015, pp. 1–9

    2015

  4. [11]

    Towards automatic constraints elicitation in pair-wise testing based on a linguistic approach: elicitation support using coupling strength,

    H. Nakagawa and T. Tsuchiya, “Towards automatic constraints elicitation in pair-wise testing based on a linguistic approach: elicitation support using coupling strength,” in International Workshop on Requirements Engineering and Testing , 2015, pp. 34– 36

    2015

  5. [12]

    Towards automatic constraint elicitation in test design: Preliminary evaluation based on collective intelligence,

    ——, “Towards automatic constraint elicitation in test design: Preliminary evaluation based on collective intelligence,” in In- ternational Workshop on Automated Software Engineering , 2015, pp. 58–61

    2015

  6. [13]

    Visualization of com- binatorial models and test plans,

    R. Tzoref-Brill, P . Wojciak, and S. Maoz, “Visualization of com- binatorial models and test plans,” in International Conference on Automated Software Engineering, 2016, pp. 144–154

    2016

  7. [14]

    Val- idation of constraints among configuration parameters using search-based combinatorial interaction testing,

    A. Gargantini, J. Petke, M. Radavelli, and P . Vavassori, “Val- idation of constraints among configuration parameters using search-based combinatorial interaction testing,” in International Symposium on Search Based Software Engineering, 2016, pp. 49–63

    2016

  8. [15]

    Combinatorial inter- action testing for automated constraint repair,

    A. Gargantini, J. Petke, and M. Radavelli, “Combinatorial inter- action testing for automated constraint repair,” in International Workshop on Combinatorial Testing, 2017, pp. 239–248

    2017

  9. [16]

    An orchestrated survey of available algorithms and tools for combinatorial testing,

    S. K. Khalsa and Y. Labiche, “An orchestrated survey of available algorithms and tools for combinatorial testing,” in International Symposium on Software Reliability Engineering, 2014, pp. 323–334

    2014

  10. [17]

    Finite field construc- tions of combinatorial arrays,

    L. Moura, G. L. Mullen, and D. Panario, “Finite field construc- tions of combinatorial arrays,” Designs, Codes and Cryptography , vol. 78, no. 1, pp. 197–219, 2016

    2016

  11. [18]

    Upper bounds on the size of covering arrays,

    K. Sarkar and C. J. Colbourn, “Upper bounds on the size of covering arrays,” SIAM Journal on Discrete Mathematics , vol. 31, no. 2, pp. 1277–1293, 2017

    2017

  12. [19]

    Partial covering arrays: algorithms and asymptotics,

    K. Sarkar, C. J. Colbourn, A. De Bonis, and U. Vaccaro, “Partial covering arrays: algorithms and asymptotics,” Theory of Comput- ing Systems, vol. 62, no. 6, pp. 1470–1489, 2018

    2018

  13. [20]

    Algebraic models for arbitrary strength cover- ing arrays over v-ary alphabets,

    L. Kampel, D. E. Simos, B. Garn, I. S. Kotsireas, and E. Zhereshchin, “Algebraic models for arbitrary strength cover- ing arrays over v-ary alphabets,” in International Conference on Algebraic Informatics, 2019, pp. 177–189

    2019

  14. [21]

    Constructions of optimal orthogonal arrays with repeated rows,

    C. J. Colbourn, D. R. Stinson, and S. Veitch, “Constructions of optimal orthogonal arrays with repeated rows,” Discrete Mathe- matics, vol. 342, no. 9, pp. 2455–2466, 2019

    2019

  15. [22]

    Survey of covering arrays,

    J. Torres-Jimenez and I. Izquierdo-Marquez, “Survey of covering arrays,” in International Symposium on Symbolic and Numeric Algo- rithms for Scientific Computing, 2013, pp. 20–27

    2013

  16. [23]

    A bibliometric assessment of software engineering schol- ars and institutions (2010–2017),

    D. Karanatsiou, Y. Li, E.-M. Arvanitou, N. Misirlis, and W. E. Wong, “A bibliometric assessment of software engineering schol- ars and institutions (2010–2017),” Journal of Systems and Software , vol. 147, pp. 246–261, 2019

    2010

  17. [24]

    Constrained interaction testing: A systematic literature study,

    B. S. Ahmed, K. Z. Zamli, W. Afzal, and M. Bures, “Constrained interaction testing: A systematic literature study,” IEEE Access , vol. 5, pp. 25 706–25 730, 2017

    2017

  18. [25]

    Combinatorial test design in practice,

    M. B. Cohen and S. Ur, “Combinatorial test design in practice,” in International Conference on Software Engineering, 2010, pp. 495–496

    2010

  19. [26]

    An empirical com- parison of combinatorial testing, random testing and adaptive random testing,

    H. Wu, J. Petke, Y. Jia, M. Harman et al. , “An empirical com- parison of combinatorial testing, random testing and adaptive random testing,” IEEE Transactions on Software Engineering, 2018

    2018

  20. [27]

    Interaction testing of highly-configurable systems in the presence of constraints,

    M. B. Cohen, M. B. Dwyer, and J. Shi, “Interaction testing of highly-configurable systems in the presence of constraints,” in International Symposium on Software Testing and Analysis, 2007, pp. 129–139

    2007

  21. [28]

    Prioritized interaction testing for pair-wise coverage with seeding and constraints,

    R. C. Bryce and C. J. Colbourn, “Prioritized interaction testing for pair-wise coverage with seeding and constraints,” Information and Software Technology, vol. 48, no. 10, pp. 960–970, 2006. 14

    2006

  22. [29]

    The density algorithm for pairwise interaction testing,

    ——, “The density algorithm for pairwise interaction testing,” Software Testing, Verification and Reliability, vol. 17, no. 3, pp. 159– 182, 2007

    2007

  23. [30]

    Test case-aware combinatorial interaction testing,

    C. Yilmaz, “Test case-aware combinatorial interaction testing,” IEEE Transactions on Software Engineering , vol. 39, no. 5, pp. 684– 706, 2012

    2012

  24. [31]

    CITLAB: A laboratory for com- binatorial interaction testing,

    A. Gargantini and P . Vavassori, “CITLAB: A laboratory for com- binatorial interaction testing,” in 1st International Workshop on Combinatorial Testing, 2012, pp. 559–568

    2012

  25. [32]

    Numerical constraints for combinatorial interaction testing,

    P . M. Kruse, J. Bauer, and J. Wegener, “Numerical constraints for combinatorial interaction testing,” in International Conference on Software Testing, Verification and Validation, 2012, pp. 758–763

    2012

  26. [33]

    Combinatorial testing with shield- ing parameters,

    B. Chen, J. Yan, and J. Zhang, “Combinatorial testing with shield- ing parameters,” in Asia-Pacific Software Engineering Conference , 2010, pp. 280–289

    2010

  27. [34]

    Simplified modeling of combinatorial test spaces,

    I. Segall, R. Tzoref-Brill, and A. Zlotnick, “Simplified modeling of combinatorial test spaces,” in 1st International Workshop on Combinatorial Testing, 2012, pp. 573–579

    2012

  28. [35]

    Embedded functions in combinatorial test de- signs,

    G. Sherwood, “Embedded functions in combinatorial test de- signs,” in 4th International Workshop on Combinatorial Testing, 2015, pp. 1–10

    2015

  29. [36]

    Embedded functions for constraints and variable strength in combinatorial testing,

    ——, “Embedded functions for constraints and variable strength in combinatorial testing,” in 5th International Workshop on Combi- natorial Testing, 2016, pp. 65–74

    2016

  30. [37]

    Practical combi- natorial interaction testing: Empirical findings on efficiency and early fault detection,

    J. Petke, M. B. Cohen, M. Harman, and S. Yoo, “Practical combi- natorial interaction testing: Empirical findings on efficiency and early fault detection,” IEEE Transactions on Software Engineering , vol. 41, no. 9, pp. 901–924, 2015

    2015

  31. [38]

    Efficiency and early fault detection with lower and higher strength com- binatorial interaction testing,

    J. Petke, S. Yoo, M. B. Cohen, and M. Harman, “Efficiency and early fault detection with lower and higher strength com- binatorial interaction testing,” in InternationalSymposium on the Foundations of Software Engineering, 2013, pp. 26–36

    2013

  32. [39]

    The AETG system: An approach to testing based on combinatorial design,

    D. M. Cohen, S. R. Dalal, M. L. Fredman, and G. C. Patton, “The AETG system: An approach to testing based on combinatorial design,” IEEE Transactions on Software Engineering , vol. 23, no. 7, pp. 437–444, 1997

    1997

  33. [40]

    Managing conflicts when us- ing combination strategies to test software,

    M. Grindal, J. Offutt, and J. Mellin, “Managing conflicts when us- ing combination strategies to test software,” inAustralian Software Engineering Conference, 2007, pp. 255–264

    2007

  34. [41]

    Input-input relationship con- straints in t-way testing,

    R. R. Othman and K. Z. Zamli, “Input-input relationship con- straints in t-way testing,” in Interational Symposium on Industrial Electronics and Applications, 2011, pp. 527–531

    2011

  35. [42]

    A practical strategy for testing pair-wise coverage of network interfaces,

    A. W. Williams and R. L. Probert, “A practical strategy for testing pair-wise coverage of network interfaces,” in International Conference on Software Reliability Engineering, 1996, pp. 246–254

    1996

  36. [43]

    Automating test case genera- tion for the new generation mission software system,

    Y.-W. Tung and W. S. Aldiwan, “Automating test case genera- tion for the new generation mission software system,” in IEEE Arospace Conference, 2000, pp. 431–437

    2000

  37. [44]

    Pairwise testing in real world: Practical exten- sions to test case generator,

    J. Czerwonka, “Pairwise testing in real world: Practical exten- sions to test case generator,” in Pacific Northwest Software Quality Conference, 2006, pp. 419–430

    2006

  38. [45]

    Testing product generation in software product lines using pairwise for features coverage,

    B. P . Lamancha and M. Polo, “Testing product generation in software product lines using pairwise for features coverage,” in Testing Software and Systems, 2010, pp. 111–125

    2010

  39. [46]

    A new method of reducing pair-wise combinatorial test suite,

    L. Wang and R. Wan, “A new method of reducing pair-wise combinatorial test suite,” Computer and Information Science, vol. 3, no. 1, pp. 35–41, 2010

    2010

  40. [47]

    Design and implementation of a harmony-search-based variable-strength t-way testing strategy with constraints support,

    A. Alsewari and K. Z. Zamli, “Design and implementation of a harmony-search-based variable-strength t-way testing strategy with constraints support,” Information and Software Technology , vol. 54, no. 6, pp. 553–568, 2012

    2012

  41. [48]

    Constraints dependent t-way test suite generation using harmony search strategy,

    ——, “Constraints dependent t-way test suite generation using harmony search strategy,” in Pacific Rim Knowledge Acquisition Workshop, 2012, pp. 1–11

    2012

  42. [49]

    T-wise combinatorial inter- action test suites construction based on coverage inheritance,

    A. Calvagna and A. Gargantini, “T-wise combinatorial inter- action test suites construction based on coverage inheritance,” Software Testing, Verification and Reliability, vol. 22, no. 7, pp. 507– 526, 2012

    2012

  43. [50]

    Combinatorial test cases with constraints in software systems,

    L. Li, Y. Cui, and Y. Yang, “Combinatorial test cases with constraints in software systems,” in International Conference on Computer Supported Cooperative Work in Design, 2012, pp. 195–199

    2012

  44. [51]

    ACTS: a combinato- rial test generation tool,

    L. Yu, Y. Lei, R. N. Kacker, and D. R. Kuhn, “ACTS: a combinato- rial test generation tool,” in6th International Conference on Software Testing, Verification and Validation, 2013, pp. 370–375

    2013

  45. [52]

    Variable strength t-way test suite generator with constraints support,

    R. R. Othman, N. Khamis, and K. Z. Zamli, “Variable strength t-way test suite generator with constraints support,” Malaysian Journal of Computer Science, vol. 27, no. 3, pp. 204–217, 2014

    2014

  46. [53]

    Generating com- binatorial test cases using simplified swarm optimization (SSO) algorithm for automated GUI functional testing,

    B. S. Ahmed, M. A. Sahib, and M. Y. Potrus, “Generating com- binatorial test cases using simplified swarm optimization (SSO) algorithm for automated GUI functional testing,” Engineering Science and Technology, an International Journal , vol. 17, no. 4, pp. 218–226, 2014

    2014

  47. [54]

    Combi- natorial testing with order requirements,

    E. Farchi, I. Segall, R. Tzoref-Brill, and A. Zlotnick, “Combi- natorial testing with order requirements,” in 3rd International Workshop on Combinatorial Testing, 2014, pp. 118–127

    2014

  48. [55]

    SPLBA: An interaction strategy for testing software product lines using the bat-inspired algorithm,

    Y. A. Alsariera, M. A. Majid, and K. Z. Zamli, “SPLBA: An interaction strategy for testing software product lines using the bat-inspired algorithm,” in International Conference on Software Engineering and Computer Systems, 2015, pp. 148–153

    2015

  49. [56]

    PROW: A pairwise algorithm with constraints, order and weight,

    B. P . Lamancha, M. Polo, and M. Piattini, “PROW: A pairwise algorithm with constraints, order and weight,” Journal of Systems and Software, vol. 99, pp. 1–19, 2015

    2015

  50. [57]

    Automatic test case generation for WS-agreements using com- binatorial testing,

    M. Palaciosa, J. Garcia-Fanjula, J. Tuyaa, and G. Spanoudakisb, “Automatic test case generation for WS-agreements using com- binatorial testing,” Computer Standards & Interfaces , vol. 38, pp. 84–100, 2015

    2015

  51. [58]

    Ef- fective test generation for combinatorial decision coverage,

    R. Gao, L. Hu, W. E. Wong, H.-L. Lu, and S.-K. Huang, “Ef- fective test generation for combinatorial decision coverage,” in International Conference on Software Quality, Reliability and Security Companion, 2016, pp. 47–54

    2016

  52. [59]

    An extension of category partition testing for highly constrained systems,

    S. K. Khalsa and Y. Labiche, “An extension of category partition testing for highly constrained systems,” in International Sympo- sium on High Assurance Systems Engineering , 2016, pp. 47–54

    2016

  53. [60]

    Extending category partition’s base choice criterion to bet- ter support constraints,

    ——, “Extending category partition’s base choice criterion to bet- ter support constraints,” Journal of Software: Evolution and Process, vol. 30, no. 3, pp. 1–23, 2018

    2018

  54. [61]

    Exploiting constraint solv- ing history to construct interaction test suites,

    M. B. Cohen, M. B. Dwyer, and J. Shi, “Exploiting constraint solv- ing history to construct interaction test suites,” in Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques, 2007, pp. 121–132

    2007

  55. [62]

    An improved meta- heuristic search for constrained interaction testing,

    B. J. Garvin, M. B. Cohen, and M. B. Dwyer, “An improved meta- heuristic search for constrained interaction testing,” in Interna- tional Symposium on Search Based Software Engineering , 2009, pp. 13–22

    2009

  56. [63]

    Improved extremal optimization for constrained pairwise testing,

    J. Yuan, C. Jiang, and Z. Jiang, “Improved extremal optimization for constrained pairwise testing,” in International Conference on Research Challenges in Computer Science, 2009, pp. 108–111

    2009

  57. [64]

    Automated incremental pair- wise testing of software product lines,

    S. Oster, F. Markert, and P . Ritter, “Automated incremental pair- wise testing of software product lines,” in Software Product Lines: Going Beyond, 2010, pp. 196–210

    2010

  58. [66]

    An efficient algorithm for constraint handling in combinatorial test generation,

    L. Yu, Y. Lei, M. N. Borazjany, R. N. Kacker, and D. R. Kuhn, “An efficient algorithm for constraint handling in combinatorial test generation,” in International Conference on Software Testing, Verification and Validation, 2013, pp. 242–251

    2013

  59. [67]

    Using feature model knowledge to speed up the generation of cover- ing arrays,

    E. N. Haslinger, R. E. Lopez-Herrejon, and A. Egyed, “Using feature model knowledge to speed up the generation of cover- ing arrays,” in International Workshop on Variability Modelling of Software-intensive Systems, 2013, pp. 16:1–16:6

    2013

  60. [68]

    An efficient algorithm for pairwise test case generation in presence of constraints,

    S. Gao, B. Du, Y. Jiang, J. Lv, and S. Ma, “An efficient algorithm for pairwise test case generation in presence of constraints,” in International Conference on Systems and Informatics , 2014, pp. 406– 410

    2014

  61. [69]

    Applying random test- ing to constrained interaction testing,

    Y. Hirasaki, H. Kojima, and T. Tsuchiya, “Applying random test- ing to constrained interaction testing,” in International Conference on Software Engineering and Knowledge Engineering , 2014, pp. 193– 198

    2014

  62. [70]

    A parallel evolutionary algorithm for prioritized pairwise testing of software product lines,

    R. E. Lopez-Herrejon, J. Ferrer, F. Chicano, E. N. Haslinger, A. Egyed, and E. Alba, “A parallel evolutionary algorithm for prioritized pairwise testing of software product lines,” in Annual Conference on Genetic and Evolutionary Computation , 2014, pp. 1255–1262

    2014

  63. [71]

    Learning combina- torial interaction test generation strategies using hyperheuristic search,

    Y. Jia, M. B. Cohen, M. Harman, and J. Petke, “Learning combina- torial interaction test generation strategies using hyperheuristic search,” in International Conference on Software Engineering , 2015, pp. 540–550

    2015

  64. [72]

    TCA: An efficient two-mode meta-heuristic algorithm for combinatorial test generation,

    J. Lin, C. Luo, S. Cai, K. Su, D. Hao, and L. Zhang, “TCA: An efficient two-mode meta-heuristic algorithm for combinatorial test generation,” in International Conference on Automated Software Engineering, 2015, pp. 1–12

    2015

  65. [73]

    In- cLing: efficient product-line testing using incremental pairwise 15 sampling,

    M. Al-Hajjaji, S. Krieter, T. Thum, M. Lochau, and G. Saake, “In- cLing: efficient product-line testing using incremental pairwise 15 sampling,” in International Conference on Generative Programming: Concepts and Experiences, 2016, pp. 144–155

    2016

  66. [74]

    Greedy combinatorial test case generation using unsatisfiable cores,

    A. Yamada, A. Biere, C. Artho, T. Kitamura, and E.-H. Choi, “Greedy combinatorial test case generation using unsatisfiable cores,” in International Conference on Automated Software Engineer- ing, 2016, pp. 614–624

    2016

  67. [75]

    Constraint test cases generation based on particle swarm optimization,

    Y. Sheng, C. Wei, and S. Jiang, “Constraint test cases generation based on particle swarm optimization,” International Journal of Reliability, Quality and Safety Engineering , vol. 24, no. 5, pp. 1–21, 2017

    2017

  68. [76]

    Recast IPOG-D algorithm with constraint handling for combinatorial testing,

    M. Shwetha, “Recast IPOG-D algorithm with constraint handling for combinatorial testing,” in International Conference on Recent Advances in Electronics and Communication Technology , 2017, pp. 179–185

    2017

  69. [77]

    Late acceptance hill climbing for constrained covering arrays,

    M. Bazargani, J. H. Drake, and E. K. Burke, “Late acceptance hill climbing for constrained covering arrays,” in International Conference on Applications of Evolutionary Computation , 2018, pp. 778–793

    2018

  70. [78]

    Enumerator: An efficient approach for enumerating all valid t-tuples,

    H. Mercan, K. Kaya, and C. Yilmaz, “Enumerator: An efficient approach for enumerating all valid t-tuples,” in International Workshops on Combinatorial Testing, 2018, pp. 302–305

    2018

  71. [79]

    Combinatorial testing with constraints for negative test cases,

    K. Fogen and H. Lichter, “Combinatorial testing with constraints for negative test cases,” inInternational Workshops on Combinatorial Testing, 2018, pp. 328–331

    2018

  72. [80]

    A penalty-based tabu search for constrained covering arrays,

    P . Galinier, S. Kpodjedo, and G. Antoniol, “A penalty-based tabu search for constrained covering arrays,” in Genetic and Evolution- ary Computation Conference, 2017, pp. 1288–1294

    2017

  73. [81]

    Handling constraints in combinatorial interaction testing in the presence of multi objective particle swarm and multithreading,

    B. S. Ahmed, L. M. Gambardella, W. Afzal, and K. Z. Zamli, “Handling constraints in combinatorial interaction testing in the presence of multi objective particle swarm and multithreading,” Information and Software Technology, vol. 86, pp. 20–36, 2017

    2017

  74. [82]

    Problems and algorithms for cover- ing arrays,

    A. Hartman and L. Raskin, “Problems and algorithms for cover- ing arrays,” Discrete Mathematics, vol. 284, no. 1-3, pp. 149–156, 2004

    2004

  75. [83]

    Repairing GUI test suites using a genetic algorithm,

    S. Huang, M. B. Cohen, and A. M. Memon, “Repairing GUI test suites using a genetic algorithm,” in International Conference on Software Testing, Verification and Validation, 2010, pp. 245–254

    2010

  76. [84]

    Constrained pairwise test case generation approach based on statistical user profile,

    S. Nakornburi and T. Suwannasart, “Constrained pairwise test case generation approach based on statistical user profile,” in International Multi Conference of Engineers and Computer Scientists , 2016, pp. 1–4

    2016

  77. [85]

    A tool for constrained pairwise test case generation using statistical user profile based prioritization,

    ——, “A tool for constrained pairwise test case generation using statistical user profile based prioritization,” in International Joint Conference on Computer Science and Software Engineering , 2016, pp. 1–6

    2016

  78. [86]

    Generating an automated test suite by variable strength combinatorial testing for web services,

    Y. Li, Z. an Sun, and J.-Y. Fang, “Generating an automated test suite by variable strength combinatorial testing for web services,” Journal of computing and information technology , vol. 24, no. 3, pp. 271–282, 2016

    2016

  79. [87]

    Covering array sampling of input event sequences for automated GUI testing,

    X. Yuan, M. B. Cohen, and A. M. Memon, “Covering array sampling of input event sequences for automated GUI testing,” in International Conference on Automated Software Engineering, 2007, pp. 405–408

    2007

  80. [88]

    GUI interaction testing: Incorporating event context,

    ——, “GUI interaction testing: Incorporating event context,” IEEE Transactions on Software Engineering , vol. 37, no. 4, pp. 559– 574, 2010

    2010

  81. [89]

    Constraint-based approaches to the covering test problem,

    B. Hnich, S. D. Prestwich, and E. Selensky, “Constraint-based approaches to the covering test problem,” in Joint Annual Work- shop of ERCIM/CoLogNet on Constraint Solving and Constraint Logic Programming, 2005, pp. 199–219

    2005

  82. [90]

    Con- straint models for the covering test problem,

    B. Hnich, S. D. Prestwich, E. Selensky, and B. M. Smith, “Con- straint models for the covering test problem,” Constraints, vol. 11, no. 23, pp. 199–219, 2006

    2006

  83. [91]

    Using SRI SAL model checker for combinatorial tests generation in the presence of temporal constraints,

    A. Calvagna and A. Gargantini, “Using SRI SAL model checker for combinatorial tests generation in the presence of temporal constraints,” in AFM Automated Formal Methods - workshop of CAV, 2008, pp. 1–10

    2008

  84. [92]

    A logic-based approach to combinatorial testing with constraints,

    ——, “A logic-based approach to combinatorial testing with constraints,” in International conference on Tests and proofs , 2008, pp. 66–83

    2008

  85. [93]

    Combining satisfiability solving and heuristics to con- strained combinatorial interaction testing,

    ——, “Combining satisfiability solving and heuristics to con- strained combinatorial interaction testing,” in International Con- ference on Tests and Proofs, 2009, pp. 27–42

    2009

  86. [94]

    Interaction coverage meets path coverage by smt constraint solving,

    W. Grieskamp, X. Qu, X. Wei, N. Kicillof, and M. B. Cohen, “Interaction coverage meets path coverage by smt constraint solving,” in International Conference on Testing of Software and Communication Systems, 2009, pp. 97–112

    2009

  87. [95]

    A formal logic approach to con- strained combinatorial testing,

    A. Calvagna and A. Gargantini, “A formal logic approach to con- strained combinatorial testing,” Journal of Automated Reasoning , vol. 45, no. 4, pp. 331–358, 2010

    2010

  88. [96]

    Au- tomated and scalable t-wise test case generation strategies for software product lines,

    G. Perrouin, S. Sen, J. Klein, B. Baudry, and Y. L. Traon, “Au- tomated and scalable t-wise test case generation strategies for software product lines,” in International Conference on Software Testing, Verification and Validation, 2010, pp. 459–468

    2010

  89. [97]

    Answer-set programming as a new approach to event- sequence testing,

    E. Erdem, K. Inoue, J. Oetsch, J. Puhrer, H. Tompits, and C. Yil- maz, “Answer-set programming as a new approach to event- sequence testing,” in 3rd International Conference on Advances in System Testing and Validation Lifecycle, 2011, pp. 25–34

    2011

  90. [98]

    PACOGEN: Automatic generation of pairwise test configurations from feature models,

    A. Hervieu, B. Baudry, and A. Gotlieb, “PACOGEN: Automatic generation of pairwise test configurations from feature models,” in International Symposium on Software Reliability Engineering, 2011, pp. 120–129

    2011

  91. [99]

    Properties of realis- tic feature models make combinatorial testing of product lines feasible,

    M. F. Johansen, O. Haugen, and F. Fleurey, “Properties of realis- tic feature models make combinatorial testing of product lines feasible,” in International conference on Model driven engineering languages and systems, 2011, pp. 638–652

    2011

  92. [100]

    Constructing test sets for pairwise testing: A sat-based approach,

    T. Nanba, T. Tsuchiya, and T. Kikuno, “Constructing test sets for pairwise testing: A sat-based approach,” in International Confer- ence on Networking and Computing, 2011, pp. 271–274

    2011

  93. [101]

    Using satisfiability solving for pairwise testing in the presence of constraints,

    ——, “Using satisfiability solving for pairwise testing in the presence of constraints,” IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences , vol. 95, no. 9, pp. 1501–1505, 2012

    2012

  94. [102]

    Minimum pairwise cov- erage using constraint programming techniques,

    A. Gotlieb, A. Hervieu, and B. Baudry, “Minimum pairwise cov- erage using constraint programming techniques,” in International Conference on Software Testing, Verification and Validation, 2012, pp. 773–774

    2012

  95. [103]

    PLEDGE: A product line editor and test generation tool,

    C. Henard, M. Papadakis, G. Perrouin, J. Klein, and Y. L. Traon, “PLEDGE: A product line editor and test generation tool,” in17th International Software Product Line Conference Workshops , 2013, pp. 126–129

    2013

  96. [104]

    Multi-objective optimal test suite computation for software product line pairwise testing,

    R. E. Lopez-Herrejon, F. Chicano, J. Ferrer, A. Egyed, and E. Alba, “Multi-objective optimal test suite computation for software product line pairwise testing,” in International Conference on Soft- ware Maintenance, 2013, pp. 404–407

    2013

  97. [105]

    Practical pairwise testing for software product lines,

    D. Marijan, A. Gotlieb, S. Sen, and A. Hervieu, “Practical pairwise testing for software product lines,” in 17th International Software Product Line Conference, 2013, pp. 227–235

    2013

  98. [106]

    Cascade: A test genera- tion tool for combinatorial testing,

    Y. Zhao, Z. Zhang, J. Yan, and J. Zhang, “Cascade: A test genera- tion tool for combinatorial testing,” in 2nd International Workshop on Combinatorial Testing, 2013, pp. 267–270

    2013

  99. [107]

    Multi-objective test generation for software product lines,

    C. Henard, M. Papadakis, G. Perrouin, J. Klein, and Y. L. Traon, “Multi-objective test generation for software product lines,” in International Software Product Line Conference, 2013, pp. 62–71

    2013

  100. [108]

    Bypassing the combinatorial explosion: Using similarity to generate and prioritize t-wise test configurations for software product lines,

    C. Henard, M. Papadakis, G. Perrouin, J. Klein, P . Heymans, and Y. L. Traon, “Bypassing the combinatorial explosion: Using similarity to generate and prioritize t-wise test configurations for software product lines,”IEEE Transactions on Software Engineering, vol. 40, no. 7, pp...

    2014

  101. [109]

    Generating combi- natorial test suite using combinatorial optimization,

    Z. Zhang, J. Yan, Y. Zhao, and J. Zhang, “Generating combi- natorial test suite using combinatorial optimization,” Journal of Systems and Software, vol. 98, no. 0, pp. 191–207, 2014

    2014

  102. [110]

    Flattening or not of the combinatorial interaction testing models?

    C. Henard, M. Papadakis, and Y. L. Traon, “Flattening or not of the combinatorial interaction testing models?” in 4th International Workshop on Combinatorial Testing, 2015, pp. 1–4

    2015

  103. [111]

    Evaluating recon- figuration impact in self-adaptive systems - an approach based on combinatorial interaction testing,

    S. Sen, S. D. Alesio, D. Marijan, and A. Sarkar, “Evaluating recon- figuration impact in self-adaptive systems - an approach based on combinatorial interaction testing,” in Euromicro Conference on Software Engineering and Advanced Applications, 2015, pp. 250–254

    2015

  104. [112]

    Optimization of combinatorial testing by incremental sat solving,

    A. Yamada, T. Kitamura, C. Artho, E.-H. Choi, Y. Oiwa, and A. Biere, “Optimization of combinatorial testing by incremental sat solving,” in International Conference on Software Testing, Verifi- cation and Validation, 2015, pp. 1–10

    2015

  105. [113]

    Practi- cal minimization of pairwise-covering test configurations using constraint programming,

    A. Hervieu, D. Marijan, A. Gotlieb, and B. Baudry, “Practi- cal minimization of pairwise-covering test configurations using constraint programming,” Information and Software Technology , vol. 71, pp. 129–146, 2016

    2016

  106. [114]

    A constraint solving problem towards unified combinatorial interaction testing,

    H. Mercan and C. Yilmaz, “A constraint solving problem towards unified combinatorial interaction testing,” in Workshop on Con- straint Solvers in Testing, Verification, and Analysis, 2016, pp. 1–7

    2016

  107. [115]

    Generating covering arrays with pseudo-boolean constraint solving and balancing heuristic,

    H. Liu, F. Ma, and J. Zhang, “Generating covering arrays with pseudo-boolean constraint solving and balancing heuristic,” in Pacific Rim International Conference on Artificial Intelligence , 2016, pp. 262–270. 16

    2016

  108. [116]

    catnap: Generating test suites of con- strained combinatorial testing with answer set programming,

    M. Banbara, K. Inoue, H. Kaneyuki, T. Okimoto, T. Schaub, T. Soh, and N. Tamura, “catnap: Generating test suites of con- strained combinatorial testing with answer set programming,” in International Conference on Logic Programming and Nonmonotonic Reasoning, 2017, pp. 265–278

    2017

  109. [117]

    A satisfiability- based approach to generation of constrained locating arrays,

    H. Jin, T. Kitamura, E.-H. Choi, and T. Tsuchiya, “A satisfiability- based approach to generation of constrained locating arrays,” in International Workshops on Combinatorial Testing, 2018, pp. 285–294

    2018

  110. [118]

    Combinatorial test case selection with markovian usage models,

    S. Vilkomir, W. T. Swain, and J. H. Poore, “Combinatorial test case selection with markovian usage models,” in International Conference on Information Technology: New Generations , 2008, pp. 3–8

    2008

  111. [119]

    An interaction- based test sequence generation approach for testing web ap- plication,

    W. Wang, S. Sampath, Y. Lei, and R. N. Kacker, “An interaction- based test sequence generation approach for testing web ap- plication,” in International Conference on High Assurance Systems Engineerng, 2008, pp. 209–218

    2008

  112. [120]

    Covering arrays avoiding forbidden edges,

    P . Danziger, E. Mendelsohn, L. Moura, and B. Stevens, “Covering arrays avoiding forbidden edges,” Theoretical Computer Science , vol. 410, no. 52, pp. 5403–5414, 2009

    2009

  113. [121]

    Covering arrays avoiding forbidden edges and edge clique covers,

    E. Maltais, “Covering arrays avoiding forbidden edges and edge clique covers,” Ph.D. dissertation, University of Ottawa, 2009

    2009

  114. [122]

    Software input space modeling with constraints among parameters,

    S. Vilkomir, W. T. Swain, and J. H. Poore, “Software input space modeling with constraints among parameters,” in International Computers, Software & Applications Conference, 2009, pp. 136–141

    2009

  115. [123]

    Hardness results for covering arrays avoiding forbidden edges and error-locating arrays,

    E. Maltais and L. Moura, “Hardness results for covering arrays avoiding forbidden edges and error-locating arrays,” Theoretical Computer Science, vol. 412, no. 46, pp. 6517–6530, 2011

    2011

  116. [124]

    Calculating prioritized interaction test sets with constraints using binary decision dia- grams,

    E. Salecker, R. Reicherdt, and S. Glesner, “Calculating prioritized interaction test sets with constraints using binary decision dia- grams,” in International Conference on Software Testing, Verification and Validation Workshops, 2011, pp. 278–285

    2011

  117. [125]

    Using binary decision di- agrams for combinatorial test design,

    I. Segall, R. Tzoref-Brill, and E. Farchi, “Using binary decision di- agrams for combinatorial test design,” in International Symposium on Software Testing and Analysis, 2011, pp. 254–264

    2011

  118. [126]

    Test sequence generation from classification trees,

    P . M. Kruse and J. Wegener, “Test sequence generation from classification trees,” in International Conference on Software Testing, Verification and Validation, 2012, pp. 539–548

    2012

  119. [127]

    Efficient algorithms for t-way test sequence generation,

    L. Yu, Y. Lei, R. N. Kacker, D. R. Kuhn, and J. Lawrence, “Efficient algorithms for t-way test sequence generation,” in International Conference on Engineering of Complex Computer Systems , 2012, pp. 220–229

    2012

  120. [128]

    Efficient combinatorial test gen- eration based on multivalued decision diagrams,

    A. Gargantini and P . Vavassori, “Efficient combinatorial test gen- eration based on multivalued decision diagrams,” in International Haifa Verification Conference, 2014, pp. 220–235

    2014

  121. [129]

    Graph methods for generating test cases with universal and existential constraints,

    S. Halle, E. L. Chance, and S. Gaboury, “Graph methods for generating test cases with universal and existential constraints,” in IFIP International Conference on Testing Software and Systems , 2015, pp. 55–70

    2015

  122. [130]

    Optimizing IPOG’s vertical growth with constraints based on hypergraph coloring,

    F. Duan, Y. Lei, L. Yu, R. N. Kacker, and D. R. Kuhn, “Optimizing IPOG’s vertical growth with constraints based on hypergraph coloring,” in International Workshop on Combinatorial Testing, 2017, pp. 181–188

    2017

  123. [131]

    Input parameter modeling for combi- nation strategies,

    M. Grindal and J. Offutt, “Input parameter modeling for combi- nation strategies,” in IASTED International Conference on Software Engineering, 2007, pp. 255–260

    2007

  124. [132]

    Deriving combinatorial test design model from uml activity diagram,

    P . Satish, K. Sheeba, and K. Rangarajan, “Deriving combinatorial test design model from uml activity diagram,” in2nd International Workshop on Combinatorial Testing, 2013, pp. 331–337

    2013

  125. [133]

    Extracting the combina- torial test parameters and values from uml sequence diagrams,

    P . Satish, A. Paul, and K. Rangarajan, “Extracting the combina- torial test parameters and values from uml sequence diagrams,” in 3rd International Workshop on Combinatorial Testing , 2014, pp. 88–97

    2014

  126. [134]

    Building combinatorial test input model from use case artefacts,

    P . Satish, M. B., M. S. Narayan, and K. Rangarajan, “Building combinatorial test input model from use case artefacts,” in Inter- national Workshop on Combinatorial Testing, 2017, pp. 220–228

    2017

  127. [135]

    Automated inference of classi- fications and dependencies for combinatorial testing,

    C. D. Nguyen and P . Tonella, “Automated inference of classi- fications and dependencies for combinatorial testing,” in Inter- national Conference on Automated Software Engineering , 2013, pp. 622–627

    2013

  128. [136]

    Priority integration for weighted combinatorial testing,

    E.-H. Choi, T. Kitamura, C. Artho, A. Yamada, and Y. Oiwa, “Priority integration for weighted combinatorial testing,” in In- ternational Computers, Software & Applications Conference, 2015, pp. 242–247

    2015

  129. [137]

    The optimal testing order in the presence of switching cost,

    H. Wu, C. Nie, and F.-C. Kuo, “The optimal testing order in the presence of switching cost,” Information and Software Technology , vol. 80, pp. 57–72, 2016

    2016

  130. [138]

    A deterministic density algorithm for pairwise interaction coverage,

    C. J. Colbourn, M. B. Cohen, and R. C. Turnban, “A deterministic density algorithm for pairwise interaction coverage,” in IASTED International Conference on Software Engineering, 2004, pp. 345–352

    2004

  131. [139]

    Theoretical and numerical constraint-handling techniques used with evolutionary algorithms: a survey of the state of the art,

    C. A. C. Coello, “Theoretical and numerical constraint-handling techniques used with evolutionary algorithms: a survey of the state of the art,” Computer methods in applied mechanics and engi- neering, vol. 191, no. 11, pp. 1245–1287, 2002

    2002

  132. [140]

    The representation of a graph by set intersections,

    P . Erdos, A. W. Goodman, and L. P ´osa, “The representation of a graph by set intersections,” Canad. J. Math , vol. 18, no. 106-112, p. 86, 1966

    1966

  133. [141]

    Using projections to debug large combinatorial models,

    E. Farchi, I. Segall, and R. Tzoref-Brill, “Using projections to debug large combinatorial models,” in 2nd International Workshop on Combinatorial Testing, 2013, pp. 311–320

    2013

  134. [142]

    Validation of models and tests for constrained combinatorial interaction testing,

    P . Arcaini, A. Gargantini, and P . Vavassori, “Validation of models and tests for constrained combinatorial interaction testing,” in3rd International Workshop on Combinatorial Testing, 2014, pp. 98–107

    2014

  135. [143]

    Lattice-based semantics for com- binatorial model evolution,

    R. Tzoref-Brill and S. Maoz, “Lattice-based semantics for com- binatorial model evolution,” in International Symposium on Auto- mated Technology for Verification and Analysis, 2015, pp. 276–292

    2015

  136. [144]

    A visual logical language for system modelling in combinatorial test design,

    M. Spichkova, A. Zamansky, and E. Farchi, “A visual logical language for system modelling in combinatorial test design,” in International Workshops on Advanced Information Systems Engineer- ing, 2016, pp. 116–121

    2016

  137. [145]

    Syntactic and semantic differencing for combinatorial models of test designs,

    R. Tzoref-Brill and S. Maoz, “Syntactic and semantic differencing for combinatorial models of test designs,” in International Confer- ence on Software Engineering, 2017, pp. 621–631

    2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.