Pith. sign in

REVIEW 3 major objections 5 minor 73 references

Navigating the growing field of research on AI for software testing -- the taxonomy for AI-augmented software testing and an ontology-driven literature survey

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that ai4st, a new ontology for AI-augmented software testing, classifies the field's full range of purposes and automation levels, as shown on 38 recent papers.

desk verdict Useful OWL infrastructure for AI4ST classification, but the coverage claim rests on a circular validation. read the letter →

arxiv 2506.14640 v1 pith:WFGKSPJF submitted 2025-06-17 cs.SE cs.AI

Ina K. Schieferdecker This is my paper · ORCID
classification cs.SEcs.AI
keywords SoftwareTestingTestAutomationArtificialIntelligenceai4stOntologyEngineeringSystematicLiteratureReviewSemanticWeb
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Software testing is the industry's main quality gate, and AI is being inserted into nearly every testing activity, but the field has lacked a shared, machine-readable way to say what a piece of research actually does. This paper tries to fill that gap with ai4st, an ontology that classifies AI-for-testing research along five dimensions: research topic, solution purpose, testing target, AI type, and automation level. The paper argues that ai4st is valid by running an ontology-driven literature review: from 1,643 recent papers it identified 38 original research papers and showed that this small set already touches every purpose and every automation level the ontology defines. If the claim holds, ai4st becomes a reusable knowledge base that lets researchers query the growing literature as a database and extend it as new terms appear.

What carries the argument

The central object is the ai4st ontology itself: a five-dimension classification scheme (research topic, solution purpose, software testing target, AI type, automation level) formalized in OWL and grounded in a four-layer stack—a lightweight foundational ontology, a software-testing concept ontology derived from standardized testing terms, the predecessor AI-for-software-engineering ontology, and ai4st itself. The machinery works in two directions. For classification, testing-concept classes are reified into individuals that serve as target classifiers, so each paper is annotated with purpose, target, AI type, and level. For discovery, the same concept maps, enriched with synonyms, replace keyword search strings in the literature review, and the review's findings feed new terms and synonyms back into the ontology. This closes a loop between ontology and literature, which is what lets the small sample double as a validity check.

What would settle it

Run the same ontology against a larger, independently assembled corpus of AI-for-software-testing papers from several venues and years, and count the papers that cannot be assigned a purpose, target, AI type, or automation level without adding a new top-level dimension. A finding that any meaningful fraction—say, more than five percent—of the corpus falls outside ai4st's categories, or that an established research stream such as AI-based test data generation or self-healing test automation is absent, would falsify the claim of full coverage.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is a demonstration of coverage: the ai4st ontology, built by layering a software-testing concept ontology of over 200 classes under four design dimensions inherited from an AI-for-software-engineering taxonomy, is rich enough that 38 selected research papers could be classified without inventing new top-level categories. The classification forced only additive changes at the leaf level—eleven new testing terms such as 'visual testing', 'assertion', 'penetration testing', 'test generation', and 'test prioritization', plus one synonym—and the author reports that the sample covers all potential purposes and levels of automation supported by AI. The paper therefore proposes that ai4st is not just a description of this sample but a reusable instrument for organizing the entire research field.

Load-bearing premise

The claim that ai4st covers all purposes and automation levels rests on a sample drawn mainly from one recent conference plus complementary library searches; if that sample misses major research streams, the coverage claim is unsupported.

Editorial extensions

If this is right

  • New research papers on AI for software testing can be filed into ai4st's five dimensions and immediately compared with the existing 38-paper corpus, turning literature reviews into database queries.
  • The ontology can double as a gap finder: queries over the classified corpus show which testing targets, AI types, and automation levels are under-researched.
  • Because the term base is linked to standardized testing vocabulary, classifications become stable and comparable across research groups, reducing synonym noise such as 'unit test' versus 'unit-level test'.
  • The ontology-driven literature review method—using concept maps with synonyms instead of search strings—can be reused in other software engineering subfields and can feed newly found terms back into the ontology, so the instrument improves as it is used.
  • If widely adopted, ai4st could become the shared index for the field, adding machine reasoning to the anchoring that standard glossaries already provide.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The coverage result is an existence proof, not a census: 38 papers touching every level shows the dimensions are expressive, but does not prove that no additional dimensions, such as human oversight, data quality, or cost, are needed for the full field.
  • The same taxonomy could classify industrial test-automation tools and AI-based quality-assurance products, not just research papers, giving practitioners a way to compare vendor claims about 'AI-powered' testing.
  • The automation-level scale invites an explicit parallel with autonomous-driving levels; a natural extension would add criteria for trust, verification of the AI's outputs, and who remains accountable when an AI-generated test is wrong.
  • Because the survey explicitly excluded testing of AI systems, a sibling ontology for testing the AI itself—covering data drift, fairness, and model robustness—would complement ai4st rather than extend it.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This manuscript presents ai4st, an ontology-driven taxonomy for AI-augmented software testing built on an existing software testing concepts ontology (stc), an AI-for-SE taxonomy (ai4se), and the gUFO foundational ontology. The paper describes a lightweight systematic literature review that screened 1643 papers from ICSE 2025 proceedings and a complementary IEEE/ACM 2020–2025 search, selected 38 original research papers, and used them to classify AI-for-ST research along five dimensions: Research Topic, Solution Purpose, ST Target, AI Type, and Automation Level. The SLR results are also used to extend stc and ai4st with new terms and synonyms. The authors report positive answers to RQ1–RQ3, claim that the selected research covers all potential purposes and automation levels supported by AI, and provide downloadable OWL artifacts, SPARQL query examples, and GitHub links for reuse.

Significance. If the coverage claims were adequately supported, ai4st would be a useful reusable classification scheme: it is formally defined in OWL, linked to ISTQB and SEVOCAB terminology, released under CC-BY-SA, and demonstrated through SPARQL queries over a classified research corpus. The paper also contributes an ontology-driven SLR methodology and makes its artifacts publicly available, which are genuine assets for the community. However, the evidence provided for the central coverage claim is circular and the SLR sample is not shown to be representative of the field. At this stage the contribution is best read as a proof-of-concept artifact and an initial classification exercise, not as a validated ontology whose coverage has been externally established.

major comments (3)
  1. [Section 4] The validation of the ai4st taxonomy is circular. The text states that "the SLR results are used to verify, improve and extend the ontologies further" and that classifying the unique research "led to an extension of the stc ontology" with new terms such as 'visual testing', 'assertion', and 'penetration testing'. The same 38 papers are then classified with the extended ontology, RQ3 is answered positively, and the paper concludes that "this small selection of unique research covers all potential purposes and levels of automation supported by AI". Because the taxonomy was extended until it fit the selected corpus, the fit provides no independent evidence of coverage; it only demonstrates internal consistency with those 38 papers. The coverage claim should either be explicitly limited to the classified corpus or supported by an external validation, such as classifying a holdout set of papers that did not inform the ontology extensions and reporting coverage statistics.
  2. [Section 4, SLR protocol] The SLR protocol is not reported with enough detail to establish representativeness or reproducibility. The paper gives inclusion and exclusion criteria and a two-step selection process, but no exact search expressions, no per-source query strings, no date boundaries for the complementary IEEE and ACM search, and no decision log explaining how 1643 papers were reduced to 949, then 656, then 38. The statement that "forward snowballing was in this case unnecessary, as ICSE 2025 represented the most recent research publications at that time" does not justify omitting forward snowballing, and the reliance on one conference plus an unspecified complementary search is a weak basis for the claim that the selected papers cover all potential purposes and levels of automation. The protocol needs a reproducible search string appendix and a justification of the sample's representativeness.
  3. [Section 4, extended stc ontology list] The reported counts in the ontology-extension results are internally inconsistent. The text says that the SLR led to "eight new terms not in the ISTQB Glossary", but the bullet list enumerates two test techniques, five test activities, one non-functional testing, and one basic concept, which totals nine entries; moreover, 'penetration testing' is listed both among the three new ISTQB-defined terms and again among the non-ISTQB terms as a non-functional testing type. These inconsistencies make it impossible to verify which terms were actually added and need to be corrected before the results can be checked.
minor comments (5)
  1. [Figure 1 caption] The caption contains a typo: "sumbolic" should be "symbolic".
  2. [Section 2] There is a duplicated phrase "rather than than focusing on the established body of knowledge" that should be corrected.
  3. [Section 5] The sentence "and and carefully revise new term and synonym candidates" contains a duplicated "and".
  4. [Section 4] The acronyms SLR and SRL are used inconsistently; for example, "the SRL focused on ST rather than AI" should say "SLR".
  5. [Table 1] Reference [12] appears twice in Table 1; the second entry, "Using Large Language Models to Generate Concise and Understandable Test Case Summaries", corresponds to bibliography entry [13] and should be relabeled.

Circularity Check

1 steps flagged · score 6.0 of 10

Coverage claim is partially circular: stc/ai4st target terms are harvested from the same 38 papers that are then used to demonstrate that the taxonomy covers all purposes and levels.

  1. fitted input called prediction [Section 4, 'Synthesis process' and the response to RQ1/RQ2/RQ3]
    "Additionally, the SLR results are used to verify, improve and extend the ontologies further. ... Classifying the unique research led to an extension of the stc ontology with: ... 'visual testing' [19, 45, 43], 'assertion' [66, 44], and 'penetration testing' [28, 10] ... In response to RQ1 and RQ2, the software testing targets in the unique research papers were successfully classified by combining the new terms and synonyms with the terms in stc, and hence in ai4st. Alongside this, RQ3 can also be answered positively."

    The ontology is modified using the same data that are then used to test it. The protocol states that the SLR results are fed back into the ontology ('the SLR results are used to verify, improve and extend the ontologies further'), and the paper then reports that classifying the unique research 'led to an extension of the stc ontology' with target terms such as 'visual testing', 'assertion', and 'penetration testing'. Those same papers are then classified with the extended vocabulary, and this success is offered as the positive answer to RQ3 and as evidence that the selection 'covers all potential purposes and levels'.

full rationale

The core dimensions of ai4st (Solution Purpose, ST Target, AI Type, Automation Level) predate the SLR and derive from the author's earlier ai4se taxonomy, so the ontology has genuine independent structure. However, the validation of the ontology's coverage is weakened by a train-on-test loop: the SLR protocol explicitly uses the selected papers to improve and extend the stc/ai4st ontologies, new target terms are added from the 38 selected papers, and then the same 38 papers are classified with the extended ontology to answer RQ3 positively and to claim that the selection 'covers all potential purposes and levels of automation supported by AI'. For the target dimension, the positive result is guaranteed by construction, since any target appearing in the corpus could be added to the ontology and then counted as classified. The purpose and level dimensions were not extended from the corpus and provide partial independent evidence. Separately, the paper's sample justification ('Forward snowballing was in this case unnecessary, as ICSE 2025 represented the most recent research publications at that time') is a sampling limitation that bears on the correctness of the universal coverage claim, but it is not itself a circularity. The OWL/GitHub artifacts are real evidence of the ontology's formality and availability, but they do not independently establish completeness of coverage. Overall, the central coverage claim is partially circular because the evaluation set was used to fit the classifier, so the score is 6 rather than higher.

Assumptions & free parameters 0 free parameters · 4 assumptions · 1 invented entities

The paper introduces no fitted numerical parameters. Its contributions rest on domain assumptions about terminology sources, the chosen ontology stack, a representative SLR sample, and the automation-level scale. The ai4st ontology itself is the paper's main invented artifact, whose independent validity is not yet established.

assumptions (4)
  • domain assumption The ISTQB Glossary is a comprehensive and authoritative source for software testing terminology.
    The stc ontology and ai4st taxonomy are built primarily on ISTQB terms (Section 3).
  • domain assumption The four-layer ontology architecture (gUFO, stc, ai4se, ai4st) is appropriate.
    Section 3 states this architecture without justification.
  • domain assumption The selected SLR sample (ICSE 2025, backward snowballing, IEEE/ACM 2020-2025) is representative of AI4ST research.
    Section 4 selection process; forward snowballing omitted.
  • domain assumption The five-level automation scale is a valid way to characterize AI4ST approaches.
    Section 3 'Level' dimension; no empirical grounding.
invented entities (1)
  • ai4st ontology
    purpose: Classify and query research contributions on AI for software testing.
    The ontology is introduced in this paper; its validity is evaluated only via the author's own SLR, with no external application yet.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Navigating the growing field of research on AI for software testing -- the taxonomy for AI-augmented software testing and an ontology-driven literature survey." pith.science (2026). https://pith.science/paper/WFGKSPJF

@misc{pith2026250614640,
  author       = {Pith},
  title        = {Pith review of: Navigating the growing field of research on AI for software testing -- the taxonomy for AI-augmented software testing and an ontology-driven literature survey},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WFGKSPJF}},
  note         = {Machine review of arXiv:2506.14640}
}
read the original abstract

In industry, software testing is the primary method to verify and validate the functionality, performance, security, usability, and so on, of software-based systems. Test automation has gained increasing attention in industry over the last decade, following decades of intense research into test automation and model-based testing. However, designing, developing, maintaining and evolving test automation is a considerable effort. Meanwhile, AI's breakthroughs in many engineering fields are opening up new perspectives for software testing, for both manual and automated testing. This paper reviews recent research on AI augmentation in software test automation, from no automation to full automation. It also discusses new forms of testing made possible by AI. Based on this, the newly developed taxonomy, ai4st, is presented and used to classify recent research and identify open research questions.

Figures

Figures reproduced from arXiv: 2506.14640 by the authors.

Figure 1
Figure 1. Research Paper Classification with ai4st [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Overview on the ontology-driven systematic literature review (SLR) combined with the SLR [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

73 extracted references · 47 canonical work pages

  1. [12]

    In: Proceedings of the 3rd ACM/IEEE International Conference on Automation of Software Test

    De Santiago Júnior, V.A.: A method and experiment to evaluate deep neural networks as test oracles for scientific software. In: Proceedings of the 3rd ACM/IEEE International Conference on Automation of Software Test. pp. 40–51. ACM, Pittsburgh Pennsylvania (May 2022). https://doi.org/10.1145/ 3524481.3527232, https://dl.acm.org/doi/10.1145/3524481.3527232

    arXiv 2022

  2. [1]

    In: 2023 IEEE International Conference On Artificial Intelligence Testing (AITest)

    Abdelkarim, M., ElAdawi, R.: TCP-Net++: Test Case Prioritization Using End-to-End Deep Neural Networks - Deployment Analysis and Enhancements. In: 2023 IEEE International Conference On Artificial Intelligence Testing (AITest). pp. 99–106. IEEE, Athens, Greece (Jul 2023). https://doi.org/ 10.1109/AITest58265.2023.00024, https://ieeexplore.ieee.org/document...

    work page arXiv 2023

  3. [2]

    Ontologies for software engineering and software technology pp

    Abran, A., Cuadrado, J.J., García-Barriocanal, E., Mendes, O., Sánchez-Alonso, S., Sicilia, M.A.: Engineering the ontology for the SWEBOK: Issues and techniques. Ontologies for software engineering and software technology pp. 103–121 (2006)

    work page 2006

  4. [3]

    URL http://purl

    Almeida, J.P.A., Guizzardi, G., Falbo, R., Sales, T.P.: gufo: a lightweight implementation of the unified foundational ontology (ufo). URL http://purl. org/nemo/doc/gufo (2019)

    work page 2019

  5. [4]

    In: Seke

    Barbosa, E.F., Nakagawa, E.Y., Maldonado, J.C.: Towards the establishment of an ontology of software testing. In: Seke. vol. 6, pp. 522–525 (2006)

    work page 2006

  6. [5]

    In: Knowledge Engineering and Knowledge Management: 20th International Conference, EKAW 2016, Bologna, Italy, November 19-23, 2016, Proceedings 20

    Borges Ruy, F., de Almeida Falbo, R., Perini Barcellos, M., Dornelas Costa, S., Guizzardi, G.: SEON: A software engineering ontology network. In: Knowledge Engineering and Knowledge Management: 20th International Conference, EKAW 2016, Bologna, Italy, November 19-23, 2016, Proceedings 20. pp. 527–542. Springer (2016)

    work page 2016

  7. [6]

    In: 2023 Innovations in Intelligent Systems and Applications Conference (ASYU)

    Caglar, O., Taskin, F., Baglum, C., Asik, S., Yayan, U.: Development of Cloud and Artificial Intel- ligence based Software Testing Platform (ChArIoT). In: 2023 Innovations in Intelligent Systems and Applications Conference (ASYU). pp. 1–6. IEEE, Sivas, Turkiye (Oct 2023). https://doi.org/10.1109/ ASYU58738.2023.10296551, https://ieeexplore.ieee.org/docume...

    arXiv 2023

  8. [7]

    Springer Science & Business Media (2006)

    Calero, C., Ruiz, F., Piattini, M.: Ontologies for software engineering and software technology. Springer Science & Business Media (2006)

    work page 2006

Show all 73 references
  1. [8]

    In: Companion Proceedings of the 30th Interna- tional Conference on Intelligent User Interfaces

    Calvano, M., Curci, A., Lanzilotti, R., Piccinno, A., Ragone, A.: Leveraging Large Language Mod- els for Usability Testing: a Preliminary Study. In: Companion Proceedings of the 30th Interna- tional Conference on Intelligent User Interfaces. pp. 78–81. ACM, Cagliari Italy (Mar...

    2025

  2. [9]

    CC: CC BY-SA 4.0 license, attribution-sharealike 4.0 international, legal code (2025), https:// creativecommons.org/licenses/by-sa/4.0/legalcode.en

    2025

  3. [10]

    In: 2022 IEEE Aerospace Conference (AERO)

    Confido, A., Ntagiou, E.V., Wallum, M.: Reinforcing Penetration Testing Using AI. In: 2022 IEEE Aerospace Conference (AERO). pp. 1–15. IEEE, Big Sky, MT, USA (Mar 2022). https://doi.org/10. 1109/AERO53065.2022.9843459, https://ieeexplore.ieee.org/document/9843459/

    2022

  4. [13]

    In: Early Research Achievements (ERA)

    Djajadi, N., Deljouyi, A., Zaidman, A.: Using Large Language Models to Generate Concise and Un- derstandable Test Case Summaries. In: Early Research Achievements (ERA). https://doi.org/10.1109/ ICPC66645.2025.00040, https://azaidman.github.io/publications/djajadiICPC2025.pdf

    2025

  5. [14]

    Software Quality Journal25, 1269–1305 (2017)

    Engström, E., Petersen, K., Ali, N.B., Bjarnason, E.: SERP-test: a taxonomy for supporting industry– academia communication. Software Quality Journal25, 1269–1305 (2017)

    2017

  6. [15]

    International Journal on Software Tools for Technology Transfer16, 559–568 (2014)

    Felderer, M., Schieferdecker, I.: A taxonomy of risk-based testing. International Journal on Software Tools for Technology Transfer16, 559–568 (2014)

    2014

  7. [16]

    Software testing, verification and reliability26(2), 119–148 (2016) 12 I

    Felderer, M., Zech, P., Breu, R., Büchler, M., Pretschner, A.: Model-based security testing: a taxonomy and systematic classification. Software testing, verification and reliability26(2), 119–148 (2016) 12 I. Schieferdecker

    2016

  8. [17]

    https://doi.org/10.48550/arXiv.2504.07244, http:// arxiv.org/abs/2504.07244, arXiv:2504.07244 [cs]

    Ferreira, M., Viegas, L., Faria, J.P., Lima, B.: Acceptance Test Generation with Large Language Models: An Industrial Case Study (Apr 2025). https://doi.org/10.48550/arXiv.2504.07244, http:// arxiv.org/abs/2504.07244, arXiv:2504.07244 [cs]

    2025 doi

  9. [18]

    In: 2025 IEEE Conference on Software Testing, Verification and Validation (ICST)

    Franzosi, D.B., Alégroth, E., Isaac, M.: LLM-Based Labelling of Recorded Automated GUI-Based Test Cases. In: 2025 IEEE Conference on Software Testing, Verification and Validation (ICST). pp. 453–463. IEEE, Napoli, Italy (Mar 2025). https://doi.org/10.1109/ICST62969.2025.109889...

    2025

  10. [19]

    In: 2023 5th Novel Intelligent and Leading Emerging Sciences Conference (NILES)

    Gamal, A., Emad, R., Mohamed, T., Mohamed, O., Hamdy, A., Ali, S.: Owl Eye: An AI-Driven Visual Testing Tool. In: 2023 5th Novel Intelligent and Leading Emerging Sciences Conference (NILES). pp. 312–315. IEEE, Giza, Egypt (Oct 2023). https://doi.org/10.1109/NILES59815.2023.102...

    2023

  11. [20]

    Gao, H., Yang, Y., Sun, M., Wu, J., Zhou, Y., Xu, B.: ClozeMaster: Fuzzing Rust Compiler by Har- nessing LLMs for Infilling Masked Real Programs. pp. 712–712. IEEE Computer Society (Mar 2025). https://doi.org/10.1109/ICSE55347.2025.00175, https://www.computer.org/csdl/proceedi...

    2025

  12. [21]

    In: 2022 IEEE International Conference On Artificial Intelligence Testing (AITest)

    Gao, J., Li, S., Tao, C., He, Y., Anumalasetty, A.P., Joseph, E.W., Sripathi, A.H.K., Nayani, H.: An Approach to GUI Test Scenario Generation Using Machine Learning. In: 2022 IEEE International Conference On Artificial Intelligence Testing (AITest). pp. 79–86. IEEE, Newark, CA...

    2022

  13. [22]

    In: 2023 International Conference on Advances in Computation, Communication and Information Technology (ICAICCIT)

    Garg, A., Sharma, D.: Generative AI for Software Test Modelling with a focus on ERP Software. In: 2023 International Conference on Advances in Computation, Communication and Information Technology (ICAICCIT). pp. 187–193. IEEE, Faridabad, India (Nov 2023). https://doi.org/10.1...

    2023

  14. [23]

    In: 2024 5th IEEE Global Conference for Advancement in Technology (GCAT)

    Garlapati, A., Satya Sai Muni Parmesh, M.N.V., Savitha, S, J.: AI-Powered Multi-Agent Frame- work for Automated Unit Test Case Generation: Enhancing Software Quality through LLM’s. In: 2024 5th IEEE Global Conference for Advancement in Technology (GCAT). pp. 1–5. IEEE, Ban- ga...

    2024

  15. [24]

    In: Proceedings of the 1st ACM SIGSOFT International Workshop on Languages and Tools for Next-Generation Testing

    Ghimis, B., Paduraru, M., Stefanescu, A.: RIVER 2.0: an open-source testing framework using AI techniques. In: Proceedings of the 1st ACM SIGSOFT International Workshop on Languages and Tools for Next-Generation Testing. pp. 13–18. ACM, Virtual USA (Nov 2020). https://doi.org/...

    2020

  16. [25]

    In: Conceptual Modeling: 37th International Conference, ER 2018, Xi’an, China, October 22–25, 2018, Proceedings 37

    Guizzardi, G., Fonseca, C.M., Benevides, A.B., Almeida, J.P.A., Porello, D., Sales, T.P.: Endurant types in ontology-driven conceptual modeling: Towards ontouml 2.0. In: Conceptual Modeling: 37th International Conference, ER 2018, Xi’an, China, October 22–25, 2018, Proceedings...

    2018

  17. [26]

    In: 2025 IEEE Interna- tional Conference on Software Testing, Verification and Validation Workshops (ICSTW)

    Hagar, J., Wissink, T.: AIs Understanding of Software Test Architecture. In: 2025 IEEE Interna- tional Conference on Software Testing, Verification and Validation Workshops (ICSTW). pp. 194–

    2025

  18. [27]

    In: 2024 IEEE Frontiers in Education Conference (FIE)

    Haldar, S., Pierce, M., Capretz, L.F.: WIP: Assessing the Effectiveness of ChatGPT in Preparatory Testing Activities. In: 2024 IEEE Frontiers in Education Conference (FIE). pp. 1–5. IEEE, Washington, DC, USA (Oct 2024). https://doi.org/10.1109/FIE61694.2024.10893214, https://i...

    2024

  19. [28]

    In: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

    Happe, A., Cito, J.: Getting pwn’d by AI: Penetration Testing with Large Language Models. In: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. pp. 2082–2086. ACM, San Francisco CA USA (Nov 2023...

    2023

  20. [29]

    In: 2024 International Telecommunications Conference The ai4st taxonomy and its use 13 (ITC-Egypt)

    Helmy, M., Sobhy, O., ElHusseiny, F.: AI-Driven Testing: Unleashing Autonomous Systems for Su- perior Software Quality Using Generative AI. In: 2024 International Telecommunications Conference The ai4st taxonomy and its use 13 (ITC-Egypt). pp. 1–6. IEEE, Cairo, Egypt (Jul 2024...

    2024

  21. [30]

    https://doi.org/10.48550/arXiv.2405.03786, http://arxiv.org/abs/2405.03786, arXiv:2405.03786 [cs]

    Hossain, S.B., Dwyer, M.: TOGLL: Correct and Strong Test Oracle Generation with LLMs (Dec 2024). https://doi.org/10.48550/arXiv.2405.03786, http://arxiv.org/abs/2405.03786, arXiv:2405.03786 [cs]

    2024 doi arXiv

  22. [31]

    ISO, IEC, IEEE: ISO/IEC/IEEE 24765 international standard, second edition: Systems and software engineering – vocabulary (2017), https://pascal.computer.org

    2017

  23. [32]

    istqb.org

    ISTQB: Glossary of the international software testing qualifications board (2025), https://glossary. istqb.org

    2025

  24. [33]

    In: 2025 Fifth International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT)

    Kapoor, S.: AI-Assisted Test Script Generation for GUI Applications. In: 2025 Fifth International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT). pp. 1–5. IEEE, Bhilai, India (Jan 2025). https://doi.org/10.1109/ICAECT63952....

    2025

  25. [34]

    In: 2020 10th International Con- ference on Cloud Computing, Data Science & Engineering (Confluence)

    Kaur, A.: An Approach To Extract Optimal Test Cases Using AI. In: 2020 10th International Con- ference on Cloud Computing, Data Science & Engineering (Confluence). pp. 649–654. IEEE, Noida, India (Jan 2020). https://doi.org/10.1109/Confluence47617.2020.9058244, https://ieeexpl...

    2020

  26. [35]

    In: Proceedings of the 5th ACM/IEEE International Conference on Automation of Software Test (AST 2024)

    Leu, B., Volken, J., Kropp, M., Dogru, N., Anslow, C., Biddle, R.: Reducing Workload in Using AI- based API REST Test Generation. In: Proceedings of the 5th ACM/IEEE International Conference on Automation of Software Test (AST 2024). pp. 147–148. ACM, Lisbon Portugal (Apr 2024...

    2024

  27. [36]

    In: Proceedings of the XXIII Brazilian Symposium on Software Quality

    Maia, C.J.D.L., Aguiar, Y.P.C.: AI-Driven Acceptance Testing: first insights exploring the educational potential for test analysts. In: Proceedings of the XXIII Brazilian Symposium on Software Quality. pp. 665–672. ACM, Salvador Bahia Brazil (Nov 2024). https://doi.org/10.1145...

    2024

  28. [37]

    In: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings

    Martin-Lopez, A.: AI-driven web API testing. In: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings. pp. 202–205. ACM, Seoul South Ko- rea (Jun 2020). https://doi.org/10.1145/3377812.3381388, https://dl.acm.org/doi/10.1145/...

    2020

  29. [38]

    W3C recommen- dation 10(10), 2004 (2004)

    McGuinness, D.L., Van Harmelen, F., et al.: Owl web ontology language overview. W3C recommen- dation 10(10), 2004 (2004)

    2004

  30. [39]

    In: 2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)

    Mohacsi, S., Felderer, M.: AI-Based Enhancement of Test Models in an Industrial Model-Based Testing Tool. In: 2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). pp. 636–638. IEEE, Honolulu, HI, USA (Mar 2021). https://doi.org/10.1109/...

    2021

  31. [40]

    In: 2024 International Conference on Circuit, Systems and Communication (ICCSC)

    Naimi, L., Bouziane, E.M., Manaouch, M., Jakimi, A.: A new approach for automatic test case gener- ation from use case diagram using LLMs and prompt engineering. In: 2024 International Conference on Circuit, Systems and Communication (ICCSC). pp. 1–5. IEEE, Fes, Morocco (Jun 2...

    2024

  32. [41]

    In: 2024 36th International Conference on Software Engineering Education and Train- ing (CSEE&T)

    Olmez, M.M., Gehringer, E.: Automation of Test Skeletons Within Test-Driven Development Projects. In: 2024 36th International Conference on Software Engineering Education and Train- ing (CSEE&T). pp. 1–10. IEEE, Würzburg, Germany (Jul 2024). https://doi.org/10.1109/ CSEET6...

    2024

  33. [42]

    Peixoto, M., Baía, D., Nascimento, N., Alencar, P., Fonseca, B., Ribeiro, M.: On the Effectiveness of LLMsforManualTestVerifications.In:2025IEEE/ACMInternationalWorkshoponDeepLearningfor Testing and Testing for Deep Learning (DeepTest). pp. 45–52. IEEE, Ottawa, ON, Canada (May...

    2025

  34. [43]

    In: 2021 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)

    Prasetya, I.S.W.B., Shirzadehhajimahmood, S., Ansari, S.G., Fernandes, P., Prada, R.: An Agent- based Architecture for AI-Enhanced Automated Testing for XR Systems, a Short Paper. In: 2021 IEEE International Conference on Software Testing, Verification and Validation Workshops...

    2021

  35. [44]

    https://doi.org/10.1109/AST66626.2025.00008, http://arxiv.org/abs/2502

    Primbs, S., Fein, B., Fraser, G.: AsserT5: Test Assertion Generation Using a Fine-Tuned Code Lan- guage Model (Feb 2025). https://doi.org/10.1109/AST66626.2025.00008, http://arxiv.org/abs/2502. 02708, arXiv:2502.02708 [cs]

    2025

  36. [45]

    In: 2023 IEEE 15th International Con- ference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Envi- ronment, and Management (HNICEM)

    Ragel, R.K.C., Balahadia, F.F.: Visual Test Framework: Enhancing Software Test Automation with Vi- sual Artificial Intelligence and Behavioral Driven Development. In: 2023 IEEE 15th International Con- ference on Humanoid, Nanotechnology, Information Technology, Communication a...

    2023

  37. [46]

    In: 2011 IEEE Third International Conference on Cloud Computing Technology and Science

    Robinson, P., Ragusa, C.: Taxonomy and requirements rationalization for infrastructure in cloud-based software testing. In: 2011 IEEE Third International Conference on Cloud Computing Technology and Science. pp. 454–461. IEEE (2011)

    2011

  38. [47]

    Communications of the ACM55(11), 12–12 (2012)

    Rous, B.: Major update to acm’s computing classification system. Communications of the ACM55(11), 12–12 (2012)

    2012

  39. [48]

    Data Intelligence 2(3), 379–416 (2020)

    Salatino, A.A., Thanapalasingam, T., Mannocci, A., Birukou, A., Osborne, F., Motta, E.: The com- puter science ontology: A comprehensive automatically-generated taxonomy of research areas. Data Intelligence 2(3), 379–416 (2020)

    2020

  40. [49]

    arXiv preprint arXiv:2409.18048 (2024)

    Schieferdecker, I.K.: Next-gen software engineering: Ai-assisted big models. arXiv preprint arXiv:2409.18048 (2024)

    2024 arXiv

  41. [50]

    Schieferdecker, I.K.: ai4se - the AI for software engineering ontology (2025), https://github.com/ schieferdecker/ai4se

    2025

  42. [51]

    Schieferdecker, I.K.: ai4st - the AI for software testing ontology (2025), https://github.com/ schieferdecker/ai4st

    2025

  43. [52]

    com/schieferdecker/ai4stpaper

    Schieferdecker, I.K.: Annex for ’a taxonomy for ai-augmented software testing’ (2025), https://github. com/schieferdecker/ai4stpaper

    2025

  44. [53]

    Schieferdecker, I.K.: stc - the software testing concept ontology (2025), https://github.com/ schieferdecker/stc

    2025

  45. [54]

    In: Proceedings of the 12th International Workshop on Automating TEST Case Design, Selection, and Evaluation

    Shirzadehhajimahmood, S., Prasetya, I.S.W.B., Dignum, F., Dastani, M., Keller, G.: Using an agent- based approach for robust automated testing of computer games. In: Proceedings of the 12th International Workshop on Automating TEST Case Design, Selection, and Evaluation. pp. 1...

    2021

  46. [55]

    In: 29th Annual International Computer Software and Application Conference (COMPSAC), Edinburgh, UK

    Sicilia, M., Cuadrado, J.J., García, E., Rodríguez, D., Hilera, J.R.: The evaluation of ontological representation of the SWEBOK as a revision tool. In: 29th Annual International Computer Software and Application Conference (COMPSAC), Edinburgh, UK. pp. 26–28 (2005)

    2005

  47. [56]

    Ap- plied Ontology 12(1), 59–90 (2017)

    Souza, É.F.d., Falbo, R.d.A., Vijaykumar, N.L.: ROoST: Reference ontology on software testing. Ap- plied Ontology 12(1), 59–90 (2017)

    2017

  48. [57]

    strengthening the bond between development and test

    Spillner, A., Bremenn, H.: The w-model. strengthening the bond between development and test. In: Int. Conf. on Software Testing, Analysis and Review. pp. 15–17 (2002)

    2002

  49. [58]

    Spillner, A., Linz, T.: Software testing foundations: A study guide for the certified tester exam- foundation level-ISTQB® compliant. dpunkt. verlag (2021)

    2021

  50. [59]

    In: Central European conference on information and intelligent systems

    Stapic, Z., López, E.G., Cabot, A.G., de Marcos Ortega, L., Strahonja, V.: Performing systematic lit- erature review in software engineering. In: Central European conference on information and intelligent systems. p. 441. Faculty of Organization and Informatics Varazdin (2012)

    2012

  51. [60]

    In: 2021 IEEE International Conference on Artificial Intelligence Testing (AITest)

    Strandberg, P.E., Frasheri, M., Enoiu, E.P.: Ethical AI-Powered Regression Test Selection. In: 2021 IEEE International Conference on Artificial Intelligence Testing (AITest). pp. 83–84. IEEE, Oxford, United Kingdom (Aug 2021). https://doi.org/10.1109/AITEST52744.2021.00025, ht...

    2021

  52. [61]

    In: CIbSE

    Tebes, G., Olsina, L., Peppino, D., Becker, P.: TestTDO: A top-domain software testing ontology. In: CIbSE. pp. 364–377 (2020)

    2020

  53. [62]

    Information and Software Technology123, 106298 (2020) The ai4st taxonomy and its use 15

    Tebes, G., Peppino, D., Becker, P., Matturro, G., Solari, M., Olsina, L.: Analyzing and documenting the systematic review results of software testing ontologies. Information and Software Technology123, 106298 (2020) The ai4st taxonomy and its use 15

    2020

  54. [63]

    Information and Software Technology 85, 43–59 (May 2017)

    Usman, M., Britto, R., Börstler, J., Mendes, E.: Taxonomies in software engineering: A Systematic mapping study and a revised taxonomy development method. Information and Software Technology 85, 43–59 (May 2017). https://doi.org/10.1016/j.infsof.2017.01.006, https://www.scienc...

    2017 doi

  55. [64]

    Software testing, verification and reliability22(5), 297–312 (2012)

    Utting, M., Pretschner, A., Legeard, B.: A taxonomy of model-based testing approaches. Software testing, verification and reliability22(5), 297–312 (2012)

    2012

  56. [65]

    In: 2015 10th Iberian Conference on Information Systems and Technologies (CISTI)

    Villalón, J.C.M., Agustin, G.C., Gilabert, T.S.F., de Jesús Jiménez Puello, J.: A taxonomy for software testing projects. In: 2015 10th Iberian Conference on Information Systems and Technologies (CISTI). pp. 1–6 (2015). https://doi.org/10.1109/CISTI.2015.7170545

    2015

  57. [66]

    In: Proceedings of the 2024 IEEE/ACM First International Conference on AI Foundation Models and Software Engineering

    Wang, H., Xu, T., Wang, B.: Deep Multiple Assertions Generation. In: Proceedings of the 2024 IEEE/ACM First International Conference on AI Foundation Models and Software Engineering. pp. 1–11. ACM, Lisbon Portugal (Apr 2024). https://doi.org/10.1145/3650105.3652293, https://dl...

    2024

  58. [67]

    Washizaki, H.e.: Guide to the Software Engineering Body of Knowledge (SWEBOK Guide), Version 4.0 (2024), http://www.swebok.org

    2024

  59. [68]

    Wille, C., Abran, A., Desharnais, J.M., Dumke, R.: The quality concepts and sub concepts in SWE- BOK:Anontologychallenge.In:InternationalWorkshoponSoftwareMeasurement(IWSM),Montreal. vol. 18 (2003)

    2003

  60. [69]

    In: 2024 IEEE 15th International Conference on Software Engineering and Service Science (ICSESS)

    Yao, Y., Wang, J., Hu, Y., Wang, L., Zhou, Y., Chen, J., Gai, X., Wang, Z., Liu, W.: BugBlitz-AI: An Intelligent QA Assistant. In: 2024 IEEE 15th International Conference on Software Engineering and Service Science (ICSESS). pp. 57–63. IEEE, Changsha, China (Sep 2024). https:/...

    2024

  61. [70]

    In: 2024 5th International Conference on Artificial Intelligence and Computer Engineering (ICAICE)

    Zhang, Y.: New Approaches to Automated Software Testing Based on Artificial Intelligence. In: 2024 5th International Conference on Artificial Intelligence and Computer Engineering (ICAICE). pp. 806–

    2024

  62. [71]

    In: Software Evolution with UML and XML, pp

    Zhu, H., Huo, Q.: Developing software testing ontology in UML for a software growth environment of web-based applications. In: Software Evolution with UML and XML, pp. 263–295. IGI Global (2005)

    2005

  63. [72]

    In: 2023 38th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW)

    Zimmermann, D., Koziolek, A.: GUI-Based Software Testing: An Automated Approach Using GPT-4 and Selenium WebDriver. In: 2023 38th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW). pp. 171–174. IEEE, Luxembourg, Luxembourg (Sep 2023). https: ...

    2023

  64. [199]

    https://doi.org/10.1109/ICSTW64639.2025.10962517, https: //ieeexplore.ieee.org/document/10962517/

    IEEE, Naples, Italy (Mar 2025). https://doi.org/10.1109/ICSTW64639.2025.10962517, https: //ieeexplore.ieee.org/document/10962517/

    2025

  65. [810]

    https://doi.org/10.1109/ICAICE63571.2024.10863866, https: //ieeexplore.ieee.org/document/10863866/

    IEEE, Wuhu, China (Nov 2024). https://doi.org/10.1109/ICAICE63571.2024.10863866, https: //ieeexplore.ieee.org/document/10863866/

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.