Pith. sign in

REVIEW 4 major objections 4 minor 66 references

A new pipeline gets an LLM to draw a control-flow graph from a natural-language use case first, then creates one test case per graph path, which yields more complete and less redundant test suites than asking the LLM to write tests directly

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 18:08 UTC pith:CXOCPPYY

load-bearing objection Useful incremental pipeline — LLM-generated CFGs as an intermediate representation for test generation — but the evaluation's grounding in a subjective manual oracle is the real soft spot, not the method itself. the 4 major comments →

arxiv 2512.06401 v2 pith:CXOCPPYY submitted 2025-12-06 cs.SE

LLMCFG-TGen: Using LLM-Generated Control Flow Graphs to Automatically Create Test Cases from Use Cases

Zhenzhen Yang , Chenhui Cui , Tao Li , Rubing Huang , Nan Niu , Dave Towey , Shikai Guo This is my paper
classification cs.SE
keywords test case generationcontrol flow graphlarge language modeluse caserequirements-based testingpath coveragenatural language requirementsJSON representation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper claims that asking a large language model to first convert a natural-language use case into a structured control-flow graph (CFG), then generating one test case per path in that graph, produces more complete and less redundant test suites than asking the model to write test cases directly. On 42 use cases across four domains, the LLM-generated CFGs closely matched manually built ones (node F1 around 0.9, structural similarity 0.93), and the resulting test suites aligned with the ground-truth path counts far more often than the baselines. Practitioners rated the resulting test cases higher on relevance, completeness, correctness, and clarity. If right, the approach removes the need for manual modeling while keeping the rigor of path-based test design.

Core claim

The central claim is that an LLM, guided by a prompt that includes an explicit CFG-construction algorithm, can transform unstructured use-case text into a well-formed JSON control-flow graph, and that deriving test cases from the enumerated paths of that graph yields full path coverage with minimal redundancy. The evaluation shows the generated graphs are structurally close to manually constructed ones, and the test-case counts track the ground-truth path counts (discrepancy rate 2.38% versus 57% for direct LLM prompting and 33% for a structured two-step baseline). The paper therefore argues that coupling LLM semantic reasoning with an explicit graph model bridges the gap between NL requirem

What carries the argument

The central object is the LLM-generated JSON control-flow graph, whose nodes are use-case steps and whose edges carry branch conditions. A purpose-built prompt supplies a small algorithm for building the graph, and a validator rejects graphs with isolated nodes, unreachable non-root nodes, or dangling edge references, regenerating until structurally sound. Depth-first search then enumerates all root-to-leaf paths (with cycles pruned), and each path, with conditions kept as steps, is handed to the LLM to produce a structured test case. The graph is what turns free-form NL into a finite, ordered set of test scenarios.

Load-bearing premise

The load-bearing premise is that a structurally connected CFG that passes the three validity checks also represents all real conditional branches of the use case; if the LLM merges or omits a branch while still producing a connected graph, the pipeline emits a complete-looking but incomplete test suite.

What would settle it

Take a use case with an explicit if/then/else with both outcomes; run the pipeline; if the generated test suite contains only the then-branch (or only one path) while the graph still passes validation, the coverage claim fails. More generally, compare the set of branch-condition strings in the use-case text with the set of conditions appearing on CFG edges.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Test suites generated this way cover every path in the LLM-built CFG, in contrast to direct LLM prompting that tends to over-generate redundant cases, or to structured pipelines that can omit branches.
  • The approach works across diverse use-case formats and domains without requiring template standardization or manual model construction.
  • Path-based generation makes each test case traceable to a specific execution path, which improves clarity and ease of review for practitioners.
  • The pipeline is model-agnostic: different LLMs can produce valid CFGs, although performance varies with the model and prompt tuning.
  • Because the CFG itself is the artifact, the same pipeline could in principle be adapted to other NL requirement forms such as user stories.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The completeness guarantee is relative to the LLM-built graph, not to the requirements: a branch the LLM silently merges or drops while keeping the graph connected will be absent from the test suite, and the structural validator will not catch it.
  • Path-count agreement with a manual CFG is a proxy for behavioral fidelity; counting paths cannot distinguish a missing branch from a paraphrased one.
  • A natural next step is to check whether the generated graph's paths cover each conditional outcome stated in the use-case text itself, rather than only comparing counts to a second manual graph.
  • The per-path generation strategy could be extended to generate executable scripts or prioritized execution order, as the authors themselves note as future work.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes LLMCFG-TGen, a three-step pipeline that (1) uses an LLM (GPT-4o in the main evaluation) to convert a natural-language use-case description into a JSON control-flow graph (CFG), (2) enumerates all paths in that CFG with DFS, and (3) uses a second LLM prompt to turn each path into a structured abstract test case. The approach is template-agnostic and requires no manual model construction. The evaluation uses 42 use cases from four datasets, comparing LLM-generated CFGs with manually constructed ground-truth CFGs via node/edge precision, recall, F1, normalized graph edit distance, and path-count discrepancy; it also compares generated test cases with two baselines (direct LLM prompting and AGORA) using path-count consistency and a four-expert practitioner study. The paper reports high structural similarity to the manual CFGs, a very low path-count discrepancy rate (2.38%) for LLMCFG-TGen, and higher practitioner ratings than AGORA on relevance, completeness, correctness, and clarity.

Significance. If the central claim is accepted, LLMCFG-TGen would be a practically attractive contribution: it replaces manual behavioral modeling with an LLM-generated intermediate representation, supports diverse use-case formats, and yields deterministic, traceable test paths. The paper has clear strengths that deserve credit: public code, datasets, and evaluation scripts; temperature 0 for reproducibility; multiple domains; a published pipeline (AGORA) as a baseline; and a practitioner study with anonymized side-by-side comparison. The main risk is validity rather than implementation: the evaluation does not yet demonstrate that the LLM-generated CFGs are behaviorally faithful to the NL requirements, and several load-bearing metrics are anchored to an unvalidated manual oracle. This can likely be fixed with additional analysis within the manuscript's scope.

major comments (4)
  1. [§3.2.3, CFG Verification] The validation step rejects only isolated nodes, unreachable non-root nodes, and dangling edge references. It does not check whether every conditional branch stated in the NL use case is actually represented in the CFG. A connected single-root CFG can still merge or omit a branch, after which the pipeline will report 'full path coverage' of that CFG while missing requirement scenarios. This makes the RQ2 conclusion that LLMCFG-TGen 'better captures behavioral logic' not yet sustained; the claim of 'full path coverage' should be explicitly scoped to the generated CFG, not the requirements.
  2. [§4.4, Ground-Truth CFGs] The manual ground-truth CFGs are built by three doctoral students, but the paper reports no inter-annotator agreement metric. Saying that 'discrepancies among their initial versions were resolved through collaborative discussion' describes an adjudication process, not a reliability measure. Since node/edge precision/recall, nGED, and path-count discrepancy are all computed against this oracle, the headline numbers may reflect shared annotator interpretation as much as LLM capability. Please report pairwise or Fleiss' kappa, or independently audit a subset of the ground-truth CFGs.
  3. [§5.2, Table 3] The Discrepancy Rate and Avg.|Δ| are based only on the number of paths enumerated from each CFG. Two behaviorally different CFGs can have the same number of paths, so the low discrepancy rate does not by itself show that branches were neither merged nor omitted. The claim of 'more complete and less redundant test cases' needs a behavior-level check; for example, annotate whether each conditional in the original use case appears in at least one generated test case, or introduce branch-deletion mutations and show that the metrics detect them.
  4. [§4.4.1, Node Matching Threshold] The cosine-similarity threshold of 0.75 is a free parameter selected based on 'pilot experiments and earlier studies,' but no sensitivity analysis is reported. All structural metrics — node/edge precision, recall, F1, and nGED — are computed after Hungarian matching with this threshold, so the reported values are conditional on an unexamined parameter. A threshold sweep (e.g., 0.6–0.9) should be reported to show that the conclusions are not an artifact of this choice.
minor comments (4)
  1. [§4.4.1, Eq. (4)] The normalization in Eq. (4) uses |V1|+|E1|+|V2|+|E2| as the denominator. This is nonstandard and can yield negative nGED values when GED exceeds that sum. Please align the formula with the cited references and specify the exact GED cost model used.
  2. [Abstract and §1] The phrase 'full path coverage' appears repeatedly. Since DFS enumerates every path in the generated CFG by construction, this should be qualified as 'full path coverage of the generated CFG' to avoid overclaiming relative to the original requirements.
  3. [§3.2.1] The sentence introducing zero-shot versus few-shot prompting is missing a period and is slightly disconnected from the next paragraph. Consider polishing for readability.
  4. [§5.5, Internal validity] The discussion says temperature was fixed to 0, but LLM outputs can still vary across API versions and sampling implementations. A brief statement about API versions or a small repeatability check would strengthen the reproducibility claim.

Circularity Check

1 steps flagged

The headline 'full path coverage' is a definitional property of the pipeline's own DFS path enumeration; the main empirical comparisons against manual CFGs are not circular.

specific steps
  1. self definitional [Abstract; Section 1 (contributions); Section 3.3 Step 2 and Algorithm 2]
    "The generated CFGs enable full path coverage in the model, with the path-based procedure improving the clarity and accuracy of the resulting test cases. ... Step ❷ Test-Path Extraction: Starting from the root node, the CFG is traversed to enumerate all test paths. Any cyclic branch is pruned to keep the graph acyclic: This ensures that all feasible paths are collected."

    'Full path coverage' is defined by the pipeline itself: Algorithm 2's DFS enumerates every path of the LLM-generated CFG, and Step 3 creates one test case per enumerated path ('The LLM produces a complete test case for each test path'). Therefore any structurally validated CFG yields 'full path coverage' by construction, whether or not the LLM preserved every conditional branch of the NL use case. The claim reduces to the definition of the extraction algorithm rather than an empirical discovery; the non-circular evidence for completeness is the separate comparison of path counts against manually constructed CFGs.

full rationale

The paper's central quantitative claims are grounded in external comparisons and are not circular. RQ1 measures LLM-generated CFGs against manually constructed ground-truth CFGs using node/edge precision, recall, F1, and nGED; this is an empirical correspondence check, even though the manual oracle itself is constructed by the authors' students and lacks reported inter-annotator agreement. RQ2 compares the number of produced test cases with the number of paths in the manual CFGs (Discrepancy Rate, Avg.|Δ|), and RQ3 uses independent practitioner ratings. None of these metrics are fitted parameters renamed as predictions. The cosine-similarity threshold (0.75) is an evaluation hyperparameter, not part of the generation claim, and no load-bearing uniqueness theorem or ansatz is imported via self-citation. Reference [64] is a self-cited survey used only for background, not to force the paper's conclusions. The only reduction-by-construction found is the 'full path coverage' headline: because Algorithm 2 guarantees enumeration of all paths in the LLM-generated CFG and Step 3 emits one test case per path, that particular claim is true by definition and carries no independent empirical content. This is a partial circularity in the framing, not in the core comparison to manual CFGs, so the overall score is moderate.

Axiom & Free-Parameter Ledger

1 free parameters · 5 axioms · 0 invented entities

The central claims rest on a human-constructed ground truth, a semantic-similarity threshold, and the assumption that a structurally valid CFG is behaviorally faithful. No new physical or conceptual entities are introduced; the CFG is a standard software-engineering artifact.

free parameters (1)
  • cosine similarity threshold for node matching = 0.75
    Selected 'based on pilot experiments and earlier studies' (§4.4.1). This threshold determines which LLM-generated nodes count as matching the manual ground-truth nodes, and therefore drives all precision/recall/F1/nGED results.
axioms (5)
  • domain assumption A use-case description can be faithfully represented as a finite CFG whose nodes are flow steps and whose edges encode sequencing, branching, and looping.
    The entire pipeline, including Algorithm 1 and the prompt, assumes this representational adequacy without formal justification (§2.2, §3.2.1).
  • domain assumption Structural well-formedness of a CFG implies behavioral correctness
    CFG verification (§3.2.3) only checks for isolated nodes, unreachable roots, and dangling edge references. It does not check whether the graph's branch structure matches the actual requirement semantics.
  • domain assumption Manually constructed CFGs by three doctoral students are a reliable ground truth
    The reference CFGs were produced by three students who resolved disagreements through discussion (§4.4). No inter-annotator agreement or independent validation is reported.
  • domain assumption DFS with cycle pruning enumerates all test-relevant paths
    Algorithm 2 stops a path whenever a node is revisited, discarding cyclic continuations. This assumes loop iterations do not create distinct test behaviors, which is not argued.
  • ad hoc to paper BERT-based cosine similarity at a threshold of 0.75 captures semantic equivalence of steps
    The threshold is a tuning choice made from pilot experiments, and the paper does not report sensitivity to it (§4.4.1).

pith-pipeline@v1.3.0-alltime-deepseek · 23211 in / 8481 out tokens · 86569 ms · 2026-08-03T18:08:44.396396+00:00 · methodology

0 comments
read the original abstract

Appropriate test-case generation is critical in software testing and significantly impacts testing quality. Requirements-Based Test Generation (RBTG) derives test cases from software requirements to verify whether system behavior aligns with user needs and expectations. Requirements are often documented in Natural Language (NL), with use-case descriptions being a popular method for capturing functional behaviors and interaction flows in a structured, readable form. Recently, Large Language Models (LLMs) have shown strong potential for automating test generation from NL requirements. However, existing LLM-based approaches often fail to ensure comprehensive and non-redundant coverage, and may not adequately capture complex conditional logic, leading to incomplete test cases. To address these limitations, we propose an end-to-end approach called Test Generation based on LLM-generated Control Flow Graphs (LLMCFG-TGen), which generates test cases from NL use-case descriptions. It consists of three steps: (1) CFG Generation, where an LLM transforms a use case into a structured JSON-based Control Flow Graph capturing all potential branches; (2) Test-Path Extraction, where the CFG is traversed to derive execution paths; and (3) Test-Case Creation, where test cases are generated from these paths. We evaluate the approach on six use-case datasets across diverse domains. Results show that LLMs can effectively construct structured CFGs from NL use cases. Compared with two baselines, LLMCFG-TGen produces more complete and structurally consistent test cases by better capturing behavioral logic and execution flows. Both LLM-based and practitioner-based evaluations further confirm improved comprehensiveness and logical coherence while reducing manual effort.

Figures

Figures reproduced from arXiv: 2512.06401 by Chenhui Cui, Dave Towey, Nan Niu, Rubing Huang, Shikai Guo, Tao Li, Zhenzhen Yang.

Figure 1
Figure 1. Figure 1: Example test case. machines for deriving test suites, but did not provide an implementation or case study. de Figueiredo et al. [12] proposed using a formal CSP (Communicating Sequential Processes) behavioral model, built from use cases, as a basis for test generation. However, this remained a conceptual proposal, without implementation or empirical validation. Alrawashed et al. [5] defined a pipeline that… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the proposed approach, LLMCFG-TGen. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: End-to-end process, from an NL use case, to the LLM-generated CFG and derived test cases. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: LLM prompt for generating the CFG from a use case. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: LLM prompt for generating test cases based on a test path. [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Web-based interface showing the original use case and two anonymized test case sets. [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: CFG evaluation comparison (LLM-generated compared with ground-truth). [PITH_FULL_IMAGE:figures/full_fig_p018_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: “View alert” use case with generated test cases. [PITH_FULL_IMAGE:figures/full_fig_p019_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

66 extracted references · 5 linked inside Pith

  1. [1]

    Zeina Abu-Aisheh, Romain Raveaux, Jean-Yves Ramel, and Patrick Martineau. 2015. An exact graph edit distance algorithm for solving pattern recognition problems. InProceedings of the 4th International Conference on Pattern Recognition Applications and Methods. 271–278

    2015

  2. [2]

    Tanwir Ahmad, Junaid Iqbal, Adnan Ashraf, Dragos Truscan, and Ivan Porres. 2019. Model-based testing using UML activity diagrams: A systematic mapping study.Computer Science Review33 (2019), 98–112

    2019

  3. [3]

    Sai Chaithra Allala, Juan P Sotomayor, Dionny Santiago, Tariq M King, and Peter J Clarke. 2019. Towards transforming user requirements to test cases using MDE and NLP. InProceedings of the IEEE 43rd Annual Computer Software and Applications Conference, Vol. 2. 350–355

    2019

  4. [4]

    Sai Chaithra Allala, Juan P Sotomayor, Dionny Santiago, Tariq M King, and Peter J Clarke. 2022. Generating abstract test cases from user requirements using MDSE and NLP. InProceedings of the 22th International Conference on Software Quality, Reliability and Security. 744–753. , Vol. 77, No. 77, Article 777. Publication date: April 2025. 777:24 Yang et al

    2022

  5. [5]

    Thamer A Alrawashed, Ammar Almomani, Ahmad Althunibat, and Abdelfatah Tamimi. 2019. An automated approach to generate test cases from use case description model.Computer Modeling in Engineering & Sciences119, 3 (2019), 409–425

    2019

  6. [6]

    Anaconda, Inc. 2024. Conda Package Files, Version 24.5.0. https://anaconda.org/anaconda/conda/files?version=24.5.0. Accessed: November 25, 2025

    2024

  7. [7]

    Chetan Arora, Tomas Herda, and Verena Homm. 2024. Generating test scenarios from NL requirements using retrieval- augmented LLMs: An industrial study. InProceedings of the IEEE 32nd International Requirements Engineering Conference. 240–251

    2024

  8. [8]

    Shreya Bhatia, Tarushi Gandhi, Dhruv Kumar, and Pankaj Jalote. 2024. System test case design from requirements specifications: Insights and challenges of using ChatGPT.arXiv preprint arXiv:2412.03693(2024). Retrieved from https://arxiv.org/abs/2412.03693

    Pith/arXiv arXiv 2024

  9. [9]

    2000.Testing Object-Oriented Systems: Models, Patterns, and Tools

    Robert Binder. 2000.Testing Object-Oriented Systems: Models, Patterns, and Tools. Addison-Wesley Professional

    2000

  10. [10]

    International Software Testing Qualifications Board. 2014. Standard glossary of terms used in software testing

    2014

  11. [11]

    2008.Writing Effective Use Cases

    Alistair Cockburn and Lord Cockburn. 2008.Writing Effective Use Cases. Pearson Education India

    2008

  12. [12]

    André LL de Figueiredo, Wilkerson L Andrade, and Patrícia DL Machado. 2006. Generating interaction test cases for mobile phone systems from use case specifications.ACM SIGSOFT Software Engineering Notes31, 6 (2006), 1–10

    2006

  13. [13]

    Gabriele De Vito, Fabio Palomba, Carmine Gravino, Sergio Di Martino, and Filomena Ferrucci. 2023. ECHO: An approach to enhance use case quality exploiting large language models. InProceedings of the 49th Euromicro Conference on Software Engineering and Advanced Applications. 53–60

    2023

  14. [14]

    Gabriele De Vito, Gabriele Vassallo, Fabio Palomba, and Filomena Ferrucci. 2024. AGORA: An approach for generating acceptance test cases from use cases. InProceedings of the 50th Euromicro Conference on Software Engineering and Advanced Applications. 126–133

    2024

  15. [15]

    Anurag Dwarakanath and Shubhashis Sengupta. 2012. Litmus: Generation of test cases from functional requirements in natural language. InProceedings of the 17th International Conference on Applications of Natural Language Processing and Information Systems. 58–69

    2012

  16. [16]

    Mohamed El-Attar and James Miller. 2010. Developing comprehensive acceptance tests from use cases and robustness diagrams.Requirements Engineering15, 3 (2010), 285–306

    2010

  17. [17]

    Hugging Face. 2021. sentence-transformers/all-mpnet-base-v2. https://huggingface.co/sentence-transformers/all- mpnet-base-v2. Accessed: November 24, 2025

    2021

  18. [18]

    Alessio Ferrari, Sallam Abualhaija, and Chetan Arora. 2024. Model generation with LLMs: From requirements to UML sequence diagrams. InProceedings of the 32nd International Requirements Engineering Conference Workshops. 291–300

    2024

  19. [19]

    Alessio Ferrari, Giorgio Oronzo Spagnolo, and Stefania Gnesi. 2017. PURE: A dataset of public requirements documents. InProceedings of the IEEE 25th International Requirements Engineering Conference. 502–505

    2017

  20. [20]

    Peter Fröhlich and Johannes Link. 2000. Automated test case generation from dynamic models. InProceedings of the European Conference on Object-Oriented Programming. 472–491

    2000

  21. [21]

    Google DeepMind. 2025. Gemini 2.5 Flash Model Documentation. https://ai.google.dev/gemini-api/docs/models?hl=zh- cn#gemini-2.5-flash_1. Accessed: November 25, 2025

    2025

  22. [22]

    Javier J Gutierrez, Maria J Escalona, Manuel Mejias, Jesus Torres, and Arturo H Centeno. 2008. A case study for generating test cases from use cases. InProceedings of the 2nd International Conference on Research Challenges in Information Science. 209–214

    2008

  23. [23]

    Aric Hagberg, Daniel Schult, Pieter Swart, and NetworkX Developers. 2025. NetworkX: Network Analysis in Python. https://networkx.org/. Accessed: November 24, 2025

    2025

  24. [24]

    Navid Bin Hasan, Md Ashraful Islam, Junaed Younus Khan, Sanjida Senjik, and Anindya Iqbal. 2025. Automatic high- level test case generation using large language models. InProceedings of the IEEE/ACM 22nd International Conference on Mining Software Repositories. 674–685

    2025

  25. [25]

    Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. 2024. Large language models for software engineering: A systematic literature review.ACM Transactions on Software Engineering and Methodology33, 8 (2024), 1–79

    2024

  26. [26]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. GPT-4o system card.arXiv preprint arXiv:2410.21276(2024). Retrieved from https://arxiv.org/abs/2410.21276

    Pith/arXiv arXiv 2024

  27. [27]

    Adisak Intana, Kuljaree Tantayakul, Kanjana Laosen, and Suraiya Charoenreh. 2023. An approach of test case generation with software requirement ontology.International Journal of Advanced Computer Science and Applications14, 8 (2023)

    2023

  28. [28]

    Adisak Intana, Monchanok Thongthep, Phatcharee Thepnimit, Phaplak Saethapan, and Tanawat Monpipat. 2020. SYNTest: Prototype of syntax test case generation tool. InProceedings of the 5th International Conference on Information Technology. 259–264. , Vol. 77, No. 77, Article 777. Publication date: April 2025. LLMCFG-TGen 777:25

    2020

  29. [29]

    Ivar Jacobson. 1987. Object-oriented development in an industrial environment. InProceedings of the 2nd Conference on Object-Oriented Programming Systems, Languages and Applications. 183–191

    1987

  30. [30]

    Joxan Jaffar, Jorge A Navas, and Andrew E Santosa. 2010. A Path-Sensitive Control Flow Graph

    2010

  31. [31]

    Mingyue Jiang and Zuohua Ding. 2011. Automation of test case generation from textual use cases. InProceedings of the 4th International Conference on Interaction Sciences. 102–107

    2011

  32. [32]

    Kunxiang Jin and Kevin Lano. 2021. Generation of test cases from UML diagrams - A systematic literature review. In Proceedings of the 14th Innovations in Software Engineering Conference. 1–10

    2021

  33. [33]

    Long Kang, Jun Ai, and Minyan Lu. 2024. Automated structural test case generation for human-computer interaction software based on large language model. InProceedings of the 11th International Conference on Dependable Systems and Their Applications. 132–140

    2024

  34. [34]

    Nader Kesserwan, Jameela Al-Jaroodi, Nader Mohamed, and Imad Jawhar. 2023. Transforming software requirements into test cases via model transformation.International Journal of Software Engineering & Applications14, 4 (2023)

    2023

  35. [35]

    Hiroyuki Kirinuki and Haruto Tanno. 2024. ChatGPT and human synergy in black-box testing: A comparative analysis. arXiv preprint arXiv:2401.13924(2024). Retrieved from https://arxiv.org/abs/2401.13924

    Pith/arXiv arXiv 2024

  36. [36]

    Brahma Reddy Korraprolu, Pavitra Pinninti, and Y Raghu Reddy. 2025. Test case generation for requirements in natural language-An LLM comparison study. InProceedings of the 18th Innovations in Software Engineering Conference. 1–5

    2025

  37. [37]

    Harold W Kuhn. 1955. The Hungarian method for the assignment problem.Naval Research Logistics Quarterly2, 1-2 (1955), 83–97

    1955

  38. [38]

    2012.Use Cases: Requirements in Context

    Daryl Kulak and Eamonn Guiney. 2012.Use Cases: Requirements in Context. Addison-Wesley

    2012

  39. [39]

    Mohammed Lafi, Thamer Alrawashed, and Ahmad Munir Hammad. 2021. Automated test cases generation from requirements specification. InProceedings of the 2021 International Conference on Information Technology. 852–857

    2021

  40. [40]

    Yihao Li, Pan Liu, Haiyang Wang, Jie Chu, and W Eric Wong. 2025. Evaluating large language models for software testing.Computer Standards & Interfaces93 (2025), 103942

    2025

  41. [41]

    Rensis Likert. 1932. A technique for the measurement of attitudes.Archives of psychology(1932)

    1932

  42. [42]

    Hyun-il Lim. 2020. An approach to comparing control flow graphs based on basic block matching.Indian Journal of Computer Science and Engineering11, 3 (2020), 289–296

    2020

  43. [43]

    Jin Wei Lim, Thiam Kian Chiew, Moon Ting Su, Simying Ong, Hema Subramaniam, Mumtaz Begum Mustafa, and Yin Kia Chiam. 2024. Test case information extraction from requirements specifications using NLP-based unified boilerplate approach.Journal of Systems and Software211 (2024), 112005

    2024

  44. [44]

    Pan Liu and Huaikou Miao. 2010. A new approach to generating high quality test cases. InProceedings of the 19th IEEE Asian Test Symposium. 71–76

    2010

  45. [45]

    Nuno Marques, Rodrigo Rocha Silva, and Jorge Bernardino. 2024. Using ChatGPT in software requirements engineering: A comprehensive review.Future Internet16, 6 (2024), 180

    2024

  46. [46]

    Alok Mathur, Shreyaan Pradhan, Prasoon Soni, Dhruvil Patel, and Rajeshkannan Regunathan. 2023. Automated test case generation using T5 and GPT-3. InProceedings of the 9th International Conference on Advanced Computing and Communication Systems, Vol. 1. 1986–1992

    2023

  47. [47]

    Meta AI. 2025. Llama 4 Model Overview. https://www.llama.com/models/llama-4/. Accessed: November 25, 2025

    2025

  48. [48]

    Ahmad Mustafa, Wan MN Wan-Kadir, Noraini Ibrahim, Muhammad Arif Shah, Muhammad Younas, Atif Khan, Mahdi Zareei, and Faisal Alanazi. 2021. Automated test case generation from requirements: A systematic literature review. Computers, Materials and Continua67, 2 (2021), 1819–1833

    2021

  49. [49]

    OpenAI. 2025. OpenAI API Reference. https://platform.openai.com/docs/api-reference/introduction. Accessed: 2025-11-24

    2025

  50. [50]

    Shuyin Ouyang, Jie M Zhang, Mark Harman, and Meng Wang. 2025. An empirical study of the non-determinism of ChatGPT in code generation.ACM Transactions on Software Engineering and Methodology34, 2 (2025), 1–28

    2025

  51. [51]

    David Powers. 2011. Evaluation: From precision, recall and F-measure to ROC, informedness, markedness and correlation.Journal of Machine Learning Technologies2, 1 (2011), 37–63

    2011

  52. [52]

    Python Software Foundation. 2024. Python 3.12.4 Release. https://www.python.org/downloads/release/python-3124/. Accessed: November 25, 2025

    2024

  53. [53]

    Tajmilur Rahman and Yuecai Zhu. 2024. Automated user story generation with test case specification using large language model.arXiv preprint arXiv:2404.01558(2024). Retrieved from https://arxiv.org/abs/2404.01558

    Pith/arXiv arXiv 2024

  54. [54]

    Nils Reimers and Iryna Gurevych. 2019. SentenceTransformers: Sentence Embeddings using Siamese BERT-Networks. https://sbert.net/. Accessed: November 24, 2025

    2019

  55. [55]

    Kaspar Riesen and Horst Bunke. 2009. Approximate graph edit distance computation by means of bipartite graph matching.Image and Vision computing27, 7 (2009), 950–959

    2009

  56. [56]

    Saurabh Tiwari and Atul Gupta. 2015. An approach of generating test requirements for agile software development. In Proceedings of the 8th India Software Engineering Conference. 186–195. , Vol. 77, No. 77, Article 777. Publication date: April 2025. 777:26 Yang et al

    2015

  57. [57]

    Saurabh Tiwari and Atul Gupta. 2020. Use case specifications: How complete are they?Journal of Software: Evolution and Process32, 1 (2020), e2218

    2020

  58. [58]

    Ravi Prakash Verma and Md Rizwan Beg. 2013. Generation of test cases from software requirements using natural language processing. InProceedings of the 6th International Conference on Emerging Trends in Engineering and Technology. 140–147

    2013

  59. [59]

    Pauli Virtanen et al. 2025. SciPy: Scientific Library for Python. https://scipy.org/. Accessed: November 24, 2025

    2025

  60. [60]

    Chunhui Wang, Fabrizio Pastore, Arda Goknil, and Lionel C Briand. 2020. Automatic generation of acceptance test cases from use case specifications: An NLP-based approach.IEEE Transactions on Software Engineering48, 2 (2020), 585–616

    2020

  61. [61]

    Fanyu Wang, Chetan Arora, Yonghui Liu, Kaicheng Huang, Chakkrit Tantithamthavorn, Aldeida Aleti, Dishan Sambat- hkumar, and David Lo. 2025. Multi-modal requirements data-based acceptance criteria generation using LLMs.arXiv preprint arXiv:2508.06888(2025). Retrieved from https://arxiv.org/abs/2508.06888

    Pith/arXiv arXiv 2025

  62. [62]

    Junjie Wang, Yuchao Huang, Chunyang Chen, Zhe Liu, Song Wang, and Qing Wang. 2024. Software testing with large language models: Survey, landscape, and vision.IEEE Transactions on Software Engineering50, 4 (2024), 911–936

    2024

  63. [63]

    Zhiyi Xue, Liangguo Li, Senyue Tian, Xiaohong Chen, Pingping Li, Liangyu Chen, Tingting Jiang, and Min Zhang. 2024. LLM4Fin: Fully automatin LLM-powered test case generation for fintech software acceptance testing. InProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. 1643–1655

    2024

  64. [64]

    Zhenzhen Yang, Rubing Huang, Chenhui Cui, Nan Niu, and Dave Towey. 2025. Requirements-based test generation: A comprehensive survey.ACM Transactions on Software Engineering and Methodology(2025). https://doi.org/10.1145/ 3771727

    2025

  65. [65]

    Hang Yin, Hamza Mohammed, and Sai Boyapati. 2024. Leveraging pre-trained large language models (LLMs) for on-premises comprehensive automated test case generation: An empirical study. InProceedings of the 9th International Conference on Intelligent Informatics and Biomedical Sciences, Vol. 9. 597–607

    2024

  66. [66]

    Man Zhang, Tao Yue, Shaukat Ali, Huihui Zhang, and Ji Wu. 2014. A systematic approach to automatically derive test cases from use cases specified in restricted natural languages. InProceedings of the 2014 International Conference on System Analysis and Modeling. 142–157. Received 3 December 2025 , Vol. 77, No. 77, Article 777. Publication date: April 2025

    2014