Evaluating Large Language Models for the Generation of Unit Tests with Equivalence Partitions and Boundary Values

Alejandro Fernandez; Gustavo Rossi; Mart\'in Rodr\'iguez

REVIEW 3 major objections 6 minor 20 references

Evaluating Large Language Models for the Generation of Unit Tests with Equivalence Partitions and Boundary Values

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LLM-generated unit tests match manual coverage but not boundary precision, so human supervision remains necessary.

desk verdict Honest small-scale comparison of GPT-4 vs. manual unit tests with a checkable artifact set; the central boundary-value finding is interesting but unblinded and unquantified. read the letter →

arxiv 2505.09830 v1 pith:PA2VUEVL submitted 2025-05-14 cs.SE cs.AI

Martín Rodríguez , Gustavo Rossi , Alejandro Fernandez This is my paper · ORCID

classification cs.SEcs.AI

keywords LLMunittestgenerationequivalencepartitionsboundaryvalueanalysisqualitymutationtestingsmellspromptengineeringhumansupervision

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether a large language model can replace a human in writing unit tests when the goal is to exercise equivalence partitions and boundary values. It compares tests generated by a tuned prompt on ten Java artifacts against tests written by an expert developer, using coverage, mutation testing, test-smell detection, and manual qualitative review. The quantitative metrics are largely similar: both approaches reach full branch coverage in most scenarios and comparable mutation scores. The qualitative analysis is where the difference appears: human tests probe values as close as possible to partition edges, while the model tends to choose representative mid-range values. The paper concludes that an LLM cannot yet write unit tests fully autonomously, and that a knowledgeable tester can guide it with precise prompts.

What carries the argument

The central mechanism is an iteratively refined prompt template that role-plays a senior Java tester, feeds in the artifact's functional specification and source code, asks for JUnit tests organized around equivalence partitions and boundary values, and is run three times per artifact through the model's API at a low temperature. The evaluation pipeline pairs automated metrics (branch and line coverage, mutation testing, test-smell detection) with a manual qualitative rubric covering partition coverage, boundary exactness, exception handling, floating-point tolerance, naming, structure, and documentation. The comparison of these two lenses is what carries the conclusion: automated metrics alone would show near-parity, while the qualitative lens exposes the boundary-value gap.

What would settle it

Run a holdout study: tune the prompt on one set of artifacts, freeze it, then ask the model to generate tests for a new set of unseen artifacts whose boundary values are known; compare the boundary-value precision and mutation scores against manual tests. If the frozen prompt yields boundary-exact tests without human correction, the conclusion that supervision is required would be weakened.

Watch

Extended reading notes

Core claim

The central finding is that the tested LLM, given an optimized prompt that includes both the functional specification and the implementation of a method, produces test suites whose measured coverage and mutation scores are generally on par with manual tests, yet whose boundary-value coverage is qualitatively weaker. Manual tests concentrate on the edges of each equivalence partition and on exact exception messages; the LLM's tests favor representative values near the middle of ranges, occasionally test beyond the boundary rather than exactly at it, and make floating-point comparisons without a tolerance delta. As a result, the paper concludes that the model cannot yet write unit tests without human supervision, although a tester can steer it effectively through well-designed prompts.

Load-bearing premise

The load-bearing premise is that iteratively tuning the prompt template on the same ten artifacts that were later used for evaluation does not inflate the model's apparent test quality.

Editorial extensions

If this is right

If the claim holds, LLMs are useful as a first-pass test generator, but a human reviewer must still check and tighten boundary values, exception messages, and floating-point assertions.
Teams adopting LLM test generation should treat prompt design as a core skill: the model's effectiveness is conditional on well-specified requirements and a clear test-design prompt.
Automated quality gates such as coverage and mutation score are not enough to judge test quality; manual qualitative review remains necessary to catch edge-case precision.
The model's boundary-value weakness suggests that future work should focus on prompting strategies that force exact edge values and explicit deltas for floating-point comparisons.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the mid-range bias is a stable property of the model rather than an artifact of the prompt, a simple prompt extension that demands exact edge values plus a floating-point delta could substantially close the gap without human rewriting.
The same protocol applied to other languages and testing frameworks, such as Python with pytest or JavaScript with Jest, would show whether the boundary-value deficit is general or tied to the Java/JUnit setting.
Because the prompt was tuned on the same ten artifacts used for evaluation, the claimed performance may be optimistic; a holdout design with a separately tuned prompt would clarify how much of the gap is intrinsic to the model.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

This is a small empirical study comparing GPT-4-generated JUnit tests against tests written by one developer across 10 ad hoc Java artifacts, with equivalence partitioning and boundary values as the test-design criteria. The best part is the artifact set: the repo ships the scenarios, specifications, prompt template, and all generated and manual tests, so the comparisons are checkable. The paper is also honest about its failures—one non-compiling test, three floating-point assertion errors—and its quantitative claims are modest and mostly match the tables. The new bit is the combination: a prompt that includes both requirements and code, plus the explicit comparison on boundary-value selections rather than just coverage or mutation scores.

What the paper does well: it reports exactly the kind of low-level observation that practitioners building test-generation tooling need—that GPT tests tend to pick representative middle values instead of edge values, and that float comparisons need a delta argument. The qualitative analysis is clearly the load-bearing finding for the "still requires human supervision" conclusion, since coverage and mutation scores are mostly at parity.

Soft spots, in order. First, the boundary-value finding is unblinded and unquantified. The claim that GPT tests "tend to use middle values" is stated without counts, examples, or an operational definition of "boundary," and the comparison is against a single developer. If an independent rater went through the test files with a checklist, would the difference survive? The paper's own limitations section mentions subjective bias but offers no mitigation. Second, the prompt template was iteratively refined on the same 10 artifacts used for evaluation (Section 3.2). The authors report this openly, which I respect, but it means the GPT scores are partly fitted to the benchmark. A holdout scenario or a frozen prompt from the first iteration would have made the comparison fairer. Third, the human baseline is one developer; that is not a population. None of these are fatal given the modest conclusions, but they all push in the same direction: treat the quantitative parity as real, treat the boundary-value difference as an interesting hypothesis rather than measured fact.

The citation pattern is fine—Guilherme, Schäfer, and Siddiq are the right prior work, and the differences are stated accurately. No invented entities, no hidden free parameters beyond what is stated (temperature 0.3, three runs).

Who gets value: anyone evaluating LLM test generation in practice, or building tooling that prompts LLMs for tests. It deserves a serious referee; the artifact availability and honest reporting make it worth a round of review, with the main work being to operationalize and blind the qualitative analysis.

Recommendation: send it to peer review, conditional on tightening the qualitative method.

Referee Report

3 major / 6 minor

Summary. This paper reports a controlled comparison between unit tests generated by GPT-4 (via a purpose-built prompt template) and tests written manually by a single developer, across 10 small Java artifacts constructed to exercise equivalence partitions and boundary values. The comparison uses branch/line coverage, mutation testing via PIT, test-smell detection via TsDetect, and a manual qualitative analysis by the authors. The main empirical claims are that both approaches achieve near-ceiling branch coverage and mutation scores; that LLM-generated tests occasionally fail to compile or mishandle floating-point comparisons; and that, qualitatively, manual tests probe boundary values more aggressively while LLM tests tend to pick representative middle values. The paper concludes that LLMs cannot yet write unit tests fully autonomously and still require human supervision.

Significance. The paper makes a modest but useful contribution to the empirical literature on LLM-based test generation. Its strengths include a publicly available repository with the artifacts, prompts, and generated tests; the honest reporting of failures (one non-compiling test and several floating-point assertion errors); the use of multiple complementary evaluation instruments (coverage, mutation, smells, and qualitative analysis); and a clearly stated, appropriately cautious conclusion. If the results hold, the finding that LLM-generated tests systematically underweight true boundary values would be a useful, actionable observation for developers and tool builders. The study is limited in scope (Java/JUnit, 10 ad hoc artifacts, one LLM, one human developer), and the authors acknowledge several of these limitations. The significance therefore rests on the reliability of the qualitative boundary-value observation and on the fairness of the prompt optimization procedure, both of which need strengthening.

major comments (3)

[Section 3.2] The iterative prompt-development procedure is a load-bearing threat to the quantitative comparison. The prompt template was explicitly refined until it produced 'effective, efficient, and robust tests' on the same 10 artifacts that were later used for the final evaluation, and the authors state that the final version was reached 'against which no substantial improvements could be obtained.' This means the LLM's near-perfect coverage and mutation scores are partly the result of fitting the prompt to the evaluation set. The manual tests were not optimized in this way, so the comparison between the two arms is not a comparison of typical or out-of-the-box performance. I ask the authors to address this directly, for example by reporting the number of prompt iterations, describing what changed between iterations, and, ideally, re-evaluating with a prompt designed without access to the evaluation artifacts or with a holdout set of scenarios. Without this, the central quantitative claim ('both approaches achieve 100% branch coverage') is difficult to interpret as evidence about LLM capability rather than prompt-tuning success.
[Section 5] The qualitative boundary-value finding is the main distinctive support for the paper's conclusion that 'LLMs still require human supervision,' but it is currently reported as an unquantified, unblinded author judgment. The text states that 'manual tests tend to be more exhaustive, using values as close as possible to the edges of each partition' while 'GPT-generated tests tend to use representative values within partitions, usually towards the middle of the range,' yet no operational definition of a boundary value or a middle value is given, no counts or per-scenario examples are reported, and no inter-rater reliability or blinding procedure is described. Because the authors themselves acknowledge in Section 6 that the qualitative analysis 'introduces a possible subjective bias,' the paper should provide concrete evidence for this claim: for example, a per-scenario table listing which boundary values were tested by each approach, or an independent blind rating of the test files against a pre-specified rubric. As it stands, the strongest and most novel conclusion rests on an assertion that cannot be checked from the paper.
[Section 3.3] The manual baseline consists of tests written by a single invited developer, but the paper refers to these as 'benchmark tests created by expert programmers' and treats them as representative of trained programmers. No information is provided about the developer's experience level, how expertise in equivalence partitions and boundary values was verified, or whether the developer had prior exposure to the artifacts beyond the provided instructions. With n=1, the comparison conflates individual variation with the human baseline. I recommend either recruiting additional independent developers or, at minimum, clearly reframing the comparison as 'tests written by one developer following the study instructions' and tempering the general claim about human testers accordingly.

minor comments (6)

[Section 4, Table 2] The text says 'four GPT tests have problems: one in scenario 06 of Run 1, and three more due to errors in floating point value comparisons,' but the table shows dashes for scenario 08 Run 2, scenario 08 Run 3, and scenario 10 Run 1 in addition to scenario 06 Run 1. Please clarify exactly which runs failed and why, since the current wording is easy to misread.
[Section 4, Table 1] For scenario 06 Run 1, the text says coverage is 0.0% due to a non-existent method, but the table reports branch coverage 0.0% and line coverage 75%. Please reconcile these numbers or explain what the 75% line coverage represents.
[Section 3.2] The statement that 'three are randomly selected' from multiple script runs lacks procedural detail; if the selection was intended to be unbiased, specify the random selection mechanism or provide the seed.
[Table 3] The column headers 'B1 B2 B3 Man' are not defined in the table caption or surrounding text; please define them as the three GPT runs and the manual tests.
[Figure 2] The label 'Numeric print' appears to be a typo; it should likely read 'Numeric precision' or 'Numeric accuracy' to match the corresponding criterion in Section 3.4.
[Section 4] The statement that 100% branch coverage indicates 'all equivalence partitions were identified and executed correctly' is an overclaim, since branch coverage alone does not guarantee that all partitions or boundary values were exercised; please soften the wording to reflect what the metric actually establishes.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found; the study is an empirical comparison whose central conclusion does not reduce to its inputs.

full rationale

The paper is an empirical comparative study, not a derivation. Its central claim that LLMs still require human supervision is supported by observed compile failures, floating-point assertion errors, and a manual qualitative comparison of boundary-value choices. None of these results is obtained by defining a quantity in terms of another, by fitting a parameter and then renaming it as a prediction, or by importing a uniqueness theorem from prior work. The quantitative metrics (JaCoCo coverage, PIT mutation score, TsDetect smells) are external tools applied to both the GPT-generated and manual tests. The qualitative analysis is explicitly acknowledged in Section 6 as carrying possible subjective bias, which is a validity limitation, not a circular step: it compares observed test properties rather than defining them into existence. The iterative refinement of the prompt template on the same ten artifacts (Section 3.2) is a legitimate threat to external validity and could inflate the apparent performance of the LLM, but it does not make the evaluation circular: the paper does not present a prediction that is statistically forced by a fitted input; if anything, the optimization would bias in favor of the LLM, making the conclusion that human supervision is still required conservative. Self-citations are limited to the availability of the dataset (reference [15]) and are not load-bearing for any argument. No step in the paper reduces to its own inputs by construction, so the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The paper introduces no equations or fitted numerical parameters. The main non-obvious input is the hand-optimized prompt template and the design choices listed above; the single-developer baseline and prompt tuning are the most consequential assumptions.

free parameters (3)

temperature = 0.3
Chosen by hand to reduce output variance; directly influences all GPT-generated test suites.
number of scenarios = 10
Ad hoc set designed by the authors; explicitly acknowledged in Section 6 as limiting generalization.
number of runs = 3
Three runs randomly selected from multiple API runs; sampling method is not seeded, so stochastic variation is not fully controlled.

assumptions (6)

domain assumption Equivalence partitioning and boundary value analysis are effective test selection strategies.
Used as the evaluation criteria; taken from standard software testing literature (Graham et al., reference [7]).
domain assumption JaCoCo coverage, PIT mutation score, and TsDetect smell counts are valid proxies for unit test quality.
The quantitative comparison rests entirely on these tools; the paper does not validate them against external judgments.
domain assumption GPT-4-0613 is representative of current general-purpose LLMs.
The study uses a single model snapshot from OpenAI; results may not transfer to other models or versions.
ad hoc to paper The single invited developer's manual tests represent trained programmers.
Section 3.3 describes one expert developer; no inter-developer variance is measured, yet conclusions compare LLMs versus trained programmers.
ad hoc to paper Prompt optimization on the evaluation artifacts does not inflate LLM scores.
Section 3.2 states the template was iterated until tests were effective on the same scenarios used for evaluation; no holdout set exists.
ad hoc to paper The authors' manual qualitative analysis is unbiased despite the lack of blinding.
Section 5 and Section 6 admit possible subjective bias; the authors know which tests are LLM-generated.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Evaluating Large Language Models for the Generation of Unit Tests with Equivalence Partitions and Boundary Values." pith.science (2026). https://pith.science/paper/PA2VUEVL

@misc{pith2026250509830,
  author       = {Pith},
  title        = {Pith review of: Evaluating Large Language Models for the Generation of Unit Tests with Equivalence Partitions and Boundary Values},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PA2VUEVL}},
  note         = {Machine review of arXiv:2505.09830}
}

read the original abstract

The design and implementation of unit tests is a complex task many programmers neglect. This research evaluates the potential of Large Language Models (LLMs) in automatically generating test cases, comparing them with manual tests. An optimized prompt was developed, that integrates code and requirements, covering critical cases such as equivalence partitions and boundary values. The strengths and weaknesses of LLMs versus trained programmers were compared through quantitative metrics and manual qualitative analysis. The results show that the effectiveness of LLMs depends on well-designed prompts, robust implementation, and precise requirements. Although flexible and promising, LLMs still require human supervision. This work highlights the importance of manual qualitative analysis as an essential complement to automation in unit test evaluation.

Figures

Figures reproduced from arXiv: 2505.09830 by the authors.

**Figure 1.** Overview of the experiment’s design flexible integration and quick adjustments to prompts, enabling the systematization of the experiment. A representative set of Java artifacts is needed as a basis for testing for the study of equivalence partitions and boundary values. While the possibility of using software artifacts that had been the focus of study in previous literature was evaluated, the literature analysis d… view at source ↗

**Figure 2.** Comparison of qualitative manual analysis 6 Threads to Validity The metrics used to evaluate the generated sets include code coverage, mutation score, and bad smells detection, all of which have been used in previous research, supporting their validity. However, a manual qualitative analysis, not used in similar studies, is incorporated. This approach allows the identification of aspects that automated metrics migh… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 10 canonical work pages

[1]

In: 2022 IEEE/ACM 44th Inter- 14 Martín Rodríguez et al

Arteca, E., Harner, S., Pradel, M., Tip, F.: Nessie: Automatically testing javascript apis with asynchronous callbacks. In: 2022 IEEE/ACM 44th Inter- 14 Martín Rodríguez et al. national Conference on Software Engineering (ICSE). pp. 1494–1505 (2022). https://doi.org/10.1145/3510003.3510106

arXiv 2022
[2]

IEEE Transactions on Software EngineeringSE-13(12), 1278–1296 (1987)

Basili, V., Selby, R.: Comparing the effectiveness of software testing strate- gies. IEEE Transactions on Software EngineeringSE-13(12), 1278–1296 (1987). https://doi.org/10.1109/TSE.1987.232881

work page arXiv 1987
[3]

In: Proceed- ings of the 25th International Symposium on Software Testing and Analysis

Coles, H., Laurent, T., Henard, C., Papadakis, M., Ventresque, A.: Pit: a practical mutation testing tool for java (demo). In: Proceed- ings of the 25th International Symposium on Software Testing and Analysis. p. 449–452. ISSTA 2016, Association for Computing Machin- ery, New York, NY, USA (2016). https://doi.org/10.1145/2931037.2948707, https://doi.org/...

arXiv 2016
[4]

In: Proceedings of the IEEE/ACM 46th International Confer- ence on Software Engineering

Du, X., Liu, M., Wang, K., Wang, H., Liu, J., Chen, Y., Feng, J., Sha, C., Peng, X., Lou, Y.: Evaluating large language models in class-level code generation. In: Proceedings of the IEEE/ACM 46th International Confer- ence on Software Engineering. ICSE ’24, Association for Computing Ma- chinery, New York, NY, USA (2024). https://doi.org/10.1145/3597503.36...

arXiv 2024
[5]

In: Proceedings of the 19th ACM SIGSOFT Sympo- sium and the 13th European Conference on Foundations of Software En- gineering

Fraser, G., Arcuri, A.: Evosuite: automatic test suite generation for object- oriented software. In: Proceedings of the 19th ACM SIGSOFT Sympo- sium and the 13th European Conference on Foundations of Software En- gineering. p. 416–419. ESEC/FSE ’11, Association for Computing Machin- ery, New York, NY, USA (2011). https://doi.org/10.1145/2025113.2025179, h...

arXiv 2011
[6]

A Large-Scale Study on the Usage of Testing Patterns that Address Maintainability Attributes (Patterns for Ease of Modification, Diagnoses, and Comprehension)

Gonzalez, D., Santos, J.C.S., Popovich, A., Mirakhorli, M., Nagappan, M.: A large-scale study on the usage of testing patterns that address maintainability at- tributes (patterns for ease of modification, diagnoses, and comprehension). CoRR abs/1704.08412 (2017), http://arxiv.org/abs/1704.08412

work page Pith review arXiv 2017
[7]

Cengage Learning, Boston, 1st edn

Graham, D., van Veenendaal, E., Evans, I., Black, R.: Foundations of Software Testing: ISTQB Certification. Cengage Learning, Boston, 1st edn. (2008)

work page 2008
[8]

In: Fontão, A.L., Paiva, D.M.B., Borges, H., Cagnin, M.I., Fernandes, P.G., Borges, V., Melo, S.M., Durelli, V.H.S., Canedo, E.D

Guilherme, V., Vincenzi, A.: An initial investigation of chatgpt unit test generation capability. In: Fontão, A.L., Paiva, D.M.B., Borges, H., Cagnin, M.I., Fernandes, P.G., Borges, V., Melo, S.M., Durelli, V.H.S., Canedo, E.D. (eds.) 8th Brazilian Symposium on Systematic and Automated Soft- ware Testing, SAST 2023, Campo Grande, MS, Brazil, September 25-...

arXiv 2023

Show all 20 references

[9]

Addison-Wesley Longman Publishing Co., Inc., USA (1999)

Koomen, T., Pol, M.: Test process improvement: a practical step-by-step guide to structured testing. Addison-Wesley Longman Publishing Co., Inc., USA (1999)

1999
[10]

Gómez del Mónaco, A.P.: Tests automatizados en aplicaciones web: Caso de apli- cación en empresa de turismo internacional. Ph.D. thesis, Universidad Nacional de La Plata (2021), http://sedici.unlp.edu.ar/handle/10915/123972, accessed: 2024- 11-05

2021
[11]

OpenAI:Promptengineering(2024),https://platform.openai.com/docs/guides/prompt- engineering, accessed: 2024-10-31

2024
[12]

Overflow, S.: How gen-ai and llms help create tests for developers cod- ing software (2024), https://stackoverflow.blog/2024/09/10/gen-ai-llm-create-test- developers-coding-software-code-quality/, accessed: 2024-10-11

2024
[13]

Peruma, A., Almalki, K., Newman, C.D., Mkaouer, M.W., Ouni, A., Palomba, F.: tsdetect: an open source test smells detection tool. In: Pro- ceedings of the 28th ACM Joint Meeting on European Software Engineer- Evaluating Large Language Models for the Generation of Unit Tests 15...

2020
[14]

McGraw-Hill, 6 edn

Pressman, R.S.: Ingeniería del Software: Un Enfoque Práctico. McGraw-Hill, 6 edn. (2005)

2005
[15]

https://doi.org/10915/datasetSJL5HU, https://datos.unlp.edu.ar/citation?persistentId=perma:10915datasetSJL5HU

Rodriguez, M.E.: Dataset para evaluar LLMs en generación de tests con valores de borde y particiones de equivalencia (2025). https://doi.org/10915/datasetSJL5HU, https://datos.unlp.edu.ar/citation?persistentId=perma:10915datasetSJL5HU

2025
[16]

Schäfer, M., Nadi, S., Eghbali, A., Tip, F.: An empirical evaluation of using large language models for automated unit test generation (2023), https://arxiv.org/abs/2302.06527

2023 arXiv
[17]

In: Proceedings of the 28th International Con- ference on Evaluation and Assessment in Software Engineering

Siddiq, M.L., Da Silva Santos, J.C., Tanvir, R.H., Ulfat, N., Al Rifat, F., Carvalho Lopes, V.: Using large language models to generate junit tests: An empirical study. In: Proceedings of the 28th International Con- ference on Evaluation and Assessment in Software Engineering....

2024
[18]

Stack Overflow: Ai | 2024 stack overflow developer survey (2024), https://survey.stackoverflow.co/2024/ai, accessed: 2025-03-29

2024
[19]

Tamkin, A., Brundage, M., Clark, J., Ganguli, D.: Understanding the ca- pabilities, limitations, and societal impact of large language models (2021), https://arxiv.org/abs/2102.02503

2021 arXiv
[20]

Software: Practice and Experi- ence 53(7), 1472–1495 (Jul 2023)

Youn, D., Lee, S., Ryu, S.: Declarative static analysis for mul- tilingual programs using CodeQL. Software: Practice and Experi- ence 53(7), 1472–1495 (Jul 2023). https://doi.org/10.1002/spe.3199, https://onlinelibrary.wiley.com/doi/10.1002/spe.3199, publisher: Wiley

2023 doi

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

In: 2022 IEEE/ACM 44th Inter- 14 Martín Rodríguez et al

Arteca, E., Harner, S., Pradel, M., Tip, F.: Nessie: Automatically testing javascript apis with asynchronous callbacks. In: 2022 IEEE/ACM 44th Inter- 14 Martín Rodríguez et al. national Conference on Software Engineering (ICSE). pp. 1494–1505 (2022). https://doi.org/10.1145/3510003.3510106

arXiv 2022

[2] [2]

IEEE Transactions on Software EngineeringSE-13(12), 1278–1296 (1987)

Basili, V., Selby, R.: Comparing the effectiveness of software testing strate- gies. IEEE Transactions on Software EngineeringSE-13(12), 1278–1296 (1987). https://doi.org/10.1109/TSE.1987.232881

work page arXiv 1987

[3] [3]

In: Proceed- ings of the 25th International Symposium on Software Testing and Analysis

Coles, H., Laurent, T., Henard, C., Papadakis, M., Ventresque, A.: Pit: a practical mutation testing tool for java (demo). In: Proceed- ings of the 25th International Symposium on Software Testing and Analysis. p. 449–452. ISSTA 2016, Association for Computing Machin- ery, New York, NY, USA (2016). https://doi.org/10.1145/2931037.2948707, https://doi.org/...

arXiv 2016

[4] [4]

In: Proceedings of the IEEE/ACM 46th International Confer- ence on Software Engineering

Du, X., Liu, M., Wang, K., Wang, H., Liu, J., Chen, Y., Feng, J., Sha, C., Peng, X., Lou, Y.: Evaluating large language models in class-level code generation. In: Proceedings of the IEEE/ACM 46th International Confer- ence on Software Engineering. ICSE ’24, Association for Computing Ma- chinery, New York, NY, USA (2024). https://doi.org/10.1145/3597503.36...

arXiv 2024

[5] [5]

In: Proceedings of the 19th ACM SIGSOFT Sympo- sium and the 13th European Conference on Foundations of Software En- gineering

Fraser, G., Arcuri, A.: Evosuite: automatic test suite generation for object- oriented software. In: Proceedings of the 19th ACM SIGSOFT Sympo- sium and the 13th European Conference on Foundations of Software En- gineering. p. 416–419. ESEC/FSE ’11, Association for Computing Machin- ery, New York, NY, USA (2011). https://doi.org/10.1145/2025113.2025179, h...

arXiv 2011

[6] [6]

A Large-Scale Study on the Usage of Testing Patterns that Address Maintainability Attributes (Patterns for Ease of Modification, Diagnoses, and Comprehension)

Gonzalez, D., Santos, J.C.S., Popovich, A., Mirakhorli, M., Nagappan, M.: A large-scale study on the usage of testing patterns that address maintainability at- tributes (patterns for ease of modification, diagnoses, and comprehension). CoRR abs/1704.08412 (2017), http://arxiv.org/abs/1704.08412

work page Pith review arXiv 2017

[7] [7]

Cengage Learning, Boston, 1st edn

Graham, D., van Veenendaal, E., Evans, I., Black, R.: Foundations of Software Testing: ISTQB Certification. Cengage Learning, Boston, 1st edn. (2008)

work page 2008

[8] [8]

In: Fontão, A.L., Paiva, D.M.B., Borges, H., Cagnin, M.I., Fernandes, P.G., Borges, V., Melo, S.M., Durelli, V.H.S., Canedo, E.D

Guilherme, V., Vincenzi, A.: An initial investigation of chatgpt unit test generation capability. In: Fontão, A.L., Paiva, D.M.B., Borges, H., Cagnin, M.I., Fernandes, P.G., Borges, V., Melo, S.M., Durelli, V.H.S., Canedo, E.D. (eds.) 8th Brazilian Symposium on Systematic and Automated Soft- ware Testing, SAST 2023, Campo Grande, MS, Brazil, September 25-...

arXiv 2023