Pith. sign in

REVIEW 5 major objections 6 minor 36 references

Seed&Steer: Guiding Large Language Models with Compilable Prefix and Branch Signals for Unit Test Generation

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Seed&Steer argues that LLM unit-test failures split into two independent problems—uncompilable prefixes and shallow assertions—and that each has a distinct remedy: seed with EvoSuite invocation examples, then steer assertions with branch…

desk verdict Useful two-stage pipeline for LLM unit test generation, but the coverage headline is inflated by counting failing tests; the compilation gains are the solid part. read the letter →

arxiv 2507.17271 v1 pith:J5ZBPKGE submitted 2025-07-23 cs.SE

Shuaiyu Zhou , Zhengran Zeng , Xiaoling Zhou , Rui Xie , Shikun Zhang , Wei Ye This is my paper
classification cs.SE
keywords unittestgenerationlargelanguagemodelsSeed&SteerEvoSuitebranchintentionscompilationpassratecodecoveragecyclomaticcomplexity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the two stages of an LLM-generated unit test fail for different reasons and should be handled separately. Test prefixes are the compilation bottleneck: the more objects, method calls, and parameters needed before the focal method runs, the less likely a model writes code that compiles. Assertions are the coverage bottleneck: the more branching and control flow a method has, the more generic test instructions produce tests that only skim part of the code. Seed&Steer therefore feeds the model compilable prefixes built from EvoSuite-generated invocation examples, then supplies branch-level intentions that name the condition under which each if, loop, switch, or exception path is taken, before asking for assertions. On 8,192 focal methods from five Java projects, the paper reports compilation pass rates of 92.77% for gpt-3.5-turbo and 95.80% for gpt-4o, recovering 792 and 887 previously failing cases, with branch and line coverage near 72–75% and relative coverage gains of 1.09× to 1.26×.

What carries the argument

The load-bearing mechanism is a two-stage prompt construction. In the Seed stage, the system runs EvoSuite on each class containing the focal method, extracts up to three to five method-invocation cases (from EvoSuite tests or from source-level calls), and uses them to prompt the LLM to emit only the test prefix, with a placeholder comment for assertions, iteratively repaired against compiler diagnostics for at most five rounds. In the Steer stage, the system statically parses the focal method's conditional branches, loops, exception handlers, and input-dependent statements, asks the LLM to translate each branch point into a natural-language triggering condition (the branch intention), and concatenates a verified prefix, the branch intentions, and a function-level intent summary into the final assertion-generation prompt. The two complexity measures—Initialization Complexity, a weighted sum of normalized variable-declaration, object-creation, pre-invocation-call, and parameter counts, and Cyclomatic Complexity from static analysis—are used to characterize why each stage is needed rather than to drive the generation directly.

What would settle it

Run Seed&Steer on a set where EvoSuite fails to generate any invocation for a substantial share of focal methods (methods depending on external resources or using non-Java-8 features) and compare compilation pass rates on just those methods against a vanilla LLM prompt; if the rate is not higher, the seed mechanism is not the cause of the reported gains.

Watch

Extended reading notes

Core claim

The central claim is that LLM-based unit test generation is best understood as two coupled but separable problems: building a compilable call context for the method under test, and writing assertions that exercise its execution paths. The paper argues that the first problem is governed by initialization complexity—how many objects, method calls, and parameters must be assembled before the focal method runs—and that supplying correct invocation examples from EvoSuite as seed prefixes largely removes the compilation bottleneck. The second problem is governed by structural complexity, and the paper argues that replacing generic “write a test” instructions with explicit branch intentions (what condition makes each if, loop, switch, or exception path fire) leads to substantially deeper branch and line coverage. On five Defects4J projects with 8,192 focal methods, Seed&Steer is shown to raise compilation pass rates to 92.77% and 95.80% on two LLM backends, to make 792 and 887 previously failing cases compile, and to reach up to roughly 73% branch and line coverage.

Load-bearing premise

The method works only if, for every function being tested, EvoSuite or the source code yields at least one correct example of how to call it—if no such example exists, the prefix-guiding seed disappears and the reported compilation advantage may disappear with it.

Editorial extensions

If this is right

  • Keeping the Seed component should push compilation pass rates on Java methods with complex initialization above 95% for strong LLM backends, regardless of the model's prior familiarity with the library.
  • Steer-style branch intentions should transfer to other languages and models whenever the target method's control flow can be parsed statically, because the signal comes from the code rather than from model weights.
  • The decomposition means compilation and coverage can be improved independently: better seed sources raise compilability without needing stronger models, while better branch summarization raises coverage without needing more repair rounds.
  • Because the full pipeline compiles slightly less often than Seed alone, users can trade a few points of compilability for substantially higher coverage by tuning how much branch guidance is injected.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested extension is whether the Seed stage can be replaced by any deterministic invocation synthesizer, such as Randoop or hand-written examples, which would remove the Java 8/EvoSuite dependency and test whether the benefit comes from the presence of a compilable example rather than from EvoSuite specifically.
  • The paper's own complexity measures imply a sharp prediction the authors did not report: methods with high Initialization Complexity but low Cyclomatic Complexity should benefit almost entirely from Seed and hardly at all from Steer, while the reverse should hold for high-Cyclomatic, low-Initialization methods.
  • For focal methods that nothing else calls, the fallback path depends on EvoSuite reaching them; one could test whether augmenting the seed pool with tests of structurally similar methods rescues those cases.
  • Coverage is measured by branch and line coverage, not by mutation score or fault detection, so whether the newly covered branches actually catch real bugs remains an open question.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes Seed&Steer, a two-stage LLM-based unit test generation approach. In the Seed stage, method-invocation examples mined from EvoSuite-generated tests and from source-level call sites are used to prompt the LLM to produce a compilable test prefix. In the Steer stage, branch intents extracted from the focal method's control flow, together with a method-level intent summary, are used to prompt the LLM to generate assertions. The method is evaluated on 8,192 focal methods from five Java projects, comparing compilation pass rate, test pass rate, branch coverage, and line coverage against ChatGPT baselines, ChatUniTest, ChatTester, and TestART. The paper reports compilation pass rates of 92.77% (gpt-3.5-turbo) and 95.80% (gpt-4o), coverage around 72–75%, and a coverage improvement of 1.09×–1.26× from the Steer component, plus ablations and an open-source model generalization study.

Significance. The core idea of decoupling prefix generation from assertion generation, and using a conventional search-based tool only to supply invocation seeds while letting the LLM handle assertions, is practical and potentially impactful for LLM-based test generation. The paper also attempts to characterize difficulty via Initialization Complexity and Cyclomatic Complexity, and it includes an ablation study and a generalization study to Qwen2.5-Coder. If the evaluation were clean, the reported compile-rate gains and coverage improvements would be a useful step beyond prompt-only baselines. However, the central coverage claim is currently undermined by the inclusion of failing tests in the coverage measurement, and the complexity-motivation analysis is partly circular. The contribution is therefore plausible but not yet established.

major comments (5)
  1. [Section 3.4, Table 2, Figure 3] The coverage metric is contaminated by failing tests. Section 3.4 states that if an error persists after δ attempts, the test is marked as 'partially valid' and its coverage is still recorded, and the error taxonomy includes assertion failures and exceptions. Table 2 shows test pass rates of 69.87% (gpt-3.5-turbo) and 69.34% (gpt-4o) while reporting branch coverage of 72.19% and 73.30% and line coverage of 71.20% and 75.26%, which is only possible if coverage is aggregated over passing and failing tests. This directly affects the RQ4 claim that Steer improves coverage by 1.09×–1.26×: a test that reaches a branch but then fails its assertion still contributes coverage, which is exactly the behavior branch-intent prompting encourages. Please report coverage computed only from tests that compile and pass, and verify that all baselines were evaluated under the same inclusion rule; otherwise the coverage numbers do not measure usable tests.
  2. [Section 2.1, Eq. (1), Figure 1a] The Initialization Complexity index is fitted on the same data that is later used as evidence for the causal claim that initialization complexity drives compilation failure. The weights [0.1, 0.1, 0.4, 0.4] are chosen 'through correlation analysis' on the very dataset whose compilation outcomes are then binned against the index. This is a post-hoc fit, not an independent validation. To support the paper's claim that initialization complexity is a major obstacle, the index should be validated on held-out methods or the correlation should be reported cross-validated (e.g., fitting weights on one split and measuring the trend on another). As written, the monotone decline in Figure 1a is partly guaranteed by construction.
  3. [Section 5.4, Table 4, Abstract] The abstract and Finding 3 attribute the recovery of 792 and 887 previously failing cases to Seed&Steer, but Table 4 shows that these numbers come from the SeedOnly variant, not the full pipeline. In fact, the full Seed&Steer pipeline has a lower compilation pass rate than SeedOnly (92.77% vs. 95.54% for gpt-3.5-turbo, and 95.80% vs. 98.03% for gpt-4o). This is a material attribution error. Relatedly, the abstract's 'approximately 7%' improvement in compilation pass rate is not directly supported by Table 2: from ChatGPT-3.5 to Seed&Steer(gpt-3.5-turbo) the gain is 18.38 percentage points, and from ChatGPT-4.0 to Seed&Steer(gpt-4o) it is 15.30 percentage points. Please state exactly which comparison yields 7% and correct the attribution of the 792/887 counts.
  4. [Tables 2–4 and Figure 3] The paper reports no variance or statistical significance for the main quantitative claims. Section 5.6 states that the evaluation 'repeat[s] each test multiple times and reports averaged outcomes,' but Tables 2–4 and Figure 3 show only point estimates. Given LLM sampling randomness, the differences between Seed&Steer and TestART (e.g., +2.79 percentage points branch coverage in Table 2) could be within noise. Please report the number of runs, standard deviations or confidence intervals, and paired significance tests across focal methods. Without this, the claim that Seed&Steer outperforms TestART on coverage is not robustly supported.
  5. [Section 4.2, Section 4.3, Table 2] The baseline comparison is not fully controlled. ChatUniTest results are 'obtained from the official results reported in TestART' rather than run under the same harness, while Section 4.3 says that all baselines 'including TestART' use the gpt-3.5-turbo API, which is inconsistent with the separate ChatGPT-4.0 row and with the later use of gpt-4o. It is also unclear whether ChatUniTest and the inherited TestART results used the same coverage-inclusion rule (passing tests only vs. partially valid tests) as the rest of Table 2. Please either re-run all baselines under identical conditions, including identical repair budgets and identical coverage aggregation, or clearly state which numbers are inherited and justify comparability.
minor comments (6)
  1. [Abstract and Section 8] The phrase 'improves the compilation pass rate by approximately 7%' is ambiguous (absolute vs. relative percentage points) and, as noted above, does not match Table 2; please clarify the comparison and metric.
  2. [Section 3.2] The choice of 'at most three' EvoSuite examples in Path 1 and 'up to five' in Path 2 is not justified; please state how these caps were chosen and whether results are sensitive to them.
  3. [Eq. (1)] The min-max normalization used for features V, O, M, and P is not specified (e.g., normalized per project or over the whole dataset), and the range of the resulting complexity score is not stated; please define this precisely.
  4. [Table 5] The column format '27.45 [12.18, 37.49]' is not explained; please label whether the bracketed values are a range, confidence interval, or quartiles, and define the averaging unit.
  5. [Table 2] In the Csv row for Seed&Steer(gpt-4o), line coverage is listed as '78.35' without a percent sign, unlike all other entries; also '78.35%' appears inconsistent with the project's branch coverage and should be checked.
  6. [Section 5.5, Figure 3] The text says relative improvement ranges from 1.09× to 1.26× but also mentions 'up to 1.6× coverage improvement' for high CCN methods; please reconcile these numbers and specify whether Figure 3 shows mean or per-method ratios.

Circularity Check

1 steps flagged · score 4.0 of 10

Initialization Complexity index is fitted to compilation outcomes and then cited as evidence; main Seed&Steer benchmark comparison is not circular.

  1. fitted input called prediction [Section 2.1, Eq. (1), Figure 1a]
    "Through correlation analysis, we observed that the number of method calls before invoking the focal method M and the number of parameters P are more strongly negatively correlated with compilation success. Therefore, we assign higher weights to these two factors in the final complexity score, resulting in a weight configuration of [0.1, 0.1, 0.4, 0.4]... tests with complexity scores between 1 and 2 achieve a pass rate of 86.58%, while those in the 9–10 range drop to only 10.72%."

    The Initialization Complexity index is not an independent measure: its weights are fit to the same compilation-success data it is then used to explain. Because M and P were given high weights precisely because they already negatively correlate with compilation success, plotting the weighted index against compilation pass rate largely replays the fitting signal. The paper presents this as a revealed causal difficulty ('initialization complexity is a major obstacle'), but the relationship is partly enforced by construction rather than independently validated. The raw correlations would be evidence; the post-hoc weighted index adds no new support.

full rationale

The main Seed&Steer pipeline is externally grounded: EvoSuite-derived prefixes, Lizard-based cyclomatic complexity, Defects4J projects, JaCoCo coverage, and ablations against ChatTester are all independent of the paper's own definitions, and no load-bearing self-citation chain appears. The one genuine circularity is the preliminary Initialization Complexity analysis: the weights [0.1, 0.1, 0.4, 0.4] are chosen because M and P correlate with compilation failure in the same dataset, and then the weighted index's decline in pass rate is reported as a finding. This is a fitted index rather than independent validation, so it partially infects the framing claim that prefix complexity primarily affects compilation. The coverage rule that 'partially valid' failing tests still count toward JaCoCo coverage is a measurement-validity concern and could inflate coverage comparisons, but it is not a circular derivation of the kind that reduces a claimed prediction to its own inputs; it is therefore noted but not counted as a circular step. The central Seed&Steer benchmark result does not reduce to the fitted index, keeping the score at 4 rather than higher.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The empirical result stands on fitted weights and hand-set budgets, plus the assumption that EvoSuite seeds and branch-intent prompts transfer to arbitrary focal methods. No new entity, particle, or force is introduced; the only invented abstraction is the Initialization Complexity metric, which is fitted rather than independently validated.

free parameters (3)
  • Initialization Complexity weights = [0.1, 0.1, 0.4, 0.4]
    Chosen after correlation analysis on the same benchmark data used to demonstrate the trend; the choice itself determines the reported relationship between the score and compilation success.
  • Seed example caps = 3 and 5
    At most three EvoSuite cases are extracted when the source class calls the focal method, and up to five in the fallback path; no sensitivity analysis is reported.
  • Repair iteration budgets = 5 iterations, 5 seed repairs, 5 test repairs
    Hand-set to control token use; no ablation of these budgets is given, yet they directly affect compile and coverage outcomes.
assumptions (3)
  • domain assumption EvoSuite-generated tests contain correct invocation patterns for the focal method
    Section 3.2 relies on these seeds to make prefixes compilable; unreleased artifacts prevent checking seed correctness or coverage of all focal methods.
  • domain assumption Branch intentions expressed in natural language improve assertion quality and coverage
    The Steer phase's effectiveness is inferred from ablations, but the causal mechanism is not isolated from prompt-length effects, model randomness, or the repair loop.
  • domain assumption The five fixed Defects4J projects and JaCoCo coverage are representative of real-world unit test usefulness
    Coverage is measured with JaCoCo and the paper self-acknowledges in Section 5.6 that the metrics do not assess semantic correctness or test adequacy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Seed&Steer: Guiding Large Language Models with Compilable Prefix and Branch Signals for Unit Test Generation." pith.science (2026). https://pith.science/paper/J5ZBPKGE

@misc{pith2026250717271,
  author       = {Pith},
  title        = {Pith review of: Seed&Steer: Guiding Large Language Models with Compilable Prefix and Branch Signals for Unit Test Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J5ZBPKGE}},
  note         = {Machine review of arXiv:2507.17271}
}
read the original abstract

Unit tests play a vital role in the software development lifecycle. Recent advances in Large Language Model (LLM)-based approaches have significantly improved automated test generation, garnering attention from both academia and industry. We revisit LLM-based unit test generation from a novel perspective by decoupling prefix generation and assertion generation. To characterize their respective challenges, we define Initialization Complexity and adopt Cyclomatic Complexity to measure the difficulty of prefix and assertion generation, revealing that the former primarily affects compilation success, while the latter influences test coverage. To address these challenges, we propose Seed&Steer, a two-step approach that combines traditional unit testing techniques with the capabilities of large language models. Seed&Steer leverages conventional unit testing tools (e.g., EvoSuite) to generate method invocations with high compilation success rates, which serve as seeds to guide LLMs in constructing effective test contexts. It then introduces branching cues to help LLMs explore diverse execution paths (e.g., normal, boundary, and exception cases) and generate assertions with high coverage. We evaluate Seed&Steer on five real-world Java projects against state-of-the-art baselines. Results show that Seed&Steer improves the compilation pass rate by approximately 7%, successfully compiling 792 and 887 previously failing cases on two LLMs. It also achieves up to ~73% branch and line coverage across focal methods of varying complexity, with coverage improvements ranging from 1.09* to 1.26*. Our code, dataset, and experimental scripts will be publicly released to support future research and reproducibility.

Figures

Figures reproduced from arXiv: 2507.17271 by the authors.

Figure 1
Figure 1. (a)With higher initialization complexity, LLMs struggle to invoke the focal method correctly. Correct invocation [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the Seed&Steer workflow. number 𝑃. Based on these, We define the overall Initialization Complexity Score as a weighted sum of four normalized static features: Initialization Complexity = 𝑤1 · 𝑉b + 𝑤2 · 𝑂b+ 𝑤3 · 𝑀b + 𝑤4 · 𝑃b (1) Here, 𝑥bdenotes the min-max normalized value of feature 𝑥, and 𝑤1, 𝑤2, 𝑤3, 𝑤4 represent the corresponding weights. Through cor￾relation analysis, we observed that the number of me… view at source ↗
Figure 3
Figure 3. Coverage Improvements from Steer Across Methods of Varying Cyclomatic Complexity [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 30 canonical work pages

  1. [1]

    Josh Achiam et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Saranya Alagarsamy, Chakkrit Tantithamthavorn, and Aldeida Aleti. 2024. A3test: assertion-augmented automated test case generation. Information and Software Technology, 176, 107565

    work page 2024

  3. [3]

    M Moein Almasi, Hadi Hemmati, Gordon Fraser, Andrea Arcuri, and Janis Benefelds. 2017. An industrial evaluation of unit test generation: finding real faults in a financial application. In2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP) . IEEE, 263–272

    work page 2017

  4. [4]

    Nadia Alshahwan, Jubin Chheda, Anastasia Finogenova, Beliz Gokkaya, Mark Harman, Inna Harper, Alexandru Marginean, Shubho Sengupta, and Eddy Wang. 2024. Automated unit test improvement using large language models at meta. In Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering , 185–196

    work page 2024

  5. [5]

    James H Andrews, Tim Menzies, and Felix CH Li. 2011. Genetic algorithms for randomized unit testing. Ieee transactions on software engineering , 37, 1, 80–94

    work page 2011

  6. [6]

    Luciano Baresi and Matteo Miraz. 2010. Testful: automatic unit-test generation for java classes. In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering-Volume 2, 281–284

    work page 2010

  7. [7]

    Shreya Bhatia, Tarushi Gandhi, Dhruv Kumar, and Pankaj Jalote. 2024. Unit test generation using generative ai: a comparative performance analysis of autogeneration tools. In Proceedings of the 1st International Workshop on Large Language Models for Code , 54–61. Seed&Steer: Guiding Large Language Models with Compilable Prefix and Branch Signals for Unit T...

    work page 2024

  8. [8]

    Arianna Blasi, Alessandra Gorla, Michael D Ernst, and Mauro Pezzè. 2022. Call me maybe: using nlp to automatically generate unit test cases respecting tem- poral constraints. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering, 1–11

    work page 2022

Show all 36 references
  1. [9]

    Christoph Csallner, Nikolai Tillmann, and Yannis Smaragdakis. 2008. Dysy: dynamic symbolic execution for invariant inference. In Proceedings of the 30th international conference on Software engineering , 281–290

    2008

  2. [10]

    Ermira Daka and Gordon Fraser. 2014. A survey on unit testing practices and problems. In 2014 IEEE 25th International Symposium on Software Reliability Engineering. IEEE, 201–211

    2014

  3. [11]

    Pedro Delgado-Pérez, Aurora Ramírez, Kevin J Valle-Gómez, Inmaculada Medina- Bulo, and José Raúl Romero. 2022. Interevo-tr: interactive evolutionary test generation with readability assessment. IEEE Transactions on Software Engi- neering, 49, 4, 2580–2596

    2022

  4. [12]

    Pouria Derakhshanfar, Xavier Devroey, and Andy Zaidman. 2022. Basic block coverage for search-based unit testing and crash reproduction. Empirical Soft- ware Engineering, 27, 7, 192

    2022

  5. [13]

    Michael D Ernst, Jeff H Perkins, Philip J Guo, Stephen McCamant, Carlos Pacheco, Matthew S Tschantz, and Chen Xiao. 2007. The daikon system for dynamic detection of likely invariants. Science of computer programming , 69, 1-3, 35–45

    2007

  6. [14]

    Zhaopeng Feng, Yan Zhang, Hao Li, Wenqiang Liu, Jun Lang, Yang Feng, Jian Wu, and Zuozhu Liu. 2024. Improving llm-based machine translation with systematic self-correction. arXiv e-prints, arXiv–2402

    2024

  7. [15]

    Gordon Fraser and Andreas Zeller. 2010. Mutation-driven generation of unit tests and oracles. InProceedings of the 19th international symposium on Software testing and analysis, 147–158

    2010

  8. [16]

    Sepideh Kashefi Gargari and Mohammd Reza Keyvanpour. 2021. Sbst challenges from the perspective of the test techniques. In2021 12th International Conference on Information and Knowledge Technology (IKT) . IEEE, 119–123

    2021

  9. [17]

    Siqi Gu, Chunrong Fang, Quanjun Zhang, Fangyuan Tian, Jianyi Zhou, and Zhenyu Chen. 2024. Improving llm-based unit test generation via template- based repair. arXiv preprint arXiv:2408.03095

    2024 arXiv

  10. [18]

    Mark Harman and Bryan F Jones. 2001. Search-based software engineering. Information and software Technology, 43, 14, 833–839

    2001

  11. [19]

    Mark Harman and Phil McMinn. 2009. A theoretical and empirical study of search-based testing: local, global, and hybrid search. IEEE Transactions on Software Engineering, 36, 2, 226–247

    2009

  12. [20]

    René Just, Darioush Jalali, and Michael D Ernst. 2014. Defects4j: a database of existing faults to enable controlled testing studies for java programs. In Proceedings of the 2014 international symposium on software testing and analysis , 437–440

    2014

  13. [21]

    Claus Klammer and Albin Kern. 2015. Writing unit tests: it’s now or never! In 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE, 1–4

    2015

  14. [22]

    Lei Ma, Cyrille Artho, Cheng Zhang, Hiroyuki Sato, Johannes Gmeiner, and Rudolf Ramler. 2015. Grt: program-analysis-guided random testing (t). In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 212–223

    2015

  15. [23]

    Daye Nam, Andrew Macvean, Vincent Hellendoorn, Bogdan Vasilescu, and Brad Myers. 2024. Using an llm to help with code understanding. InProceedings of the IEEE/ACM 46th International Conference on Software Engineering , 1–13

    2024

  16. [24]

    Michael Olan. 2003. Unit testing: test early, test often. Journal of Computing Sciences in Colleges, 19, 2, 319–328

    2003

  17. [25]

    Carlos Pacheco, Shuvendu K Lahiri, Michael D Ernst, and Thomas Ball. 2007. Feedback-directed random test generation. In 29th International Conference on Software Engineering (ICSE’07). IEEE, 75–84

    2007

  18. [26]

    Per Runeson. 2006. A survey of unit testing practices. IEEE software, 23, 4, 22–29

    2006

  19. [27]

    Kuniaki Saito, Kihyuk Sohn, Chen-Yu Lee, and Yoshitaka Ushiku. 2024. Unsu- pervised llm adaptation for question answering. arXiv e-prints, arXiv–2402

    2024

  20. [28]

    Abdelilah Sakti, Gilles Pesant, and Yann-Gaël Guéhéneuc. 2014. Instance gen- erator and problem representation to improve object oriented code coverage. IEEE Transactions on Software Engineering , 41, 3, 294–313

    2014

  21. [29]

    Michele Tufano, Dawn Drain, Alexey Svyatkovskiy, Shao Kun Deng, and Neel Sundaresan. 2020. Unit test case generation with transformers and focal context. arXiv preprint arXiv:2009.05617

    2020 arXiv

  22. [30]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems , 30

    2017

  23. [31]

    Zejun Wang, Kaibo Liu, Ge Li, and Zhi Jin. 2024. Hits: high-coverage llm-based unit test generation via method slicing. (2024). https://arxiv.org/abs/2408.11324 arXiv: 2408.11324 [cs.SE]

    2024 arXiv

  24. [32]

    Xusheng Xiao, Sihan Li, Tao Xie, and Nikolai Tillmann. 2013. Characteristic studies of loop problems for structural test generation via symbolic execu- tion. In 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 246–256

    2013

  25. [33]

    Zhuokui Xie, Yinghao Chen, Chen Zhi, Shuiguang Deng, and Jianwei Yin. 2023. Chatunitest: a chatgpt-based automated unit test generation tool.arXiv e-prints, arXiv–2305

    2023

  26. [34]

    Zhiqiang Yuan, Yiling Lou, Mingwei Liu, Shiji Ding, Kaixin Wang, Yixuan Chen, and Xin Peng. 2023. No more manual tests? evaluating and improving chatgpt for unit test generation. arXiv preprint arXiv:2305.04207

    2023 arXiv

  27. [35]

    Andreas Zeller, Rahul Gopinath, Marcel Böhme, Gordon Fraser, and Christian Holler. 2019. The fuzzing book. (2019)

    2019

  28. [36]

    Hong Zhu, Patrick AV Hall, and John HR May. 1997. Software unit test coverage and adequacy. Acm computing surveys (csur), 29, 4, 366–427

    1997

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.