REVIEW 11 cited by
TestART: Improving LLM-based Unit Testing via Co-evolution of Automated Generation and Repair Iteration
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
TestART: Improving LLM-based Unit Testing via Co-evolution of Automated Generation and Repair Iteration
read the original abstract
Unit testing is crucial for detecting bugs in individual program units but consumes time and effort. Recently, large language models (LLMs) have demonstrated remarkable capabilities in generating unit test cases. However, several problems limit their ability to generate high-quality unit test cases: (1) compilation and runtime errors caused by the hallucination of LLMs; (2) lack of testing and coverage feedback information restricting the increase of code coverage;(3) the repetitive suppression problem causing invalid LLM-based repair and generation attempts. To address these limitations, we propose TestART, a novel unit test generation method. TestART improves LLM-based unit testing via co-evolution of automated generation and repair iteration, representing a significant advancement in automated unit test generation. TestART leverages the template-based repair strategy to effectively fix bugs in LLM-generated test cases for the first time. Meanwhile, TestART extracts coverage information from successful test cases and uses it as coverage-guided testing feedback. It also incorporates positive prompt injection to prevent repetition suppression, thereby enhancing the sufficiency of the final test case. This synergy between generation and repair elevates the correctness and sufficiency of the produced test cases significantly beyond previous methods. In comparative experiments, TestART demonstrates an 18% improvement in pass rate and a 20% enhancement in coverage across three types of datasets compared to baseline models. Additionally, it achieves better coverage rates than EvoSuite with only half the number of test cases. These results demonstrate TestART's superior ability to produce high-quality unit test cases by harnessing the power of LLMs while overcoming their inherent flaws.
Forward citations
Cited by 11 Pith papers
-
Generating Complex Code Analyzers from Natural Language Questions
Merlin generates CodeQL queries from natural language questions via RAG-based iteration and a self-test technique using assistive queries, achieving 3.8x higher task accuracy and 31% less completion time in user studi...
-
An Iterative Test-and-Repair Framework for Competitive Code Generation
FixAudit improves LLM code generation on competitive programming benchmarks by training a shared model for iterative code-aware test generation and repair, achieving 35%+ gains in Pass@1 over baselines on the same 7B model.
-
Multi-Agent LLM Collaboration for Unit Test Generation via Human-Testing-Inspired Workflows
A three-agent LLM workflow plus a test-specialized knowledge graph generates unit tests that beat prior LLM and search-based tools on coverage and mutation score.
-
Mining Workflow Graphs for Black-Box Boundary Testing of Conversational LLM Agents
Mining a directly-follows workflow graph from black-box conversations with an LLM agent enables structurally targeted boundary testing that covers 23–38 distinct stateful boundaries per agent, roughly doubling prompt-...
-
TATG: Tracking-Aware Testing Objective for LLM-based Test Generation
Tracking individual static and feedback testing objectives across a coverage-then-mutation two-stage LLM workflow raises line/branch coverage and mutation score on complex Java methods versus KTester and PANTA.
-
An Iterative Test-and-Repair Framework for Competitive Code Generation
A four-stage RL framework that iteratively repairs one code candidate via a code-reading Auditor and a Fixer outperforms larger models and CURE/Specine on competitive programming Pass@1.
-
TestDecision: Sequential Test Suite Generation via Greedy Optimization and Reinforcement Learning
By proving test suite coverage is monotone submodular and training LLMs with RL to maximize marginal gains, TestDecision improves branch coverage 38-52% and bug detection up to 95% over base models on ULT and LiveCodeBench.
-
Mutation-Guided Unit Test Generation with a Large Language Model
MUTGEN incorporates mutation feedback into LLM prompts and uses iteration to generate unit tests that achieve higher mutation scores than EvoSuite or vanilla LLM prompting on 204 benchmark subjects.
-
ReProAgent: Tool-Augmented Multi-Stage Agentic Generation of Bug Reproduction Tests from Issue Reports
A tool-augmented multi-stage agent reproduces 58–70% of real GitHub issues as fail-to-pass tests, outperforming prior prompt and agent baselines at about $0.14 per issue.
-
How well LLM-based test generation techniques perform with newer LLM versions?
With newer LLMs, a plainly prompted generation loop matches or beats four engineered test-generation tools on coverage and mutation score, and a class-then-method hybrid cuts LLM queries by about 20%.
-
Knowledge Matters: Injecting Project and Testing Knowledge into LLM-based Unit Test Generation
KTester improves LLM-generated Java unit tests on a 110-method benchmark by injecting static project knowledge and decoupling test-case design from test-method generation, beating four baselines.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.