REVIEW 9 cited by
CoverUp: Effective High Coverage Test Generation for Python
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Testing is an essential part of software development. Test generation tools attempt to automate the otherwise labor-intensive task of test creation, but generating high-coverage tests remains challenging. This paper proposes CoverUp, a novel approach to driving the generation of high-coverage Python regression tests. CoverUp combines coverage analysis, code context, and feedback in prompts that iteratively guide the LLM to generate tests that improve line and branch coverage. We evaluate our prototype CoverUp implementation across a benchmark of challenging code derived from open-source Python projects and show that CoverUp substantially improves on the state of the art. Compared to CodaMosa, a hybrid search/LLM-based test generator, CoverUp achieves a per-module median line+branch coverage of 80% (vs. 47%). Compared to MuTAP, a mutation- and LLM-based test generator, CoverUp achieves an overall line+branch coverage of 89% (vs. 77%). We also demonstrate that CoverUp's performance stems not only from the LLM used but from the combined effectiveness of its components.
Forward citations
Cited by 9 Pith papers
-
PROGRESS: Property-Guided Regression Search for Semantic Falsification
PROGRESS embeds LLM-generated jqwik properties as search objectives in EvoSuite's DynaMOSA, detecting 58% of injected Java bugs that regression assertions miss and satisfying hard preconditions 4x more often than stan...
-
Sakura: An Approach for Generating Complex Tests from Natural Language Test Descriptions
Sakura is a multi-agent system that generates structurally complex tests from NL descriptions, achieving 50-78% higher compilability and 38-66% higher coverage overlap than baselines on 1,464 scenarios from 20 Apache ...
-
From Resource Flow to Executable Tests: Petri-Net-Guided LLM Test Generation for Concurrent Stateful Rust APIs
Petri-net models of Rust APIs can guide LLMs to produce executable concurrent tests whose structure stays faithful to the modeled resource flow.
-
Multi-Agent LLM Collaboration for Unit Test Generation via Human-Testing-Inspired Workflows
A three-agent LLM workflow plus a test-specialized knowledge graph generates unit tests that beat prior LLM and search-based tools on coverage and mutation score.
-
On the risk of coding before testing: An empirical study on LLM-based test generation workflow
Generating LLM tests after faulty LLM code cuts fault detection to about 14% versus about 25% when tests are generated from the task description alone, across models and prompting strategies.
-
Hamster: A Large-Scale Study and Characterization of Developer-Written Tests
A large-scale study of 1.7M developer-written Java tests shows they commonly use fixtures, mocking, structured inputs, and multi-class scope, features largely missing from automated test generation tools.
-
Combining Type Inference and Automated Unit Test Generation for Python
A proxy-based type tracing technique integrated into Pynguin infers parameter and return types during test generation and improves branch coverage.
-
How well LLM-based test generation techniques perform with newer LLM versions?
With newer LLMs, a plainly prompted generation loop matches or beats four engineered test-generation tools on coverage and mutation score, and a class-then-method hybrid cuts LLM queries by about 20%.
-
LLM-Based Agentic Systems for Software Engineering: Challenges and Opportunities
A survey of LLM-based multi-agent systems across the software development life cycle, plus a research agenda for orchestration, human coordination, cost, and data.
Discussion (0). Continue with ORCID to comment.