REVIEW 14 cited by
ChatUniTest: A Framework for LLM-Based Test Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Unit testing is an essential yet frequently arduous task. Various automated unit test generation tools have been introduced to mitigate this challenge. Notably, methods based on large language models (LLMs) have garnered considerable attention and exhibited promising results in recent years. Nevertheless, LLM-based tools encounter limitations in generating accurate unit tests. This paper presents ChatUniTest, an LLM-based automated unit test generation framework. ChatUniTest incorporates an adaptive focal context mechanism to encompass valuable context in prompts and adheres to a generation-validation-repair mechanism to rectify errors in generated unit tests. Subsequently, we have developed ChatUniTest Core, a common library that implements core workflow, complemented by the ChatUniTest Toolchain, a suite of seamlessly integrated tools enhancing the capabilities of ChatUniTest. Our effectiveness evaluation reveals that ChatUniTest outperforms TestSpark and EvoSuite in half of the evaluated projects, achieving the highest overall line coverage. Furthermore, insights from our user study affirm that ChatUniTest delivers substantial value to various stakeholders in the software testing domain. ChatUniTest is available at https://github.com/ZJU-ACES-ISE/ChatUniTest, and the demo video is available at https://www.youtube.com/watch?v=GmfxQUqm2ZQ.
Forward citations
Cited by 14 Pith papers
-
Sakura: An Approach for Generating Complex Tests from Natural Language Test Descriptions
Sakura is a multi-agent system that generates structurally complex tests from NL descriptions, achieving 50-78% higher compilability and 38-66% higher coverage overlap than baselines on 1,464 scenarios from 20 Apache ...
-
Uncovering Business Logic Bugs via Semantics-Driven Unit Test Generation
SeGa extracts business semantics from requirements to generate unit tests that detect 22-25 more real-world business logic bugs than prior LLM-based methods in industrial Go projects.
-
Efficient Black-Box Fault Localization for System-Level Test Code Using Large Language Models
A fully static, LLM-driven pipeline estimates the execution trace from a single failure log, prunes the test code, and ranks faulty locations at function, block, and line levels, tested on 785 industrial Python test cases.
-
MigrationBench: Repository-Level Code Migration Benchmark from Java 8
MigrationBench provides 5,102 real Maven repositories as a Java 8 to Java 17 migration benchmark, plus an evaluation framework and an LLM feedback agent that achieves 62.33% minimal and 27.33% maximal migration succes...
-
SAGE: Semantic-Aware Gray-Box Game Regression Testing with Large Language Models
SAGE generates gray-box game regression tests via LLM-guided RL, compresses them with Pareto optimization, and prioritizes them via LLM-analyzed update logs, beating automated baselines on unique-bug detection at lowe...
-
Scheduzz: Constraint-based Fuzz Driver Generation with Dual Scheduling
Scheduzz combines LLM-extracted API usage constraints with a dual scheduling loop to automatically generate and run fuzz drivers, beating existing tools in coverage and finding 33 bugs in real libraries.
-
Turning the Tide: Repository-based Code Reflection
A new 1,888-case, six-language benchmark and a trained model measure and improve how code LLMs edit and repair code inside multi-file repositories.
-
The Prompt Alchemist: Automated LLM-Tailored Prompt Optimization for Test Case Generation
Automatically tailored prompts, built from diverse search, failure-derived rules, and project context, improve LLM-generated unit test coverage by roughly 6 percentage points over prior prompt optimizers.
-
Improving the Readability of Automatically Generated Tests using Large Language Models
Using a multi-step prompt that restricts LLM edits to identifiers and test names, five of nine tested LLMs improved the readability of Evosuite-generated Java tests while fully preserving their code coverage.
-
Multi-Agent Collaboration for Multilingual Code Instruction Tuning
A multi-agent collaboration framework for generating multilingual code instruction data is introduced, but the reported evaluations show the fine-tuned model underperforming its own base instruct model.
-
Leveraging Large Language Models for Command Injection Vulnerability Analysis in Python: An Empirical Study on Popular Open-Source Projects
GPT-4 detected command injection in popular Python projects with 75.5% accuracy and F1 74.5%, missing 15 real cases and beating Bandit on precision.
-
Agentic Bug Reproduction for Effective Automated Program Repair at Google
An LLM agent with a code-editing model generates plausible bug reproduction tests for 28% of Google bugs, roughly tripling the LIBRO baseline, and boosts automated repair.
-
The Current Challenges of Software Engineering in the Era of Large Language Models
The paper reports 26 challenges in LLM-based software engineering, grouped into seven aspects, derived from a structured discussion among 24 academics and practitioners.
-
A Large Language Model Approach to Identify Flakiness in C++ Projects
Fine-tuned Mistral-7b, Llama2-7b, and CodeLlama-7b classify flaky test root causes in C++ and Java, with Mistral-7b reporting perfect scores on a small augmented C++ dataset.
Discussion (0). Continue with ORCID to comment.