REVIEW 6 cited by
LLM Test Generation via Iterative Hybrid Program Analysis
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Automating unit test generation remains a significant challenge, particularly for complex methods in real-world projects. While Large Language Models (LLMs) have made strides in code generation, they struggle to achieve high branch coverage due to their limited ability to reason about intricate control flow structures. To address this limitation, we introduce Panta, a technique that emulates the iterative process human developers follow when analyzing code and constructing test cases. Panta integrates static control flow analysis and dynamic code coverage analysis to systematically guide LLMs in identifying uncovered execution paths and generating better test cases. By incorporating an iterative feedback-driven mechanism, our technique continuously refines test generation based on static and dynamic path coverage insights, ensuring more comprehensive and effective testing. Our empirical evaluation, conducted on classes with high cyclomatic complexity from open-source projects, demonstrates that Panta achieves 26% higher line coverage and 23% higher branch coverage compared to the state-of-the-art.
Forward citations
Cited by 6 Pith papers
-
Knowledge-Guided Synthetic Bug Feedback for LLM-Based Unit Test Generation
Mechanism-guided synthetic-bug feedback raises real-bug detection of LLM unit tests to 72.67% RBDR on 172 Defects4J tasks, outperforming mutation and coverage baselines by ~20 points.
-
TATG: Tracking-Aware Testing Objective for LLM-based Test Generation
Tracking individual static and feedback testing objectives across a coverage-then-mutation two-stage LLM workflow raises line/branch coverage and mutation score on complex Java methods versus KTester and PANTA.
-
Hamster: A Large-Scale Study and Characterization of Developer-Written Tests
A large-scale study of 1.7M developer-written Java tests shows they commonly use fixtures, mocking, structured inputs, and multi-class scope, features largely missing from automated test generation tools.
-
Refining LLM-based Directed Test Input Generation via Runtime Value Feedback
A feedback loop that injects debugger-observed runtime values into LLM prompts improves directed test input generation, reaching target lines in three of four real-world case studies.
-
How well LLM-based test generation techniques perform with newer LLM versions?
With newer LLMs, a plainly prompted generation loop matches or beats four engineered test-generation tools on coverage and mutation score, and a class-then-method hybrid cuts LLM queries by about 20%.
-
ATLAS: Multi-View Code Representation Tool for C and C++ Source Programs
ATLAS claims no-build aligned AST/CFG/DFG extraction for C/C++ at 96.8%/91.7% CFG correctness, and a 34.7-point LLM line-coverage lift when fed its CFG paths.
Discussion (0). Continue with ORCID to comment.