Pith. sign in

REVIEW 5 cited by

Large Language Models for Unit Testing: A Systematic Literature Review

Not yet reviewed by Pith; the record is open.

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2506.15227 v1 pith:V7TO3EW2 submitted 2025-06-18 cs.SE

classification cs.SE
keywords testingunitllmsresearchfuturesystematicachievementschallenges
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Unit testing is a fundamental practice in modern software engineering, with the aim of ensuring the correctness, maintainability, and reliability of individual software components. Very recently, with the advances in Large Language Models (LLMs), a rapidly growing body of research has leveraged LLMs to automate various unit testing tasks, demonstrating remarkable performance and significantly reducing manual effort. However, due to ongoing explorations in the LLM-based unit testing field, it is challenging for researchers to understand existing achievements, open challenges, and future opportunities. This paper presents the first systematic literature review on the application of LLMs in unit testing until March 2025. We analyze \numpaper{} relevant papers from the perspectives of both unit testing and LLMs. We first categorize existing unit testing tasks that benefit from LLMs, e.g., test generation and oracle generation. We then discuss several critical aspects of integrating LLMs into unit testing research, including model usage, adaptation strategies, and hybrid approaches. We further summarize key challenges that remain unresolved and outline promising directions to guide future research in this area. Overall, our paper provides a systematic overview of the research landscape to the unit testing community, helping researchers gain a comprehensive understanding of achievements and promote future research. Our artifacts are publicly available at the GitHub repository: https://github.com/iSEngLab/AwesomeLLM4UT.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Understanding and Improving Model Editing for Secure Code Generation

    cs.CR 2026-08 conditional novelty 6.0 of 10

    Editing a code model's parameters is a stronger defense against known vulnerability types than inference-time filtering, and a new post-edit tuning step largely fixes the coding-quality regression that editing causes.

  2. MultiFixer: A Coordinator-Proposer Based Multi-Agent Framework For Fixing Multi-Hunk Bugs

    cs.SE 2026-07 conditional novelty 6.0 of 10

    Coordinator-proposer multi-agent repair schedules hunks, proposes candidate patches in parallel, and selects/refines them, fixing 326/835 Defects4J bugs with GPT-3.5 and 420 with Claude-3.5-Sonnet.

  3. SGAgent: Suggestion-Guided LLM-Based Multi-Agent Framework for Repository-Level Software Repair

    cs.SE 2026-02 conditional novelty 6.0 of 10

    A three-agent locate-suggest-fix framework with a knowledge-graph toolkit resolves 154/300 SWE-Bench-Lite issues with Claude-3.5, outperforming same-model baselines by 5-10 points.

  4. PSearch: Search-based Patch Generation in the Era of LLM-based Automated Program Repair

    cs.SE 2025-07 conditional novelty 6.0 of 10

    PSearch applies Monte Carlo Tree Search to LLM patch generation with LLM and test-based rewards, fixing 201 Defects4J bugs and resolving 164 SWE-Bench-Lite issues.

  5. ReProAgent: Tool-Augmented Multi-Stage Agentic Generation of Bug Reproduction Tests from Issue Reports

    cs.SE 2026-07 conditional novelty 5.5 of 10

    A tool-augmented multi-stage agent reproduces 58–70% of real GitHub issues as fail-to-pass tests, outperforming prior prompt and agent baselines at about $0.14 per issue.

Pith tools