Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Repair Ingredients Are All You Need: Improving Large Language Model-Based Program Repair via Repair Ingredients Search

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ReinFix claims that supplying an LLM repair agent with two kinds of searched repair ingredients—project-specific definitions and root-cause-matched historical fixes—fixes 146 bugs in Defects4J V1.2 and 145 in V2.0, beating prior LLM-based…

desk verdict ReinFix is a sensible new integration of internal dependency search and root-cause-aware external retrieval for LLM-based program repair, but the headline gains depend on an under-specified leakage filter and uncontrolled baselines that need auditing before the numbers can be trusted. read the letter →

arxiv 2506.23100 v1 pith:IZFPW32P submitted 2025-06-29 cs.SE

Jiayi Zhang , Kai Huang , Jian Zhang , Yang Liu , Chunyang Chen This is my paper
classification cs.SE
keywords automatedprogramrepairlargelanguagemodelsingredientsretrieval-augmentedgenerationReActagentsstaticdependencyanalysisrootcauseDefects4J
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that the main bottleneck for LLM-based program repair is not the model but the absence of repair ingredients: project-specific context the model does not know, and similar past fixes it cannot recall. ReinFix wraps a ReAct-style LLM agent with optional search tools so the model can fetch internal ingredients through dependency analysis during root-cause reasoning and fetch external ingredients from a vector database of historical bug-fix pairs during patch generation. On Defects4J V1.2 the full framework fixes 146 bugs (32 more than the best prior baseline) and on V2.0 it fixes 145 bugs (38 more), with ablations attributing most of the gain to combining the two ingredient searches. If true, this means program repair improves by giving agents the right search tools rather than by fine-tuning or prompt engineering alone.

What carries the argument

The central mechanism is a two-phase repair-ingredient search wrapped in the ReAct Thought-Action-Observation loop. The first phase gives the LLM optional tools built on code-property-graph dependency analysis—identify_variable, find_variable_assignments, track_variable_dataflow, trace_method_usage, analyze_method_details, find_class_loc, identify_class, get_imports—so the model can pull internal ingredients from the buggy project only when it cannot infer a root cause. The second phase is a retrieval tool that embeds the concatenation of buggy code and root cause with a text-embedding model, compares it by cosine similarity to embeddings of bug-fix triples stored in a vector database built from a 100K-sample historical corpus, and returns the top-ranked repair pattern (buggy code, fix code, root cause, similarity score) as an external ingredient. The coupled design is what carries the argument: the root cause produced in the reasoning phase becomes part of the retrieval query in the solution phase, so the retrieved fix behaviour is matched by cause rather than by syntax alone.

What would settle it

Audit every entry in the 100K-sample corpus against all Defects4J V1.2, V2.0, and RWB target bugs using a similarity threshold below exact match (e.g., edit-distance or embedding-cosine near the retrieval cutoff). If any benchmark bug's own ground-truth fix appears in the retrieved set, or if the external component's marginal gains shrink sharply after removing such entries, the retrieval-based advantage is called into question.

Watch

Extended reading notes

Core claim

On the paper's own terms, ReinFix is an LLM-agent program repair framework that integrates internal and external repair ingredients in a two-phase ReAct loop. In the reasoning phase, static dependency analysis over the buggy project retrieves variable, method, class, and file-level definitions that help the LLM pin down the root cause. In the solution phase, the root cause and buggy code are embedded together and used to retrieve a similar historical bug-fix pair as a repair pattern, which the LLM adapts into candidate patches. The paper reports 146 correct fixes on Defects4J V1.2 and 145 on V2.0 with GPT-4o, and its ablations show that removing both components drops the count to 85, keeping only internal search yields 116, keeping only external search yields 108, and replacing the searches with FitRepair-style or RAP-Gen-style retrieval yields 110 and 119 respectively. It also reports that the framework outperforms baselines on the post-training-cutoff RWB benchmarks, which the paper uses to argue the gains are not simply memorized benchmark answers.

Load-bearing premise

The load-bearing premise is that the 100K-sample external corpus contains no near-duplicate of any benchmark bug after exact-match filtering; if that filter is incomplete, a retrieved 'historical' fix could be the target bug's own ground-truth patch, making the headline comparisons circular.

Editorial extensions

If this is right

  • Adding optional ingredient-search tools to an existing LLM repair agent improves the number of correct fixes without fine-tuning, so the approach should transfer to any base LLM that supports tool calling.
  • Root-cause-aware retrieval beats code-similarity retrieval for finding useful fix behaviours, so retrieval-augmented repair systems should index causes, not only code.
  • Dependency-analysis-based internal search supplies more precise context than similarity-based identifier retrieval, reducing irrelevant donor code that misleads root-cause analysis.
  • The two ingredient searches are complementary: each alone improves over the base model (116 and 108 vs 85), and together they fix 146, so future APR agents should combine both rather than pick one.
  • The framework keeps its advantage on post-training-cutoff benchmarks, indicating the measured gains are not attributable to the LLM having memorized the benchmark fixes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the external-retrieval margin should be re-measured after pruning the 100K-sample corpus of any entry that is semantically or structurally near-duplicate of a benchmark buggy function, not just exact-match duplicates; that would quantify how much of the 38-bug margin depends on near-duplicate retrieval.
  • Editorial inference: because the agent chooses when to call each tool, a useful extension is to record tool-invocation frequency per bug and test whether the gains come from ingredient search itself or simply from giving the model more reasoning steps and a larger token budget.
  • Editorial inference: root-cause-aware retrieval is a general recipe that should transfer to other software-maintenance tasks where past fixes cluster by cause, such as vulnerability patching or build-failure repair.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. ReinFix is an LLM-agent framework for Java program repair. It augments a ReAct-based agent with two repair-ingredient search mechanisms: (i) in the reasoning phase, Joern-based dependency analysis tools (variable, method, class, and file levels) retrieve internal donor code to support root-cause analysis; (ii) in the solution phase, an embedding-based retrieval tool queries a vector database of 100K randomly sampled bug-fix pairs from TRANSFER, each augmented with an LLM-generated root cause, and returns top-ranked repair patterns to guide patch generation. The paper evaluates on Defects4J V1.2 and V2.0 plus the RWB post-cutoff benchmarks, reporting that GPT-4o-based ReinFix fixes 146 and 145 bugs respectively, 32 and 38 more than the compared baselines, with ablations attributing gains to both ingredient-search components.

Significance. If the empirical claims held, the framework would be a meaningful step in LLM-based APR: it is among the first to combine internal and external repair ingredients in an agentic loop, it ships an open-source implementation, and its ablation design separates the contributions of dependency-based internal search and root-cause-aware external retrieval. The RWB experiments are a commendable attempt to address LLM pretraining leakage. However, the current evidence is not yet at the level of the headline claims: the retrieval-corpus leakage filter is under-specified, the baseline comparisons are not controlled, and the stochasticity of a single temperature-1 run is unquantified. These issues are fixable, and the framework's design is sound enough to warrant a major revision rather than rejection.

major comments (4)
  1. [Section 4.4 and Section 6] The data-leakage filter for the external retrieval corpus is load-bearing and under-specified. The paper states only 'We use the exact match strategy to filter out overlapping samples with those in benchmarks to avoid data leakage,' without defining the comparison unit (full file hash, normalized line set, hunk-level diff, or function body) or reporting how many TRANSFER entries were removed per benchmark. TRANSFER is built from real bug-fix commits and includes Google Closure, the same project family as many Defects4J Closure bugs; a coarse exact-match filter could leave near-duplicate hunks in the corpus. The concern is concrete: the retrieved pattern for Closure-51 shown in Figure 6 is a structurally similar negative-zero guard, and the paper does not demonstrate that this entry is not a near-duplicate of the target fix. If such near-duplicates are retrieved, the external-ingredient tool effectively reveals the answer, making the 32/38-bug margins circular. Section 6's data-leakage discussion addresses LLM training data only and does not cover retrieval-corpus leakage. Please define the filter precisely, report overlap statistics, and provide an audit of near-duplicates among the retrieved patterns for the successfully fixed bugs.
  2. [Section 4.3 and Table 3] The headline comparison reuses results reported by prior papers without controlling base model version, prompt template, fault-localization assumptions, or patch sampling budget. The sampling-times row in Table 3 shows, for example, ChatRepair at 500 samples, RepairAgent at 117, and ReinFix at at most 45; model versions also differ across papers, and all baselines predate the gpt-4o version used for ReinFixGPT4o. A claim that ReinFix 'fixes 32 more bugs than the baselines' is therefore not a controlled comparison of repair methods; it could reflect differences in model capability or experimental protocol. I request a controlled rerun of the strongest baselines (at least ChatRepair and ThinkRepair) under identical model, prompt, FL assumptions, and patch budget, or a clear restriction of RQ1 to reported re-implementations with the uncontrolled nature stated.
  3. [Section 5.1 and Table 4] The Defects4J V2.0 superiority claim is an apples-to-oranges comparison. The paper states that ThinkRepair is 'specifically designed for single-function repairs' and Table 4 reports its V2.0 count as 107 single-function fixes, while the ReinFixGPT4o total of 145 includes multi-function fixes. The fair scenario-level comparison shows a much smaller margin: 109 vs 107 single-function fixes for GPT-3.5-based ReinFix, and 47 vs 48 single-line fixes against ChatRepair. The claim of '38 more bugs than the SOTA' is thus an artifact of comparing total fixes against a single-function baseline. Please report head-to-head results within each repair scenario and qualify the headline accordingly.
  4. [Section 4.5 and Table 3] All ReinFix results are reported from a single execution at sampling temperature 1 (Section 4.5). LLM-based patch generation is stochastic, and with a maximum patch space of 45 candidates per bug the 32-bug and 38-bug margins could plausibly fall within run-to-run variability. The paper should report variance over repeated runs (or a statistically meaningful sample of bugs re-run multiple times) and should state whether the reported counts are from one trajectory or aggregated. Without this, the precision of the headline numbers is not established.
minor comments (5)
  1. [Section 5.1] The text refers to 'the ChatGPT-based APR tool ThinkRepair [49]' and later 'ThinkRepair [49]' in the Defects4J V2.0 paragraph; [49] is ChatRepair, and the ThinkRepair citation should be [54].
  2. [Section 3.2 and Figure 5] Figure 5 states 'ReinFix Setting: Top n = 1', but the text and Equation (4) refer to a threshold and top-n selection without reporting the similarity threshold used in the experiments; please report the threshold and confirm the top-n value.
  3. [Section 4.4] The random 100K subsample of TRANSFER lacks a random seed and a corpus version identifier; please add these reproducibility details.
  4. [Table 6] The asterisk on 'ThinkRepair*' in Table 6 is not defined in the caption or the surrounding text.
  5. [Section 4.2, Table 2] The table header contains a typo ('Benhmarks') and the '#Bug Num' column is not consistently formatted across the two benchmark halves.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim rests on external benchmark outcomes and component ablations, not on a derivation that equates inputs to outputs.

full rationale

ReinFix is an empirical LLM-agent APR system; its claims are benchmark outcomes, not derivations from fitted parameters or first-principles equations. The internal ingredient search uses Joern CPG queries (Section 3.1), and the external search uses embedding cosine similarity over TRANSFER bug-fix pairs (Section 3.2); both are search procedures whose outputs are validated by executing the test suite, and correctness is judged against ground-truth patches. The ablation study in Table 5 decomposes the gains by component, and the RWB evaluation in Section 5.4 provides an external, post-training-cutoff benchmark that is independent of the Defects4J headline numbers. The under-specified 'exact match' filter in Section 4.4 is a data-leakage validity concern rather than demonstrated circularity: the paper does not show, and the text does not entail, that the retrieved repair patterns are the target bugs' own ground-truth fixes by construction. The paper's self-citations (e.g., [10,11,12]) are related-work or prior technical context, not load-bearing justification for the central claim. Therefore no circular step is identifiable from the text.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central evaluation depends on five domain assumptions: accurate static analysis, retrieval relevance, LLM root-cause quality, perfect fault localization, and baseline comparability. There are no fitted physical constants or invented entities; the main numeric choices are experimental settings (top-n, patch budget, corpus subsample, temperature) rather than parameters fitted to maximize the headline result, though no sensitivity analysis is given.

free parameters (5)
  • top_n (retrieved repair patterns) = 1
    Section 3.2 and Figure 5 set Top n = 1 for external ingredient retrieval; no sensitivity analysis is reported.
  • Patch generation budget = 3 attempts x 3 suggestions x 5 patches = 45
    Section 4.5 fixes the maximum candidate patch space; the effect of larger or smaller budgets is not explored.
  • TRANSFER corpus subsample size = 100,000 (random sample)
    Section 4.4 randomly selects 100K samples from TRANSFER for the vector database; retrieval quality may depend on this subsample.
  • Sampling temperature = 1
    Section 4.5 sets temperature to 1 for stochastic generation; no repeated runs are used to characterize variance.
  • Retrieval similarity threshold = unspecified
    Equation (4) defines a threshold for retrieval, but no numeric value is given; the implementation appears to use top-n selection instead.
assumptions (5)
  • domain assumption Joern can build an accurate code property graph for all Defects4J Java projects, and the defined tools return relevant variable, method, class, and file information.
    Section 3.1 relies on this for internal ingredient search; no failure analysis is reported for projects where Joern parsing may be incomplete.
  • domain assumption Cosine similarity in embedding space between concatenated buggy code and root cause retrieves historically relevant repair patterns.
    Section 3.2, Equations (1)-(3) assume this; no retrieval precision evaluation is provided.
  • domain assumption LLM-generated root causes for both the corpus entries (GPT-4o) and the query (the agent) are accurate enough to make retrieval useful.
    Section 3.2 Data Preparation; root causes are generated by prompting an LLM, with no human validation or quality measurement.
  • domain assumption The buggy function (fault location) is known in advance.
    Section 4.5: 'we follow recent works under conditions of perfect fault localization'; real-world deployment would require a separate fault localization step.
  • domain assumption Reused baseline counts from prior papers are comparable to ReinFix's setup.
    Section 4.3 reuses reported results; this assumes compatible model versions, prompts, patch budgets, and benchmark definitions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Repair Ingredients Are All You Need: Improving Large Language Model-Based Program Repair via Repair Ingredients Search." pith.science (2026). https://pith.science/paper/IZFPW32P

@misc{pith2026250623100,
  author       = {Pith},
  title        = {Pith review of: Repair Ingredients Are All You Need: Improving Large Language Model-Based Program Repair via Repair Ingredients Search},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IZFPW32P}},
  note         = {Machine review of arXiv:2506.23100}
}
read the original abstract

Automated Program Repair (APR) techniques aim to automatically fix buggy programs. Among these, Large Language Model-based (LLM-based) approaches have shown great promise. Recent advances demonstrate that directly leveraging LLMs can achieve leading results. However, these techniques remain suboptimal in generating contextually relevant and accurate patches, as they often overlook repair ingredients crucial for practical program repair. In this paper, we propose ReinFix, a novel framework that enables LLMs to autonomously search for repair ingredients throughout both the reasoning and solution phases of bug fixing. In the reasoning phase, ReinFix integrates static analysis tools to retrieve internal ingredients, such as variable definitions, to assist the LLM in root cause analysis when it encounters difficulty understanding the context. During the solution phase, when the LLM lacks experience in fixing specific bugs, ReinFix searches for external ingredients from historical bug fixes with similar bug patterns, leveraging both the buggy code and its root cause to guide the LLM in identifying appropriate repair actions, thereby increasing the likelihood of generating correct patches. Evaluations on two popular benchmarks (Defects4J V1.2 and V2.0) demonstrate the effectiveness of our approach over SOTA baselines. Notably, ReinFix fixes 146 bugs, which is 32 more than the baselines on Defects4J V1.2. On Defects4J V2.0, ReinFix fixes 38 more bugs than the SOTA. Importantly, when evaluating on the recent benchmarks that are free of data leakage risk, ReinFix also maintains the best performance.

Figures

Figures reproduced from arXiv: 2506.23100 by the authors.

Figure 1
Figure 1. A motivation example of Closure-14. knowledge of similar fixes (i,e., internal and external repair ingre￾dients) independently to generate effective patches. In this section, we explore how these repair ingredients are critical to debugging and repair, using examples to show how missing ingredients can hinder effective patch generation. 2.1 Example 1: The Role of Internal Ingredients [PITH_FULL_IMAGE:figures/full_f… view at source ↗
Figure 3
Figure 3. The workflow of ReinFix [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Searched class-level dependencies for Closure-14. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (5 more)
Figure 6
Figure 6. Figure 6: Retrieved repair pattern for Closure-51. • Ranking and Selection. The top 𝑛 results are retrieved based on similarity scores. For each retrieved entry, the corresponding buggy code, fix code, and root cause are returned to the LLM along with the similarity score. These…
Figure 7
Figure 7. Figure 7: An execution chain of the ReAct Agent. patches, while those verified by humans and deemed semantically equivalent to the ground truth are classified as correct patches. 4 EXPERIMENT SETUP 4.1 Research Questions • RQ1: How does the repair effectiveness of ReinFix com￾pa…
Figure 8
Figure 8. Figure 8: Bug fix Venn diagram on Defects4J V1.2. Unique Fixes. Specifically, we also present ReinFix’s unique repair capabilities compared to other APR tools. Specifically, we fol￾low the practice of selecting Defects4J V1.2 in the baseline work [49] to present unique fixes and…
Figure 9
Figure 9. Figure 9: The human patch of Closure-102. private void removeDuplicateDeclarations(Node root) { /*<===com.google.javascript.jscomp.Normalize.removeDuplicateDeclarations:void(com.g oogle.javascript.rhino.Node) */ Callback tickler = new ScopeTicklingCallback(); ScopeCreator scopeC…
Figure 10
Figure 10. Figure 10: The Repair process of ReinFix𝐷𝑃 and ReinFix𝐹 𝑃 for Closure-102 [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. From Historical Patches to Repair Plans: Outcome-Conditioned Reasoning for Repository-Level Program Repair

    cs.SE 2026-01 conditional novelty 6.0 of 10

    Backward reasoning distilled from a repository's already-fixed issues, injected as prompts, lifts LLM repair success on SWE-Bench Lite by up to 10.4 Pass@1 points.

Reference graph

Works this paper leans on

63 extracted references · 51 canonical work pages · cited by 1 Pith paper

  1. [1]

    DeepSeek AI. 2023. DeepSeek Coder: Let the Code Write Itself. https://github. com/deepseek-ai/DeepSeek-Coder

    work page 2023

  2. [2]

    Islem Bouzenia, Premkumar Devanbu, and Michael Pradel. 2024. RepairA- gent: An Autonomous, LLM-Based Agent for Program Repair. arXiv preprint arXiv:2403.17134 (2024)

    arXiv 2024

  3. [3]

    Islem Bouzenia and Michael Pradel. 2025. Understanding Software Engineer- ing Agents: A Study of Thought-Action-Result Trajectories. arXiv preprint arXiv:2506.18824 (2025)

    arXiv 2025

  4. [4]

    Zimin Chen, Steve Kommrusch, Michele Tufano, Louis-Noël Pouchet, Denys Poshyvanyk, and Martin Monperrus. 2019. SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair. IEEE Transactions on Software Engi- neering (TSE) 47, 9 (2019), 1943–1959

    work page 2019

  5. [5]

    Michael Fu, Chakkrit Tantithamthavorn, Trung Le, Van Nguyen, and Dinh Phung

  6. [6]

    Luca Gazzola, Daniela Micucci, and Leonardo Mariani. 2019. Automatic Software Repair: A Survey. IEEE Transactions on Software Engineering (TSE) 45, 01 (2019), 34–67

    work page 2019

  7. [7]

    Junda He, Christoph Treude, and David Lo. 2025. LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision, and the Road Ahead. ACM Transactions on Software Engineering and Methodology (TOSEM) 34, 5 (2025), 1–30

    work page 2025

  8. [8]

    Kai Huang, Xiangxin Meng, Jian Zhang, Yang Liu, Wenjie Wang, Shuhao Li, and Yuqing Zhang. 2023. An Empirical Study on Fine-Tuning Large Language Models of Code for Automated Program Repair. In 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) . 1162–1174

    work page 2023

Show all 63 references
  1. [9]

    Kai Huang, Zhengzi Xu, Su Yang, Hongyu Sun, Xuejun Li, Zheng Yan, and Yuqing Zhang. 2024. Evolving Paradigms in Automated Program Repair: Taxonomy, Challenges, and Opportunities. ACM Computing Surveys (CSUR) 57, 2 (2024), 1–43

    2024

  2. [10]

    Kai Huang, Jian Zhang, Xinlei Bao, Xu Wang, and Yang Liu. 2025. Comprehensive Fine-Tuning Large Language Models of Code for Automated Program Repair. IEEE Transactions on Software Engineering (TSE) 51, 4 (2025), 904–928

    2025

  3. [11]

    Kai Huang, Jian Zhang, Xiangxin Meng, and Yang Liu. 2025. Template-Guided Program Repair in the Era of Large Language Models. In 47th International Conference on Software Engineering (ICSE) . 367–379

    2025

  4. [12]

    Kai Huang, Jian Zhang, Xiaofei Xie, and Chunyang Chen. 2025. Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Fixing. arXiv preprint arXiv:2506.16136 (2025)

    2025 arXiv

  5. [13]

    Nan Jiang, Kevin Liu, Thibaud Lutellier, and Lin Tan. 2023. Impact of Code Language Models on Automated Program Repair. In45th IEEE/ACM International Conference on Software Engineering (ICSE) . 1430–1442. ICSE 2026, 12–18 April, 2026, Rio de Janeiro, Brazil Jiayi Zhang, Kai Hu...

    2023

  6. [14]

    Nan Jiang, Thibaud Lutellier, Yiling Lou, Lin Tan, Dan Goldwasser, and Xiangyu Zhang. 2023. KNOD: Domain Knowledge Distilled Tree Decoder for Automated Program Repair. In 45th International Conference on Software Engineering (ICSE) . 1251–1263

    2023

  7. [15]

    Nan Jiang, Thibaud Lutellier, and Lin Tan. 2021. CURE: Code-Aware Neural Ma- chine Translation for Automatic Program Repair. In43rd International Conference on Software Engineering (ICSE) . 1161–1173

    2021

  8. [16]

    joern.io. 2024. Joern: The Bug Hunter’s Workbench . https://github.com/joernio/ joern

    2024

  9. [17]

    René Just, Darioush Jalali, and Michael D Ernst. 2014. Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs. In 2014 International Symposium on Software Testing and Analysis (ISSTA) . 437–440

    2014

  10. [18]

    Dongsun Kim, Jaechang Nam, Jaewoo Song, and Sunghun Kim. 2013. Automatic Patch Generation Learned from Human-Written Patches. In 35th International Conference on Software Engineering (ICSE) . 802–811

    2013

  11. [19]

    Jiaolong Kong, Mingfei Cheng, Xiaofei Xie, Shangqing Liu, Xiaoning Du, and Qi Guo. 2024. Contrastrepair: Enhancing conversation-based automated program repair via contrastive test case pairs. arXiv preprint arXiv:2403.01971 (2024)

    2024

  12. [20]

    LangChain. 2024. Applications that can reason. Powered by LangChain. https: //www.langchain.com/

    2024

  13. [21]

    VI Lcvenshtcin. 1966. Binary coors capable or ‘correcting deletions, insertions, and reversals. In Soviet Physics-Doklady, Vol. 10

    1966

  14. [22]

    Claire Le Goues, ThanhVu Nguyen, Stephanie Forrest, and Westley Weimer. 2012. GenProg: A Generic Method for Automatic Software Repair. IEEE Transactions on Software Engineering (TSE) 38, 01 (2012), 54–72

    2012

  15. [23]

    Claire Le Goues, Michael Pradel, and Abhik Roychoudhury. 2019. Automated Program Repair. Communications of the ACM (CACM) 62, 12 (2019), 56–65

    2019

  16. [24]

    Cheryl Lee, Chunqiu Steven Xia, Jen-tse Huang, Zhouruixin Zhu, Lingming Zhang, and Michael R Lyu. 2024. A Unified Debugging Approach via LLM-Based Multi-Agent Synergy. arXiv preprint arXiv:2404.17153 (2024)

    2024

  17. [25]

    Yi Li, Shaohua Wang, and Tien N Nguyen. 2020. DLFix: Context-based Code Transformation Learning for Automated Program Repair. In 42nd International Conference on Software Engineering (ICSE) . 602–614

    2020

  18. [26]

    Yi Li, Shaohua Wang, and Tien N Nguyen. 2022. DEAR: A Novel Deep Learning- based Approach for Automated Program Repair. In 44th International Conference on Software Engineering (ICSE) . 511–523

    2022

  19. [27]

    Junwei Liu, Kaixin Wang, Yixuan Chen, Xin Peng, Zhenpeng Chen, Lingming Zhang, and Yiling Lou. 2024. Large Language Model-Based Agents for Software Engineering: A Survey. arXiv preprint arXiv:2409.02977 (2024)

    2024 arXiv

  20. [28]

    Kui Liu, Anil Koyuncu, Dongsun Kim, and Tegawendé F Bissyandé. 2019. TBar: Revisiting Template-based Automated Program Repair. In 28th International Symposium on Software Testing and Analysis (ISSTA) . 31–42

    2019

  21. [29]

    Thibaud Lutellier, Hung Viet Pham, Lawrence Pang, Yitong Li, Moshi Wei, and Lin Tan. 2020. CoCoNuT: Combining Context-Aware Neural Translation Models us- ing Ensemble for Program Repair. In29th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA). 101–114

    2020

  22. [30]

    Matias Martinez, Westley Weimer, and Martin Monperrus. 2014. Do the fix ingredients already exist? an empirical inquiry into the redundancy assump- tions of program repair approaches. In 36th International Conference on Software Engineering (ICSE). 492–495

    2014

  23. [31]

    Xiangxin Meng, Xu Wang, Hongyu Zhang, Hailong Sun, and Xudong Liu. 2022. Improving Fault Localization and Program Repair with Deep Semantic Features and Transferred Knowledge. In 44th International Conference on Software Engi- neering (ICSE). 1169–1180

    2022

  24. [32]

    Xiangxin Meng, Xu Wang, Hongyu Zhang, Hailong Sun, Xudong Liu, and Chun- ming Hu. 2023. Template-based Neural Program Repair. In 45th International Conference on Software Engineering (ICSE) . 1456–1468

    2023

  25. [33]

    Martin Monperrus. 2018. Automatic Software Repair: A Bibliography. ACM Computing Surveys (CSUR) 51, 1 (2018), 1–24

    2018

  26. [34]

    Hoang Duong Thien Nguyen, Dawei Qi, Abhik Roychoudhury, and Satish Chan- dra. 2013. Semfix: Program Repair via Semantic Analysis. In 35th International Conference on Software Engineering (ICSE) . 772–781

    2013

  27. [35]

    OpenAI. 2021. gpt-3.5-turbo-0125. https://platform.openai.com/docs/models# gpt-3-5-turbo

    2021

  28. [36]

    OpenAI. 2021. gpt-4-0613. https://platform.openai.com/docs/models#gpt-4- turbo-and-gpt-4

    2021

  29. [37]

    OpenAI. 2022. ChatGPT: Optimizing Language Models for Dialogue. https: //openai.com/blog/chatgpt/

    2022

  30. [38]

    OpenAI. 2023. gpt-4-1106-preview. https://platform.openai.com/docs/models# gpt-4-turbo-and-gpt-4

    2023

  31. [39]

    OpenAI. 2023. gpt-4o-2024-05-13. https://platform.openai.com/docs/models#gpt- 4-turbo-and-gpt-4

    2023

  32. [40]

    OpenAI. 2024. Vector Embeddings. https://platform.openai.com/docs/guides/ embeddings

    2024

  33. [41]

    Michele Tufano, Cody Watson, Gabriele Bavota, Massimiliano Di Penta, Martin White, and Denys Poshyvanyk. 2019. An Empirical Study on Learning Bug- Fixing Patches in the Wild via Neural Machine Translation. ACM Transactions on Software Engineering and Methodology (TOSEM) 28, 4 ...

    2019

  34. [42]

    Weishi Wang, Yue Wang, Shafiq Joty, and Steven CH Hoi. 2023. RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Re- pair. In 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE)...

    2023

  35. [43]

    Yuxiang Wei, Chunqiu Steven Xia, and Lingming Zhang. 2023. Copiloting the copilots: Fusing large language models with completion engines for automated program repair. In Proceedings of the 31st ACM Joint European Software Engi- neering Conference and Symposium on the Foundatio...

    2023

  36. [44]

    Yi Wu, Nan Jiang, Hung Viet Pham, Thibaud Lutellier, Jordan Davis, Lin Tan, Petr Babkin, and Sameena Shah. 2023. How Effective Are Neural Networks for Fixing Security Vulnerabilities. In 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA). 1282–1294

    2023

  37. [45]

    Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. 2025. De- mystifying LLM-Based Software Engineering Agents. Proceedings of the ACM on Software Engineering 2, FSE (2025), 801–824

    2025

  38. [46]

    Chunqiu Steven Xia, Yifeng Ding, and Lingming Zhang. 2023. The Plastic Surgery Hypothesis in the Era of Large Language Models. In38th International Conference on Automated Software Engineering (ASE) . 522–534

    2023

  39. [47]

    Chunqiu Steven Xia, Yuxiang Wei, and Lingming Zhang. 2023. Automated Program Repair in the Era of Large Pre-trained Language Models. In 45th Inter- national Conference on Software Engineering (ICSE) . 1482–1494

    2023

  40. [48]

    Chunqiu Steven Xia and Lingming Zhang. 2022. Less Training, More Repairing Please: Revisiting Automated Program Repair via Zero-Shot Learning. In 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) . 959–971

    2022

  41. [49]

    Chunqiu Steven Xia and Lingming Zhang. 2024. Automated Program Repair via Conversation: Fixing 162 out of 337 Bugs for $0.42 Each using ChatGPT. In 33rd International Symposium on Software Testing and Analysis (ISSTA) . 819–831

    2024

  42. [50]

    Jiahong Xiang, Xiaoyang Xu, Fanchu Kong, Mingyuan Wu, Haotian Zhang, and Yuqun Zhang. 2024. How Far Can We Go with Practical Function-Level Program Repair? arXiv preprint arXiv:2404.12833 (2024)

    2024 arXiv

  43. [51]

    Deheng Yang, Kui Liu, Dongsun Kim, Anil Koyuncu, Kisub Kim, Haoye Tian, Yan Lei, Xiaoguang Mao, Jacques Klein, and Tegawendé F Bissyandé. 2021. Where were the repair ingredients for defects4j bugs? exploring the impact of repair ingredient retrieval on the performance of 24 pr...

    2021

  44. [52]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. In International Conference on Learning Representations (ICLR)

    2023

  45. [53]

    He Ye, Matias Martinez, and Martin Monperrus. 2022. Neural Program Repair with Execution-based Backpropagation. In 44th International Conference on Software Engineering (ICSE). 1506–1518

    2022

  46. [54]

    Xin Yin, Chao Ni, Shaohua Wang, Zhenhao Li, Limin Zeng, and Xiaohu Yang

  47. [55]

    Wei Yuan, Quanjun Zhang, Tieke He, Chunrong Fang, Nguyen Quoc Viet Hung, Xiaodong Hao, and Hongzhi Yin. 2022. CIRCLE: Continual Repair Across Pro- gramming Languages. In 31st ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA). 678–690

    2022

  48. [56]

    Quanjun Zhang, Chunrong Fang, Yuxiang Ma, Weisong Sun, and Zhenyu Chen

  49. [57]

    Quanjun Zhang, Chunrong Fang, Yang Xie, Yuxiang Ma, Weisong Sun, Yun Yang, and Zhenyu Chen. 2024. A Systematic Literature Review on Large Language Models for Automated Program Repair. arXiv preprint arXiv:2405.01466 (2024)

    2024

  50. [58]

    Quanjun Zhang, Chunrong Fang, Tongke Zhang, Bowen Yu, Weisong Sun, and Zhenyu Chen. 2023. Gamma: Revisiting Template-Based Automated Program Re- pair via Mask Prediction. In 38th International Conference on Automated Software Engineering (ASE). 535–547

    2023

  51. [59]

    Qihao Zhu, Zeyu Sun, Yuanan Xiao, Wenjie Zhang, Kang Yuan, Yingfei Xiong, and Lu Zhang. 2021. A Syntax-Guided Edit Decoder for Neural Program Repair. In 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering ...

    2021

  52. [60]

    Qihao Zhu, Zeyu Sun, Wenjie Zhang, Yingfei Xiong, and Lu Zhang. 2023. Tare: Type-aware Neural Program Repair. In 45th International Conference on Software Engineering (ICSE). 1443–1455

    2023

  53. [2022]

    In 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE)

    VulRepair: a T5-based Automated Software Vulnerability Repair. In 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) . 935–947

  54. [2023]

    ACM Transactions on Software Engineering and Methodology (TOSEM) 33, 2 (2023), 1–69

    A Survey of Learning-based Automated Program Repair. ACM Transactions on Software Engineering and Methodology (TOSEM) 33, 2 (2023), 1–69

    2023

  55. [2024]

    In 33rd International Symposium on Software Testing and Analysis (ISSTA)

    Thinkrepair: Self-directed Automated Program Repair. In 33rd International Symposium on Software Testing and Analysis (ISSTA) . 1274–1286

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.