The Impact of Fine-tuning Large Language Models on Automated Program Repair

Anastasiia Grishina; Leon Moonen; Max Hort; Roman Mach\'a\v{c}ek

REVIEW 3 major objections 5 minor 2 cited by

The Impact of Fine-tuning Large Language Models on Automated Program Repair

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuning every weight of a code LLM often hurts its bug-fixing performance; freezing the model and training a tiny adapter fixes more bugs at under 1% of the training cost.

desk verdict A broad and useful empirical matrix, but the headline PEFT-over-FMFT advantage rests on an asymmetric checkpoint choice and misstated ratio-as-percentage claims. read the letter →

arxiv 2507.19909 v1 pith:WJXXGTNE submitted 2025-07-26 cs.SE cs.AIcs.CLcs.LG

Roman Macháček , Anastasiia Grishina , Max Hort , Leon Moonen This is my paper

classification cs.SEcs.AIcs.CLcs.LG

keywords automatedprogramrepairlargelanguagemodelsparameter-efficientfine-tuningLoRAIA3codeoverfittingAPRbenchmarks

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Automated program repair (APR) is increasingly done by adapting large pretrained code models to the bug-fixing task, and the standard adaptation is to fine-tune every weight. This paper argues that for several of these models full fine-tuning actually lowers the number of bugs they fix on three benchmark suites, because the fine-tuning data come from a different distribution than the benchmarks and the models overfit to the training corpus. The alternative the paper tests is parameter-efficient fine-tuning with LoRA and IA3 adapters, which keep the pretrained weights frozen and train only a tiny additional structure. In the reported experiments the adapters match or beat full fine-tuning in a substantial number of configurations, most clearly for CodeGen and DeepSeekCoder, while training under 1% of the parameters. If the finding holds, APR teams can improve repair performance while cutting fine-tuning cost by orders of magnitude.

What carries the argument

The load-bearing object is the adapter. LoRA freezes the pretrained weight matrix $W_0$ and learns a low-rank update $\Delta W = BA$, so only the entries of the small matrices $B$ and $A$ are trainable; IA3 freezes the base model and trains three scaling vectors per layer that re-scale the attention keys, values, and feed-forward activations. Both mechanisms restrict the hypothesis space during fine-tuning. The paper argues that this restriction is exactly what preserves the pretrained model's repair ability: adapter-trained models show near-equal training and validation metrics, indicating less overfitting to the fine-tuning corpus, while full fine-tuning moves all weights toward the distribution of that corpus and away from the benchmark distributions.

What would settle it

Re-run the same three benchmarks scoring full fine-tuning at epoch 1 rather than epoch 3 for CodeT5-small and CodeT5-base, and compare with LoRA and IA3 at epoch 1; if the fully fine-tuned models then match or exceed the adapter models on HumanEval-Java or Defects4J, the paper's central claim would not hold for those models.

Watch

Extended reading notes

Core claim

The paper's central claim is that full-model fine-tuning decreases the benchmarking performance of several code LLMs on APR, because the fine-tuning corpus and the benchmarks have different bug distributions and the models overfit, whereas parameter-efficient fine-tuning with LoRA or IA3 restricts the number of trainable parameters and achieves better results. The evidence is a systematic comparison on QuixBugs, HumanEval-Java, and Defects4J across six model families. In the reported configurations, CodeGen and DeepSeekCoder improved substantially with LoRA relative to full fine-tuning: for CodeGen-2B, LoRA used only 0.09% of the model's parameters while achieving improvements of 172%, 225%, and 153% over full fine-tuning on QuixBugs, HumanEval-Java, and Defects4J respectively. LoRA also beat IA3 in 21 of 24 comparisons. Models that were already strong zero-shot, notably DeepSeekCoder, lost performance after full fine-tuning, while weaker models such as CodeT5 gained from it.

Load-bearing premise

The headline comparison assumes that scoring full fine-tuning at epoch 3 and adapter tuning at epoch 1 is fair, because the adapter metrics plateau after the first epoch; if full fine-tuning were scored at its best epoch, the reported advantage for adapters would shrink for several models.

Editorial extensions

If this is right

APR systems can default to LoRA-style adapters for code LLMs, since most reported configurations show equal or better plausible-patch counts with far fewer trainable parameters.
Strong zero-shot models such as DeepSeekCoder should not be fully fine-tuned on bug-fix corpora; the paper's data show they lose benchmark performance, while adapters preserve or improve it.
Models too large for full fine-tuning on a single GPU node, around 6B parameters in this study, become trainable with adapters, widening the range of models small teams can use.
LoRA hyperparameter search is largely unnecessary for APR under the tested ranges, since rank and scaling factor had negligible effect on CodeBLEU for CodeGen-2B.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The headline comparison in Table V reports full fine-tuning at epoch 3 and adapter tuning at epoch 1; if full fine-tuning were scored at its best epoch, the reported advantage for adapters would shrink for several models, especially CodeT5-small and CodeT5-base.
The paper itself flags data leakage as a concern; if leakage inflates zero-shot scores for the newer models, the observed 'full fine-tuning hurts' pattern partly reflects the model forgetting leaked benchmark solutions rather than losing general repair competence.
Everything here is Java and single-hunk bugs; an obvious extension is to test whether the adapter advantage persists on multi-hunk, multiline, or non-Java repair tasks before generalizing the recommendation.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read the paper for its breadth, not for its headline. The claim that PEFT beats full fine-tuning for APR is real but overstated, because the comparison scores FMFT at epoch 3 and PEFT at epoch 1. That asymmetry flatters PEFT, and the paper compounds it by reporting ratios as percentage gains.

What's actually new is the scale: 15 models across QuixBugs, HumanEval-Java, and Defects4J, each under base, FMFT, LoRA, and IA3, plus a LoRA rank/alpha sweep. The replication package is a genuine asset. They also position the work honestly against RepairLLaMA and Li et al., noting the conceptual possibility of adapters was already raised, and they compare where possible. The Defects4J baselines and the RQ2 table of epoch-by-epoch validation metrics have independent value.

The soft spot is load-bearing. Table V puts FMFT at epoch 3 and LoRA/IA3 at epoch 1. The text says PEFT validation metrics plateau after epoch 1, but validation plateau is not benchmark performance. Table III shows FMFT epoch-1 counts beating epoch-3 counts in several cells (CodeT5-small HumanEval-Java 44 vs 39; CodeGen-350M Defects4J 67 vs 61), so epoch 3 is not FMFT's best checkpoint. PEFT results at epoch 3 are not reported, so we cannot see whether the epoch-1 plateau matches the best benchmark checkpoint. The honest fix is to score both at their best epoch, or report PEFT at epoch 3, and see whether the PEFT advantage survives.

Second, the RQ3 summary says CodeGen-2B LoRA gives 'performance gains of 172%, 225%, 153%'. Those are ratios (19/11, 81/36, 98/64); the gains are 72%, 125%, 53%. That misstates the effect.

There are secondary issues: no repeated runs, no statistical testing, and the PEFT models were selected based on performance for RQ3. Those are disclosed and fixable, but the checkpoint asymmetry is the one that changes the conclusion.

Who this is for: practitioners choosing between full fine-tuning and adapters for APR, and empirical researchers studying PEFT for code. They will get useful numbers and a solid baseline. The weaker claim—that PEFT is competitive, sometimes better, at under 1% of trainable parameters—is supported by the data; the stronger claim that FMFT 'decreases' performance is not established at matched checkpoints.

I'd send this to a serious referee. The breadth, the replication package, and the explicit comparisons to prior work justify referee time. The referee should require the epoch-asymmetry fix and corrected percentage language before acceptance.

Referee Report

3 major / 5 minor

Summary. This paper reports an empirical study of fine-tuning strategies for LLM-based automated program repair. Six code LLM families (15 checkpoints) are evaluated on QuixBugs, HumanEval-Java, and Defects4J under three regimens: no fine-tuning, full-model fine-tuning (FMFT), and parameter-efficient fine-tuning (PEFT) with LoRA and IA3. The authors find that zero-shot performance varies widely, that full fine-tuning helps smaller/weaker models but hurts stronger models on some benchmarks, and that PEFT often matches or beats full fine-tuning while updating under 1% of parameters. A fourth research question examines LoRA rank and scaling-factor sensitivity. The paper concludes that full fine-tuning decreases benchmarking performance and that PEFT achieves better results, and it provides a replication package.

Significance. If the headline comparison is fair, the paper makes a practically important contribution: adapter-based fine-tuning at <1% trainable parameters can match or exceed full fine-tuning on APR benchmarks, with clear resource implications. The study is broad (15 checkpoints, 3 benchmarks, two adapters, hyperparameter sweeps) and the replication package is a concrete strength. However, the central PEFT-vs-FMFT claim rests on an asymmetric checkpoint choice, and the abstract-level claim is stronger than the tables support for several model families. These issues are addressable, but they require substantive revision rather than copy-editing.

major comments (3)

[Section IV-C, Table V] The headline FMFT-versus-PEFT comparison is asymmetric: FMFT is scored at epoch 3, while LoRA and IA3 are scored at epoch 1. The text justifies epoch 1 for PEFT by citing validation-plateau behavior (Table IV), but benchmark performance is not necessarily aligned with validation metrics. Table III shows several FMFT cells where epoch 1 beats epoch 3 (e.g., CodeT5-small HumanEval-Java 44 vs 39, CodeGen-350M Defects4J 67 vs 61, CodeGen-2B Defects4J 66 vs 64), so epoch 3 is not clearly the best FMFT checkpoint. Conversely, Table IV reports no benchmark results for PEFT at epochs 2 or 3, so one cannot verify that the epoch-1 validation plateau corresponds to the best benchmark checkpoint. This asymmetry could shrink or even reverse the reported PEFT advantage in several cells. Please report both methods at matched epochs, or at each method's best benchmark checkpoint, and re-evaluate the RQ3 conclusions accordingly.
[Section IV-C, RQ3 summary] The summary states that LoRA for CodeGen-2B achieves 'performance gains of 172%, 225%, 153%' on QuixBugs, HumanEval-Java, and Defects4J. These numbers are ratios of plausible-patch counts (19/11=1.73, 81/36=2.25, 98/64=1.53), not percentage gains; a 172% gain would mean 2.72 times the baseline count. The correct phrasing is that the counts are 1.73x, 2.25x, and 1.53x the FMFT counts, or equivalently 73%, 125%, and 53% higher. This numerical misreport is part of the abstract-level claim and should be corrected.
[Abstract and Section V] The abstract's blanket statement that 'full fine-tuning techniques decrease the benchmarking performance of various models' is not supported by the paper's own results in full generality. Table III shows that full fine-tuning substantially improves CodeT5, Bloom, and CodeGen-350M relative to their base models, while the deterioration is concentrated in DeepSeekCoder and StarCoder. Similarly, Table V shows that PEFT does not uniformly beat FMFT: e.g., CodeT5-small on Defects4J has FMFT=60 vs LoRA=50, and CodeT5-base on Defects4J has FMFT=75 vs LoRA=67. Moreover, FMFT results are marked X for CodeGen-6B and DeepSeekCoder-6.7B, so the PEFT-vs-FMFT claim is only tested on models up to 3B parameters. Please qualify the conclusions to state which models and configurations support the PEFT advantage, and restrict claims about the 6B models accordingly.

minor comments (5)

[Section IV-D] The text says 'we investigate the following 8 values for both rank and scaling factor' but then lists seven values: 1, 2, 4, 8, 16, 32, 64; Figures 3-6 also show seven values on each axis. Please correct the count or add the missing value.
[Table II] In the StarCoder-3b row, the Exact Match value for validation epoch 2 is printed as '.019'; this is presumably 0.19 and should be formatted consistently with the other entries.
[Table IV] The caption states that the best results on the validation dataset are highlighted in bold, but no bold highlighting is visible in the typeset table, making the best values difficult to identify.
[Section IV-C, Table V] The caption marks FMFT for CodeGen-6B and DeepSeekCoder-6.7B as X due to hardware limitations; it would help to add a sentence in the main text noting that PEFT results for these two 6B models are only compared against the base model, not against FMFT.
[Table I] The text explaining discrepancies with Jiang et al. mentions that Jiang et al. did not specify the Java version and filtered out more programs by length; please add the exact Defects4J version and filtering criteria used in that reference, since some Defects4J cells differ substantially from the values in parentheses.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the PEFT versus full fine-tuning comparison is an empirical measurement on held-out APR benchmarks, not a derivation from fitted inputs.

full rationale

This paper is an empirical benchmarking study, not a derived prediction chain. The central claim that parameter-efficient fine-tuning (LoRA/IA3) achieves better results than full fine-tuning rests on Table V, where the number of plausible patches is directly measured on the held-out QuixBugs, HumanEval-Java, and Defects4J benchmarks. No parameter is fitted to those benchmarks and then reported as a prediction: the LoRA/IA3 epoch-1 checkpoint is selected from validation metrics in Table IV, and the benchmark results are separate measurements. The comparison is asymmetric (FMFT is scored at epoch 3 while PEFT is scored at epoch 1), and the RQ3 summary's '172%, 225%, 153%' phrasing describes ratios rather than percentage gains; these are validity and presentation concerns, not circularity, because the PEFT benchmark counts are not equivalent by construction to the validation plateau or to any fitted input. The only self-citation, reference [23], points to the replication package and is not load-bearing. No self-definitional, fitted-input-as-prediction, self-citation-chain, uniqueness-import, ansatz-smuggled, or renaming pattern is present. The paper is self-contained against external benchmarks, so the circularity score is 0.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

No invented entities or data-fitted constants enter the study; the ledger lists protocol choices and domain assumptions that shape all reported comparisons.

free parameters (4)

LoRA rank = r=8
Hugging Face default used for all LoRA runs; RQ4 shows limited sensitivity, but the headline comparison is anchored to this value.
LoRA scaling factor = alpha=16
Default scaling factor; RQ4 sweeps it but main tables use 16.
Fine-tuning epoch count = FMFT: 3, PEFT: 1
Asymmetric epoch count selected post hoc for PEFT; affects the central FMFT versus PEFT comparison.
Patch samples per bug = 10
Each model generates 10 candidate patches per bug; plausibility is counted if any patch passes. Sampling noise is not quantified.

assumptions (3)

domain assumption Plausibility approximates correctness
A patch that passes the provided tests is counted as fixed; the paper flags this in Threats to Validity (Section IV-E).
domain assumption APR benchmarks are not already memorized by the models
The paper relies on HumanEval-Java being a novel conversion to avoid contamination, but acknowledges data leakage risk for GitHub-sourced pretraining corpora (Section IV-E).
domain assumption CLM training data is adequate for fine-tuning
The CLM dataset of single-hunk Java bug-fix pairs is reused from Zhu et al.; the paper notes distribution shift (Sections III-A and IV-B).

how reviews work

0 comments

Cite this review

Pith. "Pith review of The Impact of Fine-tuning Large Language Models on Automated Program Repair." pith.science (2026). https://pith.science/paper/WJXXGTNE

@misc{pith2026250719909,
  author       = {Pith},
  title        = {Pith review of: The Impact of Fine-tuning Large Language Models on Automated Program Repair},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WJXXGTNE}},
  note         = {Machine review of arXiv:2507.19909}
}

read the original abstract

Automated Program Repair (APR) uses various tools and techniques to help developers achieve functional and error-free code faster. In recent years, Large Language Models (LLMs) have gained popularity as components in APR tool chains because of their performance and flexibility. However, training such models requires a significant amount of resources. Fine-tuning techniques have been developed to adapt pre-trained LLMs to specific tasks, such as APR, and enhance their performance at far lower computational costs than training from scratch. In this study, we empirically investigate the impact of various fine-tuning techniques on the performance of LLMs used for APR. Our experiments provide insights into the performance of a selection of state-of-the-art LLMs pre-trained on code. The evaluation is done on three popular APR benchmarks (i.e., QuixBugs, Defects4J and HumanEval-Java) and considers six different LLMs with varying parameter sizes (resp. CodeGen, CodeT5, StarCoder, DeepSeekCoder, Bloom, and CodeLlama-2). We consider three training regimens: no fine-tuning, full fine-tuning, and parameter-efficient fine-tuning (PEFT) using LoRA and IA3. We observe that full fine-tuning techniques decrease the benchmarking performance of various models due to different data distributions and overfitting. By using parameter-efficient fine-tuning methods, we restrict models in the amount of trainable parameters and achieve better results. Keywords: large language models, automated program repair, parameter-efficient fine-tuning, AI4Code, AI4SE, ML4SE.

Figures

Figures reproduced from arXiv: 2507.19909 by the authors.

**Figure 2.** Illustration of IA3 adapter [22]. Similarly as in LoRA, we can have multiple IA3 adapters on top of a single pre-trained model, each one trained and serving a different task. Because the base pre-trained model can be merged into the IA3 adapter, there is no overhead during inference. According to experiments by Liu et al. [22], IA3 outperformed full fine-tuning and performs better than LoRA while using fewer trainab… view at source ↗

**Figure 3.** CodeBLEU change with scaling factor for three epochs. [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

**Figure 4.** Exact match change with scaling factor for three epochs. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: CodeBLEU change with rank for three epochs. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

**Figure 6.** Figure 6: Exact match change with rank for three epochs. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

BLAgent: Agentic RAG for File-Level Bug Localization
cs.SE 2026-05 unverdicted novelty 6.0 of 10

BLAgent achieves over 78% Top-1 accuracy on SWE-bench Lite for file-level bug localization using agentic RAG, at 18x lower cost than baselines, and boosts end-to-end APR success by over 20%.
BLAgent: Agentic RAG for File-Level Bug Localization
cs.SE 2026-05 unverdicted novelty 6.0 of 10

BLAgent achieves over 78% top-1 file-level bug localization accuracy on SWE-bench-Lite with open-source models and over 86% with closed-source models while being over 18x cheaper than the strongest baseline.

Reference graph

Works this paper leans on

72 extracted references · 32 canonical work pages · cited by 1 Pith paper

[1]

On the Automation of Fixing Software Bugs

A. Arcuri. “On the Automation of Fixing Software Bugs.” In: Com- panion of the 30th International Conference on Software Engineering . Association for Computing Machinery, 2008, pp. 1003–1006. DOI: 10.1145/1370175.1370223

work page arXiv 2008
[2]

Huang, Z

K. Huang, Z. Xu, S. Yang, H. Sun, X. Li, Z. Yan, and Y . Zhang. A Survey on Automated Program Repair Techniques . 2023. DOI: 10. 48550/arXiv.2303.18184. arXiv: 2303.18184 [cs]

arXiv doi:10.48550/arxiv.2303.18184 2023
[3]

Getafix: Learning to Fix Bugs Automatically

J. Bader, A. Scott, M. Pradel, and S. Chandra. “Getafix: Learning to Fix Bugs Automatically.” In: Proc. ACM Program. Lang. 3.OOPSLA (2019). DOI: 10.1145/3360585

doi:10.1145/3360585 2019
[4]

DeepDebug: Fixing Python Bugs Using Stack Traces, Backtranslation, and Code Skeletons

D. Drain, C. B. Clement, G. Serrato, and N. Sundaresan. DeepDebug: Fixing Python Bugs Using Stack Traces, Backtranslation, and Code Skeletons. 2021. DOI: 10.48550/arXiv.2105.09352. arXiv: 2105.09352 [cs]

work page Pith review arXiv doi:10.48550/arxiv.2105.09352 2021
[5]

PyDex: Repairing Bugs in Introductory Python Assignments Using LLMs

J. Zhang, J. P. Cambronero, S. Gulwani, V . Le, R. Piskac, G. Soares, and G. Verbruggen. “PyDex: Repairing Bugs in Introductory Python Assignments Using LLMs.” In: Proc. ACM Program. Lang. 8.OOP- SLA1 (2024), 133:1100–133:1124. DOI: 10.1145/3649850

doi:10.1145/3649850 2024
[6]

Mining Fix Patterns for FindBugs Violations

K. Liu, D. Kim, T. F. Bissyande, S. Yoo, and Y . L. Traon. “Mining Fix Patterns for FindBugs Violations.” In: IEEE Transactions on Software Engineering 47.1 (2021), pp. 165–188. DOI: 10 . 1109 / TSE . 2018 . 2884955

work page 2021
[7]

A V ATAR: Fixing Semantic Bugs with Fix Patterns of Static Analysis Violations

K. Liu, A. Koyuncu, D. Kim, and T. F. Bissyande. “A V ATAR: Fixing Semantic Bugs with Fix Patterns of Static Analysis Violations.” In: 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE Computer Society, 2019, pp. 1–12. DOI: 10.1109/SANER.2019.8667970

work page arXiv 2019
[9]

A Survey of Learning-Based Automated Program Repair

Q. Zhang, C. Fang, Y . Ma, W. Sun, and Z. Chen. “A Survey of Learning-Based Automated Program Repair.” In: ACM Trans. Softw. Eng. Methodol. 33.2 (2023). DOI: 10.1145/3631974. 11

doi:10.1145/3631974 2023

Show all 72 references

[11]

Kaddour, J

J. Kaddour, J. Harris, M. Mozes, H. Bradley, R. Raileanu, and R. McHardy. Challenges and Applications of Large Language Models
[12]

L. Xu, H. Xie, S.-Z. J. Qin, X. Tao, and F. L. Wang. Parameter- Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment . 2023. DOI: 10.48550/arXiv.2312. 12148. arXiv: 2312.12148 [cs]

2023 doi arXiv
[13]

CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis

E. Nijkamp, B. Pang, H. Hayashi, L. Tu, H. Wang, Y . Zhou, S. Savarese, and C. Xiong. “CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis.” In: The Eleventh International Conference on Learning Representations . 2023

2023
[14]

CodeT5: Identifier-Aware Unified Pre-Trained Encoder-Decoder Models for Code Understanding and Generation

Y . Wang, W. Wang, S. Joty, and S. C. Hoi. “CodeT5: Identifier-Aware Unified Pre-Trained Encoder-Decoder Models for Code Understanding and Generation.” In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing . Ed. by M.-F. Moens, X. Huang, L....

2021
[15]

BLOOM: A 176B-parameter Open-Access Multi- lingual Language Model

T. L. Scao et al. “BLOOM: A 176B-parameter Open-Access Multi- lingual Language Model.” In: Corr abs/2211.5100 (2022). DOI: 10. 48550/ARXIV .2211.05100. arXiv: 2211.05100

2022
[16]

Rozi `ere et al

B. Rozi `ere et al. Code Llama: Open Foundation Models for Code
[17]

StarCoder: May the Source Be with You!

R. Li et al. “StarCoder: May the Source Be with You!” In: Transactions on Machine Learning Research (2023)

2023
[18]

Guo et al

D. Guo et al. DeepSeek-coder: When the Large Language Model Meets Programming – the Rise of Code Intelligence . 2024. DOI: 10.48550/ arXiv.2401.14196. arXiv: 2401.14196 [cs]

2024 doi arXiv
[19]

A Syntax-Guided Edit Decoder for Neural Program Re- pair

Q. Zhu, Z. Sun, Y .-a. Xiao, W. Zhang, K. Yuan, Y . Xiong, and L. Zhang. “A Syntax-Guided Edit Decoder for Neural Program Re- pair.” In: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineeri...

2021
[20]

Ren et al

S. Ren et al. CodeBLEU: A Method for Automatic Evaluation of Code Synthesis. 2020. DOI: 10.48550/arXiv.2009.10297. arXiv: 2009.10297 [cs]

2020 doi arXiv
[21]

LoRA: Low-Rank Adaptation of Large Language Models

E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen. “LoRA: Low-Rank Adaptation of Large Language Models.” In: International Conference on Learning Representations . 2022

2022
[22]

Few-Shot Parameter-Efficient Fine-Tuning Is Better and Cheaper than in-Context Learning

H. Liu, D. Tam, M. Muqeeth, J. Mohta, T. Huang, M. Bansal, and C. A. Raffel. “Few-Shot Parameter-Efficient Fine-Tuning Is Better and Cheaper than in-Context Learning.” In: Advances in Neural Informa- tion Processing Systems. Ed. by S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrav...

2022
[23]

Mach ´aˇcek, A

R. Mach ´aˇcek, A. Grishina, M. Hort, and L. Moonen. Replication Package for ”The Impact of Fine-tuning Large Language Models on Automated Program Repair” . Zenodo. 2025. DOI: 10.5281/zenodo. 16359186

2025 doi
[24]

Attention Is All You Need

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. “Attention Is All You Need.” In: International Conference on Neural Information Processing Systems (NeurIPS). Ed. by I. Guyon, U. V . Luxburg, S. Bengio, H. Wallach, R. Fergus...

2017
[25]

Bavarian, H

M. Bavarian, H. Jun, N. Tezak, J. Schulman, C. McLeavey, J. Tworek, and M. Chen. Efficient Training of Language Models to Fill in the Middle. 2022. DOI: 10.48550/arXiv.2207.14255. arXiv: 2207.14255 [cs]

2022 doi arXiv
[26]

APPT: Boosting Automated Patch Correctness Prediction via Fine- Tuning Pre-Trained Models

Q. Zhang, C. Fang, W. Sun, Y . Liu, T. He, X. Hao, and Z. Chen. “APPT: Boosting Automated Patch Correctness Prediction via Fine- Tuning Pre-Trained Models.” In: IEEE Transactions on Software Engineering 50.3 (2024), pp. 474–494. DOI: 10 . 1109 / TSE . 2024 . 3354969

2024
[27]

Fine-Tuning Large Language Models to Improve Accuracy and Comprehensibility of Automated Code Review

Y . Yu, G. Rong, H. Shen, H. Zhang, D. Shao, M. Wang, Z. Wei, Y . Xu, and J. Wang. “Fine-Tuning Large Language Models to Improve Accuracy and Comprehensibility of Automated Code Review.” In: ACM Trans. Softw. Eng. Methodol. 34.1 (2024), 14:1–14:26. DOI: 10. 1145/3695993

2024
[28]

A Large-Scale Empirical Study on Fine-Tuning Large Language Models for Unit Testing

Y . Shang, Q. Zhang, C. Fang, S. Gu, J. Zhou, and Z. Chen. “A Large-Scale Empirical Study on Fine-Tuning Large Language Models for Unit Testing.” In: Proc. ACM Softw. Eng. 2.ISSTA (2025), IS- STA074:1678–ISSTA074:1700. DOI: 10.1145/3728951

2025 doi
[29]

Towards Efficient Fine-Tuning of Language Models with Organizational Data for Automated Software Review

M. Nashaat and J. Miller. “Towards Efficient Fine-Tuning of Language Models with Organizational Data for Automated Software Review.” In: IEEE Transactions on Software Engineering 50.9 (2024), pp. 2240–

2024
[30]

Towards Understanding the Mixture-of-Experts Layer in Deep Learning

Z. Chen, Y . Deng, Y . Wu, Q. Gu, and Y . Li. “Towards Understanding the Mixture-of-Experts Layer in Deep Learning.” In: Advances in Neural Information Processing Systems . Ed. by S. Koyejo, S. Mo- hamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh. V ol. 35. Curran Associates,...

2022
[31]

M. Z. Haque, S. Afrin, and A. Mastropaolo. A Systematic Literature Review of Parameter-Efficient Fine-Tuning for Large Code Models
[32]

Intrinsic Dimension- ality Explains the Effectiveness of Language Model Fine-Tuning

A. Aghajanyan, S. Gupta, and L. Zettlemoyer. “Intrinsic Dimension- ality Explains the Effectiveness of Language Model Fine-Tuning.” In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natura...

2021 doi
[33]

T. Y . Zhuo, A. Zebaze, N. Suppattarachai, L. von Werra, H. de Vries, Q. Liu, and N. Muennighoff. Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models. 2024. arXiv: 2401.00788 [cs]

2024 arXiv
[34]

Exploring Parameter-Efficient Fine-Tuning Techniques for Code Generation with Large Language Models

M. Weyssow, X. Zhou, K. Kim, D. Lo, and H. Sahraoui. “Exploring Parameter-Efficient Fine-Tuning Techniques for Code Generation with Large Language Models.” In: ACM Trans. Softw. Eng. Methodol. (2025). DOI: 10.1145/3714461

2025 doi
[35]

Resource-Efficient & Effective Code Summarization

S. Afrin, J. Call, K.-N. Nguyen, O. Chaparro, and A. Mastropaolo. “Resource-Efficient & Effective Code Summarization.” In: 2025 IEEE/ACM Second International Conference on AI Foundation Models and Software Engineering (Forge). 2025, pp. 224–235. DOI: 10.1109/ Forge66646.2025.00032

2025
[36]

TransRepair: Context-Aware Program Repair for Compilation Errors

X. Li, S. Liu, R. Feng, G. Meng, X. Xie, K. Chen, and Y . Liu. “TransRepair: Context-Aware Program Repair for Compilation Errors.” In: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering. Association for Computing Machin- ery, 2023. DOI:...

2023
[37]

Break-It-Fix-It: Unsupervised Learning for Program Repair

M. Yasunaga and P. Liang. “Break-It-Fix-It: Unsupervised Learning for Program Repair.” In: Proceedings of the 38th International Conference on Machine Learning . 2021

2021
[38]

Automated Program Repair in the Era of Large Pre-Trained Language Models

C. S. Xia, Y . Wei, and L. Zhang. “Automated Program Repair in the Era of Large Pre-Trained Language Models.” In: Proceedings of the 45th International Conference on Software Engineering . IEEE Press, 2023, pp. 1482–1494. DOI: 10.1109/ICSE48619.2023.00129

2023
[39]

VulRepair: A T5-based Automated Software Vulnerability Repair

M. Fu, C. Tantithamthavorn, T. Le, V . Nguyen, and D. Phung. “VulRepair: A T5-based Automated Software Vulnerability Repair.” In: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineer- ing. Association ...

2022
[40]

Examin- ing Zero-Shot Vulnerability Repair with Large Language Models

H. Pearce, B. Tan, B. Ahmad, R. Karri, and B. Dolan-Gavitt. “Examin- ing Zero-Shot Vulnerability Repair with Large Language Models.” In: 2023 IEEE Symposium on Security and Privacy (SP) . IEEE Computer Society, 2023, pp. 2339–2356. DOI: 10.1109/SP46215.2023.10179420

2023
[41]

N. T. Islam, J. Khoury, A. Seong, M. B. Karkevandi, G. D. L. T. Parra, E. Bou-Harb, and P. Najafirad. LLM-powered Code Vulnerability Repair with Reinforcement Learning and Semantic Reward. 2024. DOI: 10.48550/arXiv.2401.03374. arXiv: 2401.03374 [cs]

2024 doi arXiv
[42]

DLFix: Context-Based Code Trans- formation Learning for Automated Program Repair

Y . Li, S. Wang, and T. N. Nguyen. “DLFix: Context-Based Code Trans- formation Learning for Automated Program Repair.” In: Proceedings of the ACM/IEEE 42nd International Conference on Software Engi- neering. Association for Computing Machinery, 2020, pp. 602–614. DOI: 10.1145/...

2020
[43]

Sorting and Transforming Program Repair Ingredients via Deep Learning Code Similarities

M. White, M. Tufano, M. Mart ´ınez, M. Monperrus, and D. Poshy- vanyk. “Sorting and Transforming Program Repair Ingredients via Deep Learning Code Similarities.” In: 2019 IEEE 26th Interna- tional Conference on Software Analysis, Evolution and Reengineering (SANER). 2019, pp. ...

2019
[44]

CoCoNuT: Combining Context-Aware Neural Translation Models Using Ensemble for Program Repair

T. Lutellier, H. V . Pham, L. Pang, Y . Li, M. Wei, and L. Tan. “CoCoNuT: Combining Context-Aware Neural Translation Models Using Ensemble for Program Repair.” In:Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis (2020). DOI: 10.1145/...

2020
[45]

Review4Repair: Code Review Aided Automatic Program Repairing

F. Huq, M. Hasan, M. M. A. Haque, S. Mahbub, A. Iqbal, and T. Ahmed. “Review4Repair: Code Review Aided Automatic Program Repairing.” In: 143.C (2022). DOI: 10.1016/j.infsof.2021.106765

2022
[46]

Grammar-Based Patches Generation for Automated Program Repair

Y . Tang, L. Zhou, A. Blanco, S. Liu, F. Wei, M. Zhou, and M. Yang. “Grammar-Based Patches Generation for Automated Program Repair.” In: 2021, pp. 1300–1305. DOI: 10.18653/v1/2021.findings-acl.111

2021 doi
[47]

CURE: Code-Aware Neural Ma- chine Translation for Automatic Program Repair

N. Jiang, T. Lutellier, and L. Tan. “CURE: Code-Aware Neural Ma- chine Translation for Automatic Program Repair.” In: 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) . 2021, pp. 1161–1173. DOI: 10.1109/icse43902.2021.00107. arXiv: 2103. 00073 [cs]

2021
[48]

DEAR: A Novel Deep Learning- Based Approach for Automated Program Repair

Y . Li, S. Wang, and T. N. Nguyen. “DEAR: A Novel Deep Learning- Based Approach for Automated Program Repair.” In: Proceedings of the 44th International Conference on Software Engineering . Associ- ation for Computing Machinery, 2022, pp. 511–523. DOI: 10.1145/ 3510003.3510177

2022
[49]

CIRCLE: Continual Repair across Programming Languages

W. Yuan, Q. Zhang, T. He, C. Fang, N. Q. V . Hung, X. Hao, and H. Yin. “CIRCLE: Continual Repair across Programming Languages.” In: Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis. Association for Computing Machinery, 2022, pp. 678–...

2022
[50]

Sk p: A Neural Program Corrector for MOOCs

Y . Pu, K. Narasimhan, A. Solar-Lezama, and R. Barzilay. “Sk p: A Neural Program Corrector for MOOCs.” In: Companion Proceedings of the 2016 ACM SIGPLAN International Conference on Systems, Programming, Languages and Applications: Software for Humanity . Association for Comput...

2016
[51]

Graph-Based, Self-Supervised Program Repair from Diagnostic Feedback

M. Yasunaga and P. Liang. “Graph-Based, Self-Supervised Program Repair from Diagnostic Feedback.” In: Proceedings of the 37th Inter- national Conference on Machine Learning . JMLR.org, 2020

2020
[52]

RepairNet: Contextual Sequence-to-Sequence Network for Automated Program Repair

K. Abhinav, V . Sharvani, A. Dubey, M. D’Souza, N. Bhardwaj, S. Jain, and V . Arora. “RepairNet: Contextual Sequence-to-Sequence Network for Automated Program Repair.” In: Artificial Intelligence in Education: 22nd International Conference, AIED 2021, Utrecht, the Netherlands,...

2021 doi
[53]

Silva, S

A. Silva, S. Fang, and M. Monperrus. RepairLLaMA: Efficient Repre- sentations and Fine-Tuned Adapters for Program Repair . 2024. DOI: 10.48550/arXiv.2312.15698. arXiv: 2312.15698 [cs]

2024 doi arXiv
[54]

R. Paul, M. M. Hossain, M. L. Siddiq, M. Hasan, A. Iqbal, and J. C. S. Santos. Enhancing Automated Program Repair through Fine-Tuning and Prompt Engineering . 2023. DOI: 10 . 48550 / arXiv. 2304 . 07840. arXiv: 2304.07840 [cs]

2023 doi arXiv
[55]

An Analysis of the Automatic Bug Fixing Performance of ChatGPT

D. Sobania, M. Briesch, C. Hanna, and J. Petke. “An Analysis of the Automatic Bug Fixing Performance of ChatGPT.” In: 2023 IEEE/ACM International Workshop on Automated Program Repair (APR) . IEEE Computer Society, 2023, pp. 23–30. DOI: 10.1109/APR59189.2023. 00012

2023
[56]

Multi-Objective Fine-Tuning for Enhanced Program Repair with Llms

B. Yang, H. Tian, J. Ren, H. Zhang, J. Klein, T. F. Bissyand ´e, C. L. Goues, and S. Jin. “Multi-Objective Fine-Tuning for Enhanced Program Repair with Llms.” In: Arxiv Preprint Arxiv:2404.12636 (2024). arXiv: 2404.12636

2024
[57]

A Comprehensive Evaluation of Parameter-Efficient Fine-Tuning on Automated Program Repair

G. Li, C. Zhi, J. Chen, J. Han, and S. Deng. “A Comprehensive Evaluation of Parameter-Efficient Fine-Tuning on Automated Program Repair.” In: Arxiv Preprint Arxiv:2406.05639 (2024). arXiv: 2406 . 05639

2024 arXiv
[58]

Exploring Parameter- Efficient Fine-Tuning of Large Language Model on Automated Pro- gram Repair

G. Li, C. Zhi, J. Chen, J. Han, and S. Deng. “Exploring Parameter- Efficient Fine-Tuning of Large Language Model on Automated Pro- gram Repair.” In: Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. ACM, 2024, pp. 719–

2024
[59]

Comprehensive Fine-Tuning Large Language Models of Code for Automated Program Repair

K. Huang, J. Zhang, X. Bao, X. Wang, and Y . Liu. “Comprehensive Fine-Tuning Large Language Models of Code for Automated Program Repair.” In: IEEE Transactions on Software Engineering 51.4 (2025), pp. 904–928. DOI: 10.1109/TSE.2025.3532759

2025
[60]

Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs

R. Just, D. Jalali, and M. D. Ernst. “Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs.” In: Proceedings of the 2014 International Symposium on Software Testing and Analysis. Association for Computing Machinery, 2014, pp. 437–

2014
[61]

QuixBugs: A Multi-Lingual Program Repair Benchmark Set Based on the Quixey Challenge

D. Lin, J. Koppel, A. Chen, and A. Solar-Lezama. “QuixBugs: A Multi-Lingual Program Repair Benchmark Set Based on the Quixey Challenge.” In: Proceedings Companion of the 2017 ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software fo...

2017
[62]

Chen et al

M. Chen et al. Evaluating Large Language Models Trained on Code
[63]

Llama 2: Open Foundation and Fine-Tuned Chat Models

H. Touvron et al. “Llama 2: Open Foundation and Fine-Tuned Chat Models.” In: Arxiv Preprint Arxiv:2307.09288 (2023). arXiv: 2307 . 09288

2023 arXiv
[64]

Bleu: A Method for Automatic Evaluation of Machine Translation

K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. “Bleu: A Method for Automatic Evaluation of Machine Translation.” In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. Ed. by P. Isabelle, E. Charniak, and D. Lin. Association for Computati...

2002 doi
[65]

CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation

S. Lu et al. “CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation.” In:Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks . Ed. by J. Vanschoren and S. Yeung. V ol. 1. 2021

2021
[66]

Benchmark Data Contamination of Large Language Models: A Survey

C. Xu, S. Guan, D. Greene, M. Kechadi, et al. “Benchmark Data Contamination of Large Language Models: A Survey.” In: Arxiv Preprint Arxiv:2406.04244 (2024). arXiv: 2406.04244

2024 arXiv
[67]

Automated Patch Correctness Assessment: How Far Are We?

S. Wang, M. Wen, B. Lin, H. Wu, Y . Qin, D. Zou, X. Mao, and H. Jin. “Automated Patch Correctness Assessment: How Far Are We?” In: Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering. ACM, 2020, pp. 968–980. DOI: 10. 1145/3324884.3416590. 13

2020
[440]

DOI: 10.1145/2610384.2628055
[731]

DOI: 10.1145/3691620.3695066
[2021]

arXiv: 2107.03374 [cs]

DOI: 10.48550/arXiv.2107.03374. arXiv: 2107.03374 [cs]

doi arXiv
[2023]

arXiv: 2307.10169 [cs]

DOI: 10.48550/arXiv.2307.10169. arXiv: 2307.10169 [cs]

doi arXiv
[2024]

arXiv: 2308.12950 [cs]

DOI: 10.48550/arXiv.2308.12950. arXiv: 2308.12950 [cs]

doi arXiv
[2025]

arXiv: 2504.21569 [cs]

DOI: 10.48550/arXiv.2504.21569. arXiv: 2504.21569 [cs]

doi arXiv
[2253]

DOI: 10.1109/TSE.2024.3428324

2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

On the Automation of Fixing Software Bugs

A. Arcuri. “On the Automation of Fixing Software Bugs.” In: Com- panion of the 30th International Conference on Software Engineering . Association for Computing Machinery, 2008, pp. 1003–1006. DOI: 10.1145/1370175.1370223

work page arXiv 2008

[2] [2]

Huang, Z

K. Huang, Z. Xu, S. Yang, H. Sun, X. Li, Z. Yan, and Y . Zhang. A Survey on Automated Program Repair Techniques . 2023. DOI: 10. 48550/arXiv.2303.18184. arXiv: 2303.18184 [cs]

arXiv doi:10.48550/arxiv.2303.18184 2023

[3] [3]

Getafix: Learning to Fix Bugs Automatically

J. Bader, A. Scott, M. Pradel, and S. Chandra. “Getafix: Learning to Fix Bugs Automatically.” In: Proc. ACM Program. Lang. 3.OOPSLA (2019). DOI: 10.1145/3360585

doi:10.1145/3360585 2019

[4] [4]

DeepDebug: Fixing Python Bugs Using Stack Traces, Backtranslation, and Code Skeletons

D. Drain, C. B. Clement, G. Serrato, and N. Sundaresan. DeepDebug: Fixing Python Bugs Using Stack Traces, Backtranslation, and Code Skeletons. 2021. DOI: 10.48550/arXiv.2105.09352. arXiv: 2105.09352 [cs]

work page Pith review arXiv doi:10.48550/arxiv.2105.09352 2021

[5] [5]

PyDex: Repairing Bugs in Introductory Python Assignments Using LLMs

J. Zhang, J. P. Cambronero, S. Gulwani, V . Le, R. Piskac, G. Soares, and G. Verbruggen. “PyDex: Repairing Bugs in Introductory Python Assignments Using LLMs.” In: Proc. ACM Program. Lang. 8.OOP- SLA1 (2024), 133:1100–133:1124. DOI: 10.1145/3649850

doi:10.1145/3649850 2024

[6] [6]

Mining Fix Patterns for FindBugs Violations

K. Liu, D. Kim, T. F. Bissyande, S. Yoo, and Y . L. Traon. “Mining Fix Patterns for FindBugs Violations.” In: IEEE Transactions on Software Engineering 47.1 (2021), pp. 165–188. DOI: 10 . 1109 / TSE . 2018 . 2884955

work page 2021

[7] [7]

A V ATAR: Fixing Semantic Bugs with Fix Patterns of Static Analysis Violations

K. Liu, A. Koyuncu, D. Kim, and T. F. Bissyande. “A V ATAR: Fixing Semantic Bugs with Fix Patterns of Static Analysis Violations.” In: 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE Computer Society, 2019, pp. 1–12. DOI: 10.1109/SANER.2019.8667970

work page arXiv 2019

[8] [9]

A Survey of Learning-Based Automated Program Repair

Q. Zhang, C. Fang, Y . Ma, W. Sun, and Z. Chen. “A Survey of Learning-Based Automated Program Repair.” In: ACM Trans. Softw. Eng. Methodol. 33.2 (2023). DOI: 10.1145/3631974. 11

doi:10.1145/3631974 2023