Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

Automated Repair of C Programs Using Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that coupling SBFL scores, runtime feedback, and memory of prior patches in an iterative chain-of-thought loop repairs 44.93% of 3,902 Codeflaws C bugs, 3.61 absolute points above a GPT-4-with-CoT baseline.

desk verdict A plausible integration of SBFL, iterative CoT, and memory that is undermined by an unsourced GPT-4 baseline; the headline number is unverified. read the letter →

arxiv 2509.01947 v1 pith:ZJG7WZRK submitted 2025-09-02 cs.SE

Mahdi Farzandway , Fatemeh Ghassemi This is my paper
classification cs.SE
keywords largelanguagemodelsautomaticprogramrepairspectrum-basedfaultlocalizationchain-of-thoughtpromptingiterativerefinementCodeflawsCprogramsAIdebuggingagent
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a repair agent can fix C bugs more reliably by feeding an LLM three signals it would not otherwise see: statistically suspicious lines, failing-test details, and its own previous failed attempts. It wraps those into a structured prompt that asks the model to reason step by step before writing a patch, then reruns the tests and repeats. On 3,902 Codeflaws bugs, the best configuration — Llama 3.1 405B with four iterations — reaches 44.93% full-test-suite repair accuracy, an absolute 3.61-point gain over the reported GPT-4-with-chain-of-thought baseline. The ablation shows each added signal helps: no feedback gives 29.01%, test results alone 36.80%, adding spectrum-based fault localization 41.13%, and adding iterative chain-of-thought on top reaches the headline number. Even where a full fix is not found, roughly half of previously failing tests start passing, so the loop still gives developers a partially improved starting point.

What carries the argument

The central object is the AI Debugger Agent loop with an episodic memory. Each iteration compiles and runs the C program with coverage instrumentation, extracts per-line execution spectra with gcov, computes Ochiai suspiciousness scores, filters lines with scores above 0.5, and assembles a prompt containing the failing tests, the ranked suspicious lines, runtime warnings, and a compact history of prior hypotheses and patches. The model is instructed to reason step by step over these inputs and output a new patch; the loop then recompiles, reruns the tests, and repeats until all tests pass or four iterations are reached. The memory of previous attempts is the piece that distinguishes this fro

What would settle it

Run the same 3,902 Codeflaws bugs through the exact GPT-4 chain-of-thought prompt described in the paper and record the per-bug pass/fail list; then run the proposed SBFL-plus-memory loop on the same set. If GPT-4's repair accuracy is not 41.32%, or the gap relative to 44.93% does not reproduce, the central comparison fails.

Watch

Extended reading notes

Core claim

The central claim is that iterative, memory-aware repair guided by spectrum-based fault localization is what unlocks LLM repair performance, not just model size or a single clever prompt. The paper operationalizes reasoning as a loop rather than a one-pass chain-of-thought: analyze error, analyze previous tries, hypothesize fix, generate patch, re-test, with each stage's output fed into the next prompt. In the reported results, every component contributes for every model size: for Llama 3.1 405B, adding test feedback raises accuracy from 29.01% to 36.80%; adding SBFL raises it to 41.13%; iterative CoT with test feedback but no SBFL reaches 42.92%; and the full combination reaches 44.93%. The

Load-bearing premise

The headline 3.61-point gain assumes the GPT-4-with-chain-of-thought baseline number (41.32%) comes from a like-for-like run over the same 3,902 Codeflaws bugs, and the paper gives no protocol or variance for that baseline.

Editorial extensions

If this is right

  • With the loop, a 70B model reaches 38.83% accuracy, close to the 90B model's 38.13% and well above the 405B model's 29.01% with no feedback, so structured feedback can substitute for some raw model scale.
  • Four iterations capture nearly all of the gain; extending to nine adds little, making a fixed four-iteration budget a practical operating point.
  • When a full fix is not found, roughly half of the previously failing tests begin to pass, giving developers a partially repaired codebase in about 80 seconds.
  • Each added feedback source increases accuracy across all three model sizes, so test feedback, SBFL, and iterative memory behave as complements in this pipeline.
  • The reported numbers are for Codeflaws competition-style C programs; the paper does not claim the method transfers to enterprise codebases or non-executable code.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the baseline arm uses GPT-4 while the proposed loop runs Llama models, the reported gain bundles model choice with loop design; a same-model comparison, such as GPT-4 inside the same loop, would isolate the loop's actual contribution.
  • The ablation suggests a testable prediction: adding either SBFL or iterative memory alone should move accuracy by roughly the amounts seen in Scenarios 3 and 4, so a 2x2 factorial with repeated trials and variance could show whether the two combine additively or overlap.
  • If repair success is judged on the same tests that appear in the prompt, the system may partly fit the visible suite; evaluating on hidden or held-out tests would separate genuine repair from test-suite overfitting.
  • Nothing in the mechanism is C-specific, so the same SBFL-plus-memory recipe could plausibly transfer to Java or Python bug benchmarks, though the paper does not claim this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes an automated program repair framework for C programs that combines spectrum-based fault localization (SBFL, Ochiai formula), test-case feedback, runtime warnings, and iterative Chain-of-Thought-structured prompting with memory of previous repair attempts. The system is evaluated on 3,902 Codeflaws bugs using Llama 70B/90B/405B variants across five feedback scenarios. The headline result is 44.93% repair accuracy in the full configuration (Scenario 5), described as a 3.61-percentage-point absolute improvement over a GPT-4-with-CoT baseline of 41.32%. The paper also reports time-to-repair, failure categories, and partial-improvement analysis.

Significance. If the comparative evaluation is reliable, the paper provides useful evidence that an open-weight LLM (Llama 405B) can match or exceed a GPT-4-with-CoT baseline on Codeflaws when equipped with SBFL, iterative feedback, and memory. The scenario ablation is a strength, as is the use of a large, established benchmark and the explicit failure and partial-improvement analysis. The claimed code availability is also a positive feature. However, the significance is presently conditional: the headline margin rests on an unverifiable baseline, and the evaluation lacks variance estimates for a stochastic process.

major comments (4)
  1. [Table 1 and §5 (Comparison Rationale)] The headline 3.61-point improvement is the difference between the proposed 44.93% (Scenario 5) and the GPT-4 (CoT) baseline of 41.32%. This baseline is not supported as reported. The citation [29] is the GPT-4 Technical Report, which does not contain a Codeflaws APR evaluation. The same 41.32% is listed in all five scenarios, including Scenario 1, which is defined as 'no feedback' and therefore should not equal a configuration that supplies test cases and CoT. No protocol is given for the baseline: number of runs, prompt template, temperature, patch sampling, or whether the authors ran it. Since the central claim is a comparison against this number, the authors must provide a reproducible baseline protocol and raw results, or remove/weaken the comparative claim.
  2. [§5 and Table 1] All reported accuracies are point estimates from a stochastic process (LLM sampling, API retries, and potential nondeterminism). No repeated runs, confidence intervals, or significance tests are reported. The 3.61-point gap could easily lie within run-to-run variance, especially given the API availability issues acknowledged in §5 for Llama 405B. The evaluation should include variance over repeated runs and a statistical test (e.g., paired bootstrap over bugs) for the headline comparison.
  3. [§5 (Fig 5) and Table 1] The iteration limit of four is selected by observing the same Codeflaws benchmark used to report final accuracy: 'we determined that four iterations represent the optimal configuration.' Using the test benchmark for both model selection and final reporting can inflate reported accuracy. I request a sensitivity analysis or held-out validation showing that the 3.61-point margin is robust across iteration limits (e.g., 3, 5, 6) and that the optimal-limit choice is not overfitting the benchmark.
  4. [§8 Conclusion] The conclusion states: 'it achieves a substantial 44.93% absolute accuracy improvement in bug localization and correction over baseline LLM methods.' This misstates the result. The method achieves 44.93% repair accuracy, which is a 3.61-percentage-point improvement over the GPT-4 CoT baseline, not a 44.93-point improvement. The wording should be corrected to avoid misrepresenting the magnitude of the gain.
minor comments (6)
  1. [§5.2] The text says 'four major categories of failure' but lists only three bullets (non-compilable patches, semantic errors, timeouts). Add the missing category or adjust the enumeration.
  2. [§6.1] The paragraph beginning 'ContrastRepair [11] and Conversational APR [12]' is repeated verbatim. Duplicate text should be deleted.
  3. [§2.2] The SBFL formulas are typeset incorrectly (e.g., the Ochiai formula 'Tf (e)√ Tf ×(Tf (e)+Tp(e))' is missing the denominator structure). Please rewrite the formulas with clear notation for Tf, Tp, Tf(e), and Tp(e).
  4. [Throughout] Use consistent capitalization: 'Codeflaws' vs. 'CodeFlaws'. Also, 'approximately48%' in §1 lacks a space.
  5. [Table 2 and §5.1] Table 2 reports 'Average Response Time,' while §5.1 states 'our approach achieves a median repair time of 80 seconds per defect.' Clarify whether Table 2 values are means or medians and how the 80-second figure is derived.
  6. [§4.3 and Table 3] Partial Repair Rate is defined as a metric but not reported; Table 3 reports 'Partial Improvement' instead. Unify the terminology and state how partial repair rate relates to the table.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim is an externally benchmarked repair-accuracy measurement, not a derivation from its own inputs.

full rationale

The paper's central claim—44.93% repair accuracy on 3,902 Codeflaws bugs vs. 41.32% for GPT-4 with CoT—is an empirical measurement against an external benchmark, not a quantity derived from its own assumptions. SBFL scores, test-case feedback, and CoT-structured prompts are all inputs to a repair loop whose output is evaluated by passing all test cases; there is no equation in which the reported accuracy is defined in terms of those inputs. The self-citation to SpecNLP [22] appears only as contextual support for using Codeflaws and for illustrating SBFL formulas (Sections 2.2, 4.4, 6.2); it does not carry the repair-accuracy claim, so it is not load-bearing. The four-iteration configuration is chosen from the same benchmark's convergence curve (Fig. 5), which is an in-sample model-selection concern rather than a definitional circularity: the reported accuracy is the measured value at that configuration, not a fitted parameter relabeled as a prediction. The GPT-4-with-CoT baseline is cited to the GPT-4 Technical Report [29], which contains no Codeflaws evaluation, and the same 41.32% appears in all five scenarios (Table 1) despite Scenario 1 being defined as 'no feedback'; this is a serious reproducibility/fairness weakness in the comparison, but it is a correctness/verifiability issue, not a circularity—it does not make the 44.93% figure equivalent to an input by construction. The paper contains no derivation chain that reduces to its own inputs, so the appropriate circularity finding is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

No mathematical derivation is involved; the headlined numbers are benchmark measurements. The main premises are benchmark oracle validity, stability of LLM API responses, and the authors' design choices for the suspiciousness threshold, iteration limit, and timeout. None of these is independently verified in the paper.

free parameters (3)
  • Ochiai suspiciousness threshold = > 0.5
    Only lines with Ochiai score above 0.5 enter the prompt (Section 3.1 and Section 4.5); this hand-chosen cutoff controls how much localization detail the model sees.
  • iteration limit = 4
    The paper reports four iterations as optimal after observing the convergence curve up to nine iterations on the same benchmark, then uses four for all reported results (Section 5 and Figure 5).
  • per-test timeout = 2 minutes
    Tests are stopped after 2 minutes to avoid infinite loops; bugs that hit this limit are counted as unrepaired (Section 4.5).
assumptions (3)
  • domain assumption Passing all provided Codeflaws test cases is a correct oracle for 'repaired'.
    The repair accuracy metric counts a bug as fixed only when all tests pass (Section 4.3); hidden or unbounded correctness issues are not measured.
  • domain assumption LLM API outputs during the evaluation are representative and not unduly affected by nondeterminism or retries.
    No repeated runs or variance are reported (Section 4.5 and Tables 1 and 2), so the single-point accuracies assume stable API behavior.
  • ad hoc to paper An Ochiai score greater than 0.5 is sufficient to mark the relevant faulty lines for the LLM.
    The paper offers no calibration for this cutoff beyond 'following prior best practices' (Sections 3.1 and 4.5); the central repair loop depends on this threshold.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Automated Repair of C Programs Using Large Language Models." pith.science (2026). https://pith.science/paper/ZJG7WZRK

@misc{pith2026250901947,
  author       = {Pith},
  title        = {Pith review of: Automated Repair of C Programs Using Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZJG7WZRK}},
  note         = {Machine review of arXiv:2509.01947}
}
read the original abstract

This study explores the potential of Large Language Models (LLMs) in automating the repair of C programs. We present a framework that integrates spectrum-based fault localization (SBFL), runtime feedback, and Chain-of-Thought-structured prompting into an autonomous repair loop. Unlike prior approaches, our method explicitly combines statistical program analysis with LLM reasoning. The iterative repair cycle leverages a structured Chain-of-Thought (CoT) prompting approach, where the model reasons over failing tests, suspicious code regions, and prior patch outcomes, before generating new candidate patches. The model iteratively changes the code, evaluates the results, and incorporates reasoning from previous attempts into subsequent modifications, reducing repeated errors and clarifying why some bugs remain unresolved. Our evaluation spans 3,902 bugs from the Codeflaws benchmark, where our approach achieves 44.93% repair accuracy, representing a 3.61% absolute improvement over strong state-of-the-art APR baselines such as GPT-4 with CoT. This outcome highlights a practical pathway toward integrating statistical program analysis with generative AI in automated debugging.

Figures

Figures reproduced from arXiv: 2509.01947 by the authors.

Figure 1
Figure 1. Application of different suspiciousness ranking formulas: black circles indicate executed statements within [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Iterative Refinement Workflow with LLM Integration [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Code Debugging with SBFL, Suspiciousness Scoring, Test Case Feedback, and Generated Code History & [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The structure of prompts 4 Evaluation of Approach In this section, we describe the experimental scenarios and comparisons with recent state-of-the-art program repair baselines (e.g., GPT-4 with CoT) to evaluate the effectiveness and novelty of our proposed methodology.…
Figure 5
Figure 5. Figure 5: Analysis of iterative refinement process convergence, showing optimal performance at four iterations with [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Model Accuracy Comparison Across Scenarios [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Distribution of Partial Code Improvement Scores Across Different Llama Model Variants [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. MultiFixer: A Coordinator-Proposer Based Multi-Agent Framework For Fixing Multi-Hunk Bugs

    cs.SE 2026-07 conditional novelty 6.0 of 10

    Coordinator-proposer multi-agent repair schedules hunks, proposes candidate patches in parallel, and selects/refines them, fixing 326/835 Defects4J bugs with GPT-3.5 and 420 with Claude-3.5-Sonnet.

  2. PYTHALAB-MERA: Validation-Grounded Memory, Retrieval, and Acceptance Control for Frozen-LLM Coding Agents

    cs.CL 2026-05 unverdicted novelty 5.0 of 10

    An external controller for frozen LLMs raises strict validation success on three RL coding tasks from 0/9 to 8/9 by selecting memory records and skills, running fail-fast checks, and propagating credit via eligibility traces.

  3. Augmenting unit test suites from integration tests

    cs.SE 2026-04 unverdicted novelty 5.0 of 10

    A static-plus-dynamic analysis technique extracts isolated unit tests from integration tests to improve test suite structure in Node.js projects.

Reference graph

Works this paper leans on

48 extracted references · 31 canonical work pages · cited by 3 Pith papers

  1. [29]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [1]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Nee- lakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020

    1901

  3. [2]

    Evaluating large language models trained on code

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021. 15

    arXiv 2021

  4. [3]

    Codet5: Identifier-aware unified pre-trained encoder- decoder models for code understanding and generation

    Yue Wang, Weishi Wang, Shafiq Joty, and Steven CH Hoi. Codet5: Identifier-aware unified pre-trained encoder- decoder models for code understanding and generation. arXiv preprint arXiv:2109.00859, 2021

    arXiv 2021

  5. [4]

    An Exploratory Study of Debugging Episodes

    Abdulaziz Alaboudi and Thomas D LaToza. An exploratory study of debugging episodes. arXiv preprint arXiv:2105.02162, 2021

    work page Pith review arXiv 2021

  6. [5]

    Automated c/c++ program repair for high-level synthesis via large language models

    Kangwei Xu, Grace Li Zhang, Xunzhao Yin, Cheng Zhuo, Ulf Schlichtmann, and Bing Li. Automated c/c++ program repair for high-level synthesis via large language models. In Proceedings of the 2024 ACM/IEEE International Symposium on Machine Learning for CAD, pages 1–9, 2024

    work page 2024

  7. [6]

    Autocoderover: Autonomous program improvement

    Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. Autocoderover: Autonomous program improvement. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 1592–1604, 2024

    work page 2024

  8. [7]

    Keep the conversation going: Fixing 162 out of 337 bugs for $0.42 each using chatgpt

    Chunqiu Steven Xia and Lingming Zhang. Keep the conversation going: Fixing 162 out of 337 bugs for $0.42 each using chatgpt. arXiv preprint arXiv:2304.00385, 2023

    arXiv 2023

Show all 48 references
  1. [8]

    A novel approach for automatic program repair using round-trip translation with large language models

    Fernando Vallecillos Ruiz, Anastasiia Grishina, Max Hort, and Leon Moonen. A novel approach for automatic program repair using round-trip translation with large language models. arXiv preprint arXiv:2401.07994, 2024

    2024

  2. [9]

    A study of vulnerability repair in javascript programs with large language models

    Tan Khang Le, Saba Alimadadi, and Steven Y Ko. A study of vulnerability repair in javascript programs with large language models. In Companion Proceedings of the ACM on Web Conference 2024, pages 666–669, 2024

    2024

  3. [10]

    Automatic programming: Large language models and beyond

    Michael R Lyu, Baishakhi Ray, Abhik Roychoudhury, Shin Hwei Tan, and Patanamon Thongtanunam. Automatic programming: Large language models and beyond. arXiv preprint arXiv:2405.02213, 2024

    2024 arXiv

  4. [11]

    Contrastrepair: Enhancing conversation-based automated program repair via contrastive test case pairs

    Jiaolong Kong, Mingfei Cheng, Xiaofei Xie, Shangqing Liu, Xiaoning Du, and Qi Guo. Contrastrepair: Enhancing conversation-based automated program repair via contrastive test case pairs. arXiv preprint arXiv:2403.01971, 2024

    2024

  5. [12]

    Conversational automated program repair

    Chunqiu Steven Xia and Lingming Zhang. Conversational automated program repair. arXiv preprint arXiv:2301.13246, 2023

    2023 arXiv

  6. [13]

    A survey of machine learning for big code and naturalness

    Miltiadis Allamanis, Earl T Barr, Premkumar Devanbu, and Charles Sutton. A survey of machine learning for big code and naturalness. ACM Computing Surveys (CSUR), 51(4):1–37, 2018

    2018

  7. [14]

    Samplefix: Learning to generate functionally diverse fixes

    Hossein Hajipour, Apratim Bhattacharyya, Cristian-Alexandru Staicu, and Mario Fritz. Samplefix: Learning to generate functionally diverse fixes. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 119–133. Springer, 2021

    2021

  8. [15]

    Copiloting the copilots: Fusing large language models with completion engines for automated program repair

    Yuxiang Wei, Chunqiu Steven Xia, and Lingming Zhang. Copiloting the copilots: Fusing large language models with completion engines for automated program repair. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of So...

    2023

  9. [16]

    Generating bug-fixes using pretrained transformers

    Dawn Drain, Chen Wu, Alexey Svyatkovskiy, and Neel Sundaresan. Generating bug-fixes using pretrained transformers. In Proceedings of the 5th ACM SIGPLAN International Symposium on Machine Programming, pages 1–8, 2021

    2021

  10. [17]

    Knod: Domain knowledge distilled tree decoder for automated program repair

    Nan Jiang, Thibaud Lutellier, Yiling Lou, Lin Tan, Dan Goldwasser, and Xiangyu Zhang. Knod: Domain knowledge distilled tree decoder for automated program repair. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pages 1251–1263. IEEE, 2023

    2023

  11. [18]

    Evaluating the state-of-the-art of end-to-end natural language generation: The e2e nlg challenge

    Ondˇrej Dušek, Jekaterina Novikova, and Verena Rieser. Evaluating the state-of-the-art of end-to-end natural language generation: The e2e nlg challenge. Computer Speech & Language, 59:123–156, 2020

    2020

  12. [19]

    On the accuracy of spectrum-based fault localization

    Rui Abreu, Peter Zoeteweij, and Arjan JC Van Gemund. On the accuracy of spectrum-based fault localization. In Testing: Academic and industrial conference practice and research techniques-MUTATION (TAICPART- MUTATION 2007), pages 89–98. IEEE, 2007

    2007

  13. [20]

    An evaluation of similarity coefficients for software fault localization

    Rui Abreu, Peter Zoeteweij, and Arjan JC Van Gemund. An evaluation of similarity coefficients for software fault localization. In 2006 12th Pacific Rim International Symposium on Dependable Computing (PRDC’06), pages 39–46. IEEE, 2006

    2006

  14. [21]

    Empirical evaluation of the tarantula automatic fault-localization technique

    James A Jones and Mary Jean Harrold. Empirical evaluation of the tarantula automatic fault-localization technique. In Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering , pages 273–282, 2005

    2005

  15. [22]

    Specnlp: A pre-trained model enhanced with spectrum profile for bug localization

    Mahdi Farzandway and Fatemeh Ghassemi. Specnlp: A pre-trained model enhanced with spectrum profile for bug localization. In 2024 IEEE International Conference on Artificial Intelligence Testing (AITest), pages 81–86. IEEE, 2024

    2024

  16. [23]

    Spectrum-based software fault localization: A survey of techniques, advances, and challenges

    Higor A de Souza, Marcos L Chaim, and Fabio Kon. Spectrum-based software fault localization: A survey of techniques, advances, and challenges. arXiv preprint arXiv:1607.04347, 2016. 16

    2016 arXiv

  17. [24]

    Locating faults with program slicing: an empirical analysis

    Ezekiel Soremekun, Lukas Kirschner, Marcel Böhme, and Andreas Zeller. Locating faults with program slicing: an empirical analysis. Empirical Software Engineering, 26:1–45, 2021

    2021

  18. [25]

    Natural language generation and understanding of big code for ai-assisted programming: A review

    Man-Fai Wong, Shangxin Guo, Ching-Nam Hang, Siu-Wai Ho, and Chee-Wei Tan. Natural language generation and understanding of big code for ai-assisted programming: A review. Entropy, 25(6):888, 2023

    2023

  19. [26]

    Systematic literature review on analyzing the impact of prompt engineering on efficiency, code quality, and security in crud application development

    KAA Shanuka, J Wijayanayake, and K Vidanage. Systematic literature review on analyzing the impact of prompt engineering on efficiency, code quality, and security in crud application development. Journal of Desk Research Review and Analysis, 2(1), 2024

    2024

  20. [27]

    Codeflaws: a programming competition benchmark for evaluating automated program repair tools

    Shin Hwei Tan, Jooyong Yi, Sergey Mechtaev, Abhik Roychoudhury, et al. Codeflaws: a programming competition benchmark for evaluating automated program repair tools. In 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C), pages 180–182. IEEE, 2017

    2017

  21. [28]

    Coconut: combining context-aware neural translation models using ensemble for program repair

    Thibaud Lutellier, Hung Viet Pham, Lawrence Pang, Yitong Li, Moshi Wei, and Lin Tan. Coconut: combining context-aware neural translation models using ensemble for program repair. In Proceedings of the 29th ACM SIGSOFT international symposium on software testing and analysis, p...

    2020

  22. [30]

    Staged program repair with condition synthesis

    Fan Long and Martin Rinard. Staged program repair with condition synthesis. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, pages 166–178, 2015

    2015

  23. [31]

    Angelix: Scalable multiline program patch synthesis via symbolic analysis

    Sergey Mechtaev, Jooyong Yi, and Abhik Roychoudhury. Angelix: Scalable multiline program patch synthesis via symbolic analysis. In Proceedings of the 38th international conference on software engineering, pages 691–701, 2016

    2016

  24. [32]

    Agentless: Demystifying llm-based software engineering agents, 2024

    Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. Agentless: Demystifying llm-based software engineering agents, 2024. URL https://arxiv. org/abs/2407.01489, 2024

    2024 arXiv

  25. [33]

    Automated repair of programs from large language models

    Zhiyu Fan, Xiang Gao, Martin Mirchev, Abhik Roychoudhury, and Shin Hwei Tan. Automated repair of programs from large language models. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pages 1469–1481. IEEE, 2023

    2023

  26. [34]

    Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770, 2023

    Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770, 2023

    2023 arXiv

  27. [35]

    Aligning llms for fl-free program repair

    Junjielong Xu, Ying Fu, Shin Hwei Tan, and Pinjia He. Aligning llms for fl-free program repair. arXiv preprint arXiv:2404.08877, 2024

    2024 arXiv

  28. [36]

    Cigar: Cost-efficient program repair with llms

    Dávid Hidvégi, Khashayar Etemadi, Sofia Bobadilla, and Martin Monperrus. Cigar: Cost-efficient program repair with llms. arXiv preprint arXiv:2402.06598, 2024

    2024 arXiv

  29. [37]

    Automatically finding patches using genetic programming

    Westley Weimer, ThanhVu Nguyen, Claire Le Goues, and Stephanie Forrest. Automatically finding patches using genetic programming. In 2009 IEEE 31st International Conference on Software Engineering, pages 364–374. IEEE, 2009

    2009

  30. [38]

    Fully autonomous programming with large language models

    Vadim Liventsev, Anastasiia Grishina, Aki Härmä, and Leon Moonen. Fully autonomous programming with large language models. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 1146–1155, 2023

    2023

  31. [39]

    Empirical review of java program repair tools: A large-scale experiment on 2,141 bugs and 23,551 repair attempts

    Thomas Durieux, Fernanda Madeiral, Matias Martinez, and Rui Abreu. Empirical review of java program repair tools: A large-scale experiment on 2,141 bugs and 23,551 repair attempts. In Proceedings of the 2019 27th ACM joint meeting on european software engineering conference an...

    2019

  32. [40]

    Codebert: A pre-trained model for programming and natural languages

    Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, et al. Codebert: A pre-trained model for programming and natural languages. arXiv preprint arXiv:2002.08155, 2020

    2002 arXiv

  33. [41]

    Graphcodebert: Pre-training code representations with data flow

    Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, et al. Graphcodebert: Pre-training code representations with data flow. arXiv preprint arXiv:2009.08366, 2020

    2009 arXiv

  34. [42]

    Unified pre-training for program understanding and generation

    Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. Unified pre-training for program understanding and generation. arXiv preprint arXiv:2103.06333, 2021

    2021 arXiv

  35. [43]

    Sapfix: Automated end-to-end repair at scale

    Alexandru Marginean, Johannes Bader, Satish Chandra, Mark Harman, Yue Jia, Ke Mao, Alexander Mols, and Andrew Scott. Sapfix: Automated end-to-end repair at scale. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEI...

    2019

  36. [44]

    code2vec: Learning distributed representations of code

    Uri Alon, Meital Zilberstein, Omer Levy, and Eran Yahav. code2vec: Learning distributed representations of code. Proceedings of the ACM on Programming Languages, 3(POPL):1–29, 2019

    2019

  37. [45]

    code2seq: Generating sequences from structured representations of code

    Uri Alon, Shaked Brody, Omer Levy, and Eran Yahav. code2seq: Generating sequences from structured representations of code. arXiv preprint arXiv:1808.01400, 2018

    2018 arXiv

  38. [46]

    Evosuite: automatic test suite generation for object-oriented software

    Gordon Fraser and Andrea Arcuri. Evosuite: automatic test suite generation for object-oriented software. In Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering, pages 416–419, 2011

    2011

  39. [47]

    Klee: unassisted and automatic generation of high-coverage tests for complex systems programs

    Cristian Cadar, Daniel Dunbar, Dawson R Engler, et al. Klee: unassisted and automatic generation of high-coverage tests for complex systems programs. In OSDI, volume 8, pages 209–224, 2008

    2008

  40. [48]

    History-driven build failure fixing: how far are we? In Proceedings of the 28th acm sigsoft international symposium on software testing and analysis, pages 43–54, 2019

    Yiling Lou, Junjie Chen, Lingming Zhang, Dan Hao, and Lu Zhang. History-driven build failure fixing: how far are we? In Proceedings of the 28th acm sigsoft international symposium on software testing and analysis, pages 43–54, 2019. 18

    2019

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.