Pith. sign in

REVIEW 4 major objections 6 minor 37 references

Augmenting Large Language Models with Static Code Analysis for Automated Code Quality Improvements

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Pairing SonarQube with GPT-4o and RAG can clear 100% of flagged bugs and vulnerabilities and 81% of code smells in one project at under $35.

desk verdict A modest engineering study whose headline numbers (100% bug fix) outrun its evidence; useful cost and prompt-design data, but the success metric is just SonarQube flag disappearance. read the letter →

arxiv 2506.10330 v1 pith:VBCJ35FU submitted 2025-06-12 cs.SE cs.AI

Seyed Moein Abtahi , Akramul Azim This is my paper
classification cs.SEcs.AI
keywords largelanguagemodelsstaticcodeanalysisSonarQubeGPT-3.5TurboGPT-4oretrieval-augmentedgenerationpromptengineeringautomatedrevision
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports a pipeline that turns static-analysis warnings into a checklist for large language models. SonarQube scans a project, and each flagged bug, vulnerability, or code smell is converted into a structured record—file, line, description, suggested fix—which becomes the body of a prompt. The pipeline first sends these prompts to GPT-3.5 Turbo; whatever SonarQube still flags after that pass is sent to GPT-4o, with retrieval-augmented generation pulling candidate fixes from external sources into the prompt. On one project's 7,599 flagged issues, the two models together cleared every bug and every vulnerability, 81.2 percent of code smells, and 71.6 percent of issues when all types were batched into one prompt. The motivating result is practical: more than 7,500 issues were revised in under three hours and for less than $35 in API costs.

What carries the argument

The load-bearing mechanism is the SonarQube-to-prompt transcription. Each issue is recorded as a data-file row with file path, line number, rule message, and suggested fix; Algorithm 2 turns that row into a task statement with a few-shot example of a correct fix for each issue type (bug, vulnerability, code smell). Before the prompt is sent, a RAG stage builds a search query from the issue description, queries external sources ranked by credibility, and appends the retrieved solutions to the prompt. Revision runs in two passes—GPT-3.5 Turbo first for cost, GPT-4o on the residual findings—and a custom Code Comparison App computes precision, recall, and F1 per file so a human can spot hallucinations before changes are accepted.

What would settle it

Run the pipeline on a repository that has a real test suite and compare test results before and after revision; if the revised code passes SonarQube's re-scan but fails pre-existing tests, the reported success rates are measuring rule-silence rather than code quality. The paper itself states that no test cases were available for automated verification.

Watch

Extended reading notes

Core claim

The paper's central claim is that a two-pass LLM revision loop, organized by SonarQube's own issue reports, can remove the large majority of static-analysis findings from a real codebase at an API cost low enough to be routine. In the divided setup, issues are grouped by type: 234 bugs and 61 vulnerabilities were fully cleared across the two GPT passes, while 7,304 code smells fell to 1,367 remaining, a combined success rate of 81.2 percent. In the comprehensive setup, one prompt per file carried all issue types and the combined success rate was 71.6 percent. The paper also claims that the divided approach yields higher accuracy—an average F1-score of at least 96.05 percent per file versus 94.24 percent for the comprehensive approach—and that adding retrieval-augmented generation lets the cheaper model resolve more issues than prompt engineering alone. These numbers are defined against SonarQube re-scans: a revised issue counts as resolved when it no longer appears in the follow-up scan.

Load-bearing premise

The whole result rests on treating 'no longer flagged by SonarQube' as 'fixed,' and with no test suite in the project the paper's own manual-verification step is the only guard against a revision that deletes code or changes behavior.

Editorial extensions

If this is right

  • Most routine static-analysis debt can be handled automatically: on the project tested, no bug or vulnerability survived both passes, and code smells dropped from 7,304 to 1,367.
  • Splitting prompts by issue type beats mixing types: the divided runs reached 96–100 percent per-type success, while the single-prompt comprehensive run reached 71.6 percent.
  • The cheap-first, strong-second routing means the stronger model only sees leftover findings, which keeps cost low; the paper puts total API spend at $31.96 for the divided run and $25.91 for the comprehensive run.
  • Adding retrieval-augmented generation shifts work to the cheaper GPT-3.5 Turbo and raises the reported evaluation metrics, so external-source retrieval is a lever on cost rather than only on accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I infer that the 100 percent figures are rule-silence rates, not behavioral guarantees: they certify that SonarQube's rules no longer fire, so a revision could in principle delete a flagged line and still count as success unless a human catches it.
  • A testable extension is to run the same pipeline on a repository with a pre-existing test suite; comparing pre- and post-revision test results would convert 'no longer flagged' into 'still behaves the same.'
  • The paper's own limitation section implies that the unmeasured human-verification loop may dominate end-to-end cost even when API cost is tiny; automating test generation is the step that would make the pipeline fully hands-off.
  • Because the issue-extraction layer is generic, the same two-pass design should transfer to other static-analysis rule engines by swapping the analyzer while keeping the prompt and RAG stages unchanged.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes and evaluates a pipeline that combines SonarQube static analysis, prompt-engineered GPT-3.5 Turbo and GPT-4o, and retrieval-augmented generation (RAG) to automatically revise code files and remove detected issues. The authors report high success rates in eliminating SonarQube flags (100% for bugs, 100% for vulnerabilities, 81.2% for code smells, 71.6% for a comprehensive prompt) and claim low time and API cost, with a divided-by-issue-type approach outperforming a comprehensive approach. The evaluation is based on SonarQube re-scans and a custom Code Comparison App that computes precision, recall, and F1 scores.

Significance. If the reported success rates were validated as genuine code-quality improvements, the work would offer a practical and inexpensive method for cleaning up static-analysis findings, which is a relevant contribution to automated software maintenance. The paper has some strengths: it clearly describes the pipeline, reports actual API costs, and includes a small validation experiment (Table II) showing that GPT-4o covers all issues resolved by GPT-3.5 Turbo. However, the central evaluation metric is flag disappearance rather than verified code correctness, and the manual effort in the pipeline is never quantified. As published, the evidence supports the weaker claim that SonarQube flags can be made to disappear cheaply, not that code quality was improved.

major comments (4)
  1. [Section III, Section VI, Table III] The success-rate metric is a SonarQube re-scan: an issue is counted as resolved when its flag no longer appears. This is not equivalent to fixing the issue. The manuscript itself states in Section III that 'it is assumed that no file contains test cases, necessitating manual verification' and in Section VI concedes 'lack of test cases for automated verification.' A flag can disappear because the flagged line was deleted, because the program behavior was changed, or because a suppression comment was added. Consequently, the headline numbers in Table III (100% for bugs, 100% for vulnerabilities, 81.2% for code smells, 71.6% comprehensive) should be interpreted as flag-suppression rates, not verified fixes. The central claim of the paper depends on this distinction, so the authors must either run the project's existing test suite or generate tests (as the Future Work section proposes) and compare behavior before and after revision, and they should report the human effort spent in the manual verification and correction loop shown in Fig. 1.
  2. [Section IV.A.6, Table III] The Code Comparison App is described as computing precision, recall, and F1-score, and these are reported in Table III as 'Ave. Precision in Rev. Files,' 'Ave. Recall in Rev. Files,' and 'Ave. F1-Score in Rev. Files.' However, the text also says that 'A metric value close to 100 percent indicates fewer changes,' which is a text-similarity measure, not a classification metric. Under this interpretation, a revised file with no changes at all would score 100%, and a file with a large block of code deleted might still score high. Therefore, the statement in Section V that 'the high F1-Score of the revised files confirms that these models ... are reliable tools' is unsupported. The authors should either define a proper classification task (what constitutes a true positive, false positive, etc.) or relabel these values as similarity scores and remove the accuracy interpretation.
  3. [Section V, Conclusion] The Conclusion states that 'revising over 7,500 issues took less than three hours and cost less than $35.' Table III reports only API costs ($31.96 for the divided approach and $25.91 for the comprehensive approach), and no timing measurement appears anywhere in the experimental setup. Moreover, the pipeline in Fig. 1 includes manual verification and manual correction steps, whose effort is never quantified. Thus 'less than three hours' cannot be a total pipeline time unless the authors measured it and included the human time; if it refers only to API execution time, that should be stated explicitly. Please provide a breakdown of how the time was measured, or remove the claim.
  4. [Section IV.A.5, Section IV.B] The evaluation is self-referential in a way that weakens external validity: SonarQube supplies the issue descriptions and suggested fixes that are fed into the prompts (Section IV.A.1, Table I), and the SonarQube Community is one of the RAG sources (Section IV.A.5); the same SonarQube re-scan then serves as the oracle for whether an issue was resolved. This does not automatically invalidate the results, but it means the success rates reflect the LLM's ability to satisfy SonarQube's rules, not necessarily to improve code quality. A concrete test of external validity would be to re-scan the revised code with an independent static analyzer (e.g., ESLint, PMD) or to hold out a random subset of issues for which SonarQube's suggested fixes are not included in the prompt.
minor comments (6)
  1. [Section III] There is a typo in the third paragraph: 'The the following section' should read 'The following section.'
  2. [Section II] The phrase 'RAG-bAsed' has inconsistent capitalization and should be 'RAG-based.'
  3. [Section V, Conclusion] The conclusion claims the divided approach is superior in 'both effectiveness and cost-efficiency,' but Section V states that the comprehensive approach 'proved to be more cost-effective overall, saving approximately 18.9% compared with the divided approach.' These two statements contradict each other and should be reconciled.
  4. [Figures 5 and 6] The captions for Figures 5 and 6 define 'Success Rate' and 'Accuracy (F1-Score),' respectively, but the denominators used to compute these rates are not stated in the captions; please clarify whether the success rate is the percentage of initially detected issues that no longer appear, the percentage of revised files that pass a rescan, or something else.
  5. [Section IV.A.3, Algorithm 2] Section IV.A.3 says the API request consolidates all issues into a single file and receives 'the revised file in its original format without additional comments in a single API request,' while Algorithm 2's final instruction says 'Only corrected code sections will be provided.' These two descriptions of the output format are inconsistent; please clarify the actual output format.
  6. [Section IV.A.2] The phrase 'large-scale software project' is asserted, but no file count, line count, or other measure of the EIS dataset's size is provided, making it difficult to judge the scale and reproducibility of the experiment.

Circularity Check

0 steps flagged · score 0.0 of 10

No derivation-level circularity: Table III success rates are empirical rescan counts, not fitted quantities; the self-referential SonarQube evaluation is a validity threat, not an equation-level reduction.

full rationale

The paper's load-bearing numbers (Table III) are observational counts: initial SonarQube scans, revisions by GPT-3.5 Turbo and GPT-4o, and follow-up SonarQube scans. There is no parameter fitted to a subset of data and then used to predict a quantity that is the same by construction. The 'success rate' is defined operationally as the disappearance of a SonarQube flag, and the prompts are built from SonarQube's issue descriptions and suggested solutions (Sec. IV.A.1: 'The data files extracted from SonarQube include file names, locations, issue lines, SonarQube’s suggested solutions, and issue types'), so the oracle and the hint source coincide. That self-referential evaluation weakens the external claim that code quality improved, especially because Sec. III states the manual verification step 'assumes that no file contains test cases' and Sec. VI concedes 'the lack of test cases for automated verification of the files generated by LLMs.' The reported F1 values are explicitly measures of textual similarity to the original files, not behavioral correctness (Sec. IV.A.6). These are serious validity limitations, but they do not make any derived quantity equal to its input by construction: the LLM could have failed to remove flags despite receiving SonarQube suggestions, so the 100% bug/vulnerability rates are not tautological. No self-citations, imported uniqueness theorems, or ansatz-by-citation patterns appear. I therefore find no circularity under the strict definition and score 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on three implicit assumptions: SonarQube flags are a valid ground truth, static-analysis issue removal means code-quality improvement, and the manual verification step is sufficient to catch all harmful LLM edits. None is independently validated. The hand-chosen RAG ranking, the Google top-3 cutoff, and the one-example-per-issue few-shot format are the main free design choices; the paper reports no sensitivity analysis. No invented theoretical entities are proposed.

free parameters (3)
  • RAG source priority order = GitHub > SonarQube Community > Stack Overflow > Google
    Section IV.A.5 fixes this ordering by hand with no ablation; the ranking determines which retrieved solutions enter the prompt.
  • Top ranked Google pages used = 3
    Section IV.A.5 says the system focuses on the top three ranked pages; no sensitivity analysis is reported.
  • Few-shot examples per issue type = 1
    Section III.A.1 and Algorithm 2 include one example per issue type; the choice came from iterative testing and was not systematically ablated.
assumptions (4)
  • domain assumption SonarQube issue reports are a valid proxy for code quality and issue resolution.
    Every success rate in Table III is defined as a SonarQube flag no longer appearing after revision; no independent tests or developer-review scores are used.
  • domain assumption The project's code files contain no test cases, so manual verification is sufficient to guarantee the correctness of revisions.
    Section III states 'no file contains test cases, necessitating manual verification even for the original files.' The paper relies on this manual step to catch hallucinations.
  • ad hoc to paper High text similarity between original and revised files is a valid accuracy signal.
    Section IV.A.6 says 'A metric value close to 100 percent indicates fewer changes,' which conflates diff similarity with semantic correctness.
  • ad hoc to paper External web sources (GitHub, SonarQube Community, Stack Overflow, Google) provide correct solutions for the issues.
    Section IV.A.5 ranks sources without validating retrieved solutions against tests or a held-out set of fixes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Augmenting Large Language Models with Static Code Analysis for Automated Code Quality Improvements." pith.science (2026). https://pith.science/paper/VBCJ35FU

@misc{pith2026250610330,
  author       = {Pith},
  title        = {Pith review of: Augmenting Large Language Models with Static Code Analysis for Automated Code Quality Improvements},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VBCJ35FU}},
  note         = {Machine review of arXiv:2506.10330}
}
read the original abstract

This study examined code issue detection and revision automation by integrating Large Language Models (LLMs) such as OpenAI's GPT-3.5 Turbo and GPT-4o into software development workflows. A static code analysis framework detects issues such as bugs, vulnerabilities, and code smells within a large-scale software project. Detailed information on each issue was extracted and organized to facilitate automated code revision using LLMs. An iterative prompt engineering process is applied to ensure that prompts are structured to produce accurate and organized outputs aligned with the project requirements. Retrieval-augmented generation (RAG) is implemented to enhance the relevance and precision of the revisions, enabling LLM to access and integrate real-time external knowledge. The issue of LLM hallucinations - where the model generates plausible but incorrect outputs - is addressed by a custom-built "Code Comparison App," which identifies and corrects erroneous changes before applying them to the codebase. Subsequent scans using the static code analysis framework revealed a significant reduction in code issues, demonstrating the effectiveness of combining LLMs, static analysis, and RAG to improve code quality, streamline the software development process, and reduce time and resource expenditure.

Figures

Figures reproduced from arXiv: 2506.10330 by the authors.

Figure 1
Figure 1. Automated Code Issue Detection and Revision Pipeline [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Retrieval-Augmented Generation process particularly valuable for producing accurate and explainable responses, as it enables the model to access real-time, up￾to-date information [31] and solutions for queries that may extend beyond its original training data. The retrieved data, which includes various solution approaches, were synthesized and evaluated. The ranking of the retrieved information and solutions is base… view at source ↗
Figure 3
Figure 3. Illustration of the iterative prompt engineering process [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: Success Rates by Issue Type for GPT-3.5 Turbo and GPT-4o [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 4
Figure 4. Figure 4: Files Revised by Issue Type for GPT-3.5 Turbo and GPT-4o [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 7
Figure 7. Figure 7: Revision Costs by Issue Type for GPT-3.5 Turbo and GPT-4o [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Comparison of success rates, precision, recall, and F1-scores for GPT [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 19 canonical work pages

  1. [1]

    Large language models meet nlp: A survey,

    L. Qin, Q. Chen, X. Feng, Y . Wu, Y . Zhang, Y . Li, M. Li, W. Che, and P. S. Yu, “Large language models meet nlp: A survey,” arXiv preprint arXiv:2405.12819, 2024

    arXiv 2024

  2. [2]

    Programming is hard-or at least it used to be: Educational opportunities and challenges of ai code generation,

    B. A. Becker, P. Denny, J. Finnie-Ansley, A. Luxton-Reilly, J. Prather, and E. A. Santos, “Programming is hard-or at least it used to be: Educational opportunities and challenges of ai code generation,” in Proceedings of the 54th ACM Technical Symposium on Computer Science Education V . 1, 2023, pp. 500–506

    2023

  3. [3]

    Security code review by llms: A deep dive into responses,

    J. Yu, P. Liang, Y . Fu, A. Tahir, M. Shahin, C. Wang, and Y . Cai, “Security code review by llms: A deep dive into responses,” arXiv preprint arXiv:2401.16310, 2024

    arXiv 2024

  4. [4]

    Sorald: Automatic patch suggestions for sonarqube static analysis violations,

    K. Etemadi, N. Harrand, S. Lars ´en, H. Adzemovic, H. L. Phu, A. Verma, F. Madeiral, D. Wikstr¨om, and M. Monperrus, “Sorald: Automatic patch suggestions for sonarqube static analysis violations,” IEEE Transactions on Dependable and Secure Computing , vol. 20, no. 4, pp. 2794–2810, 2023

    work page 2023

  5. [5]

    Gpt-3.5 turbo,

    OpenAI, “Gpt-3.5 turbo,” https://platform.openai.com/docs/models/ gpt-3-5-turbo, 2024

    work page 2024

  6. [6]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar et al. , “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023

    arXiv 2023

  7. [7]

    Enhancing text classification through llm-driven active learning and human annotation,

    H. Rouzegar and M. Makrehchi, “Enhancing text classification through llm-driven active learning and human annotation,” in The 18th Linguistic Annotation Workshop (LA W-XVIII) Co-located with EACL 2024 , 2024, p. 98

    work page 2024

  8. [8]

    Llm agents can autonomously exploit one-day vulnerabilities,

    R. Fang, R. Bindu, A. Gupta, and D. Kang, “Llm agents can autonomously exploit one-day vulnerabilities,” arXiv preprint arXiv:2404.08144, 2024

    arXiv 2024

Show all 37 references
  1. [9]

    Llm-powered code vulnerability repair with reinforcement learning and semantic reward,

    N. T. Islam, J. Khoury, A. Seong, G. D. L. T. Parra, E. Bou-Harb, and P. Najafirad, “Llm-powered code vulnerability repair with reinforcement learning and semantic reward,” arXiv preprint arXiv:2401.03374 , 2024

    2024 arXiv

  2. [10]

    Would static analysis tools help developers with code reviews?

    S. Panichella, V . Arnaoudova, M. Di Penta, and G. Antoniol, “Would static analysis tools help developers with code reviews?” in 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER) . IEEE, 2015, pp. 161–170

    2015

  3. [11]

    A systematic literature review and taxonomy of modern code review,

    N. Davila and I. Nunes, “A systematic literature review and taxonomy of modern code review,” Journal of Systems and Software , vol. 177, p. 110951, 2021

    2021

  4. [12]

    Combining static analysis and dynamic symbolic execution in a toolchain to detect fault injection vulnerabilities,

    G. Lacombe, D. Feliot, E. Boespflug, and M.-L. Potet, “Combining static analysis and dynamic symbolic execution in a toolchain to detect fault injection vulnerabilities,” Journal of Cryptographic Engineering, vol. 14, no. 1, pp. 147–164, 2024

    2024

  5. [13]

    Software bug report detection methods based on machine learning techniques,

    U. Srinivasarao, A. Chaganti, A. Mannam, D. Kotrikeline, and H. Katta, “Software bug report detection methods based on machine learning techniques,” in 2024 2nd International Conference on Advancement in Computation & Computer Technologies (InCACCT) . IEEE, 2024, pp. 99–104

    2024

  6. [14]

    Methods for the prevention, detection and removal of software security vulnerabilities,

    J.-E. J. Tevis and J. A. Hamilton, “Methods for the prevention, detection and removal of software security vulnerabilities,” in Proceedings of the 42nd annual Southeast regional conference , 2004, pp. 197–202

    2004

  7. [15]

    A quantitative analysis of quality and consistency in ai-generated code,

    A. Clark, D. Igbokwe, S. Ross, and M. F. Zibran, “A quantitative analysis of quality and consistency in ai-generated code,” in 2024 7th International Conference on Software and System Engineering (ICoSSE). IEEE, 2024, pp. 37–41

    2024

  8. [16]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al. , “Language models are unsupervised multitask learners,” OpenAI blog , vol. 1, no. 8, p. 9, 2019

    2019

  9. [17]

    How does code readability change during software evolution?

    V . Piantadosi, F. Fierro, S. Scalabrino, A. Serebrenik, and R. Oliveto, “How does code readability change during software evolution?” Empir- ical Software Engineering , vol. 25, pp. 5374–5412, 2020

    2020

  10. [18]

    Exploring the capabilities of llms for code change related tasks,

    L. Fan, J. Liu, Z. Liu, D. Lo, X. Xia, and S. Li, “Exploring the capabilities of llms for code change related tasks,” arXiv preprint arXiv:2407.02824, 2024

    2024 arXiv

  11. [19]

    Domain specialization as the key to make large language models disruptive: A comprehensive survey,

    C. Ling, X. Zhao, J. Lu, C. Deng, C. Zheng, J. Wang, T. Chowdhury, Y . Li, H. Cui, X. Zhang et al. , “Domain specialization as the key to make large language models disruptive: A comprehensive survey,” arXiv preprint arXiv:2305.18703, 2023

    2023 arXiv

  12. [20]

    Review of large vision models and visual prompt engineering,

    J. Wang, Z. Liu, L. Zhao, Z. Wu, C. Ma, S. Yu, H. Dai, Q. Yang, Y . Liu, S. Zhang et al. , “Review of large vision models and visual prompt engineering,” Meta-Radiology, p. 100047, 2023

    2023

  13. [21]

    Prompt engineering with chatgpt: a guide for academic writers,

    L. Giray, “Prompt engineering with chatgpt: a guide for academic writers,” Annals of biomedical engineering , vol. 51, no. 12, pp. 2629– 2633, 2023

    2023

  14. [22]

    Code search is all you need? improving code suggestions with code search,

    J. Chen, X. Hu, Z. Li, C. Gao, X. Xia, and D. Lo, “Code search is all you need? improving code suggestions with code search,” in Proceedings of the IEEE/ACM 46th International Conference on Software Engineering , 2024, pp. 1–13

    2024

  15. [23]

    Retrieval-augmented generation for ai-generated content: A survey,

    P. Zhao, H. Zhang, Q. Yu, Z. Wang, Y . Geng, F. Fu, L. Yang, W. Zhang, and B. Cui, “Retrieval-augmented generation for ai-generated content: A survey,” arXiv preprint arXiv:2402.19473 , 2024

    2024 arXiv

  16. [24]

    Explor- ing rag-based vulnerability augmentation with llms,

    S. S. Daneshvar, Y . Nong, X. Yang, S. Wang, and H. Cai, “Explor- ing rag-based vulnerability augmentation with llms,” arXiv preprint arXiv:2408.04125, 2024

    2024 arXiv

  17. [25]

    Improving real-time knowledge retrieval in large language models with a dns-style hierarchical query rag,

    A. Reynolds and F. Corrigan, “Improving real-time knowledge retrieval in large language models with a dns-style hierarchical query rag,” Authorea Preprints, 2024

    2024

  18. [26]

    What makes a code review useful to opendev developers? an empirical investigation,

    A. K. Turzo and A. Bosu, “What makes a code review useful to opendev developers? an empirical investigation,” Empirical Software Engineering, vol. 29, no. 1, p. 6, 2024

    2024

  19. [27]

    Ai-powered code review with llms: Early results,

    Z. Rasheed, M. A. Sami, M. Waseem, K.-K. Kemell, X. Wang, A. Nguyen, K. Syst ¨a, and P. Abrahamsson, “Ai-powered code review with llms: Early results,” arXiv preprint arXiv:2404.18496 , 2024

    2024

  20. [28]

    Llama-reviewer: Advancing code review automation with large language models through parameter- efficient fine-tuning,

    J. Lu, L. Yu, X. Li, L. Yang, and C. Zuo, “Llama-reviewer: Advancing code review automation with large language models through parameter- efficient fine-tuning,” in 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE) . IEEE, 2023, pp. 647–658

    2023

  21. [29]

    Learning from few examples: A summary of approaches to few-shot learning,

    A. Parnami and M. Lee, “Learning from few examples: A summary of approaches to few-shot learning,” arXiv preprint arXiv:2203.04291 , 2022

    2022 arXiv

  22. [30]

    Retrieval- augmented generation for knowledge-intensive nlp tasks,

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. K ¨uttler, M. Lewis, W.-t. Yih, T. Rockt ¨aschel et al. , “Retrieval- augmented generation for knowledge-intensive nlp tasks,” Advances in Neural Information Processing Systems , vol. 33, pp. 9459–9474, 2020

    2020

  23. [31]

    A three-step design pattern for specializing llms,

    G. Cloud, “A three-step design pattern for specializing llms,”

  24. [32]

    Unveiling llm evaluation focused on metrics: Challenges and solutions,

    T. Hu and X.-H. Zhou, “Unveiling llm evaluation focused on metrics: Challenges and solutions,” arXiv preprint arXiv:2404.09135 , 2024

    2024 arXiv

  25. [33]

    OpenAI, “Openai,” https://openai.com/, 2024, accessed: October 3, 2024

    2024

  26. [34]

    Sonarqube,

    SonarSource, “Sonarqube,” https://www.sonarsource.com/products/ sonarqube/, 2024, accessed: October 3, 2024

    2024

  27. [35]

    Team Eagle Inc

    Team Eagle Inc., “Team Eagle Inc.” Available online, 2024. [Online]. Available: https://www.team-eagle.ca/

    2024

  28. [36]

    Gpt-4 turbo and gpt-4o,

    OpenAI, “Gpt-4 turbo and gpt-4o,” https://platform.openai.com/docs/ models/gpt-4-turbo-and-gpt-4, 2024

    2024

  29. [2024]

    Available: https://cloud.google.com/blog/products/ ai-machine-learning/three-step-design-pattern-for-specializing-llms

    [Online]. Available: https://cloud.google.com/blog/products/ ai-machine-learning/three-step-design-pattern-for-specializing-llms

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.