Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Static Analysis as a Feedback Loop: Enhancing LLM-Generated Code Beyond Correctness

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Feeding static-analysis findings back into the prompt cuts security, readability, and reliability defects in LLM-generated Python code by 30–70 percentage points within ten iterations, while correctness improves only modestly.

desk verdict Promising empirical idea, but the published algorithm cannot execute as written—needs major revision and real artifacts before it is citable. read the letter →

arxiv 2508.14419 v1 pith:CPMJC2HA submitted 2025-08-20 cs.SE

Scott Blyth , Sherlock A. Licorish , Christoph Treude , Markus Wagner This is my paper
classification cs.SE
keywords LLMcodegenerationstaticanalysisfeedbackqualitybeyondcorrectnessPylintBanditpromptengineeringiterativerefinementPythonSecurityEval
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that code quality dimensions beyond functional correctness can be improved by an iterative loop in which static-analysis findings are fed back into an LLM prompt. The authors argue that correctness benchmarks like HumanEval measure only one facet of software quality, and that security, readability, reliability, and maintainability matter in practice. Their SelectIssues algorithm injects Bandit and Pylint issue descriptions at the offending lines, asks GPT-4o for a fix, and accepts an update only when a severity-weighted fitness score improves while tests still pass. On the 470-problem PythonSecurityEval dataset, the loop cut insecure snippets from over 40% to about 13%, convention violations from over 80% to about 11%, and reliability warnings from over 50% to about 11% within ten iterations; functional correctness moved much less. The result matters because it offers a cheap, execution-free mechanism for pushing generated code toward professional standards, and it quantifies the new defects that refinement itself can introduce.

What carries the argument

The load-bearing mechanism is SelectIssues, an iterative accept-if-better loop around a static-analysis feedback prompt. Each round, the algorithm selects I issues, weighting higher-severity security findings first, injects each issue's description into the code at the offending line using the format <description> <start issue> <code> <end issue>, asks GPT-4o for a revised snippet, and accepts the revision only if a severity-weighted fitness score—total issue severity for test-passing snippets, minus infinity for failing snippets—does not get worse. Quality dimensions are operationalized by mapping Bandit to security and Pylint categories to readability (Convention), reliability (Warning), m

What would settle it

Give experienced developers the original and refined PythonSecurityEval snippets without labels, ask them to rate readability and reliability, and compare those ratings to the changes in Pylint Convention and Warning counts; if the ratings do not track the flag reductions, the central quality claims would be falsified. A second check: hold out a random half of the benchmark and rerun SelectIssues with the Pylint-category mapping permuted; the headline reductions should not depend on that mapping.

Watch

Extended reading notes

Core claim

The paper claims that an LLM can be steered toward professional-grade code by closing a feedback loop with static analysis. The proposed SelectIssues prompt tags each Bandit or Pylint finding—description plus location inserted directly into the code—asks GPT-4o to produce a revised snippet, and keeps the revision only if a severity-weighted fitness score does not decrease; snippets that fail the benchmark's unit tests are scored as infinitely bad. On the 470-problem PythonSecurityEval benchmark, ten iterations of this loop reduced insecure snippets from over 40% to about 13%, Pylint convention (readability) violations from over 80% to about 11%, and warning (reliability) issues from over 50%

Load-bearing premise

The load-bearing premise is that Pylint's Convention, Warning, and Refactor messages stand in for readability, reliability, and maintainability; the authors themselves flag cases like W0641 where the mapping is questionable, and if the proxy is wrong the reported improvements are fewer linter flags rather than better code.

Editorial extensions

If this is right

  • Guided by static-analysis feedback, GPT-4o can reduce security, readability, and reliability issues in one benchmark to low-teen percentages, so static analysis can act as a practical reward signal for code generation.
  • Selecting 2–5 issues per iteration beats selecting one, and gains plateau by roughly iteration 3–4; ten iterations are enough for near-final results.
  • Functional correctness improves by at most about 11 percentage points and error-related Pylint violations by only 5–7 points, so test-passing ability is the hardest dimension for static-analysis feedback to move.
  • Refinement is not monotonic: readability problems recur, and about 5% of proposals introduce new security flaws, so any production use needs a fitness gate rather than blind acceptance.
  • The same static-analysis-driven loop can be run on a security benchmark yet improve all measured quality dimensions, implying the approach generalizes across quality aspects.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond this paper: the same issue-tag-plus-accept-if-better loop should transfer to other languages and linters (e.g., ESLint or SpotBugs), because nothing in the method depends on Python-specific internals beyond having line-located findings.
  • Beyond this paper: the 13% security figure counts vulnerabilities that Bandit can see; a manual exploitability review of the refined snippets would be needed to know whether real-world vulnerability rates fell by the same margin.
  • Beyond this paper: the selection weights in Table III are preliminary; varying the security weight downward would reveal whether the headline improvements are driven by issue prioritization or by the model's inherent ability to fix those issues.
  • Beyond this paper: because error-related issues barely move, pairing this loop with runtime error messages from failing tests, rather than only static findings, is a natural next step for improving correctness.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes an iterative prompting strategy, SelectIssues, that injects static analysis findings from Bandit and Pylint into LLM prompts to improve code quality beyond functional correctness. Using GPT-4o on the PythonSecurityEval benchmark, it reports large reductions in security issues (>40% to 13%), readability violations (>80% to 11%), and reliability warnings (>50% to 11%) within ten iterations, alongside modest correctness improvements. The paper also analyzes issue-introduction patterns and the effect of the IssuesSelected hyperparameter. The contribution is positioned as an algorithm plus empirical evidence for using static-analysis feedback in a multi-dimensional code-quality loop.

Significance. If the results are valid, the paper provides a practical, low-cost method for improving LLM-generated code along several quality dimensions simultaneously, and it extends prior security-focused feedback approaches to readability, reliability, and maintainability. The reported effect sizes are large and qualitatively consistent across tested configurations, and the authors make their artifacts and analysis available. However, the central algorithmic description is internally inconsistent, the quality measurement relies on an unvalidated mapping from static-analysis categories to quality attributes, and the absence of variance reporting limits the strength of the quantitative claims. The paper addresses an important problem and could be a useful contribution after substantial revision.

major comments (3)
  1. [Section III.B, Eq. (2) and Algorithm 1] The fitness function as defined is inconsistent with the loop guard and the reported experiments. Eq. (2) defines f(S) = -δ(S) for snippets passing tests (δ(S) ≥ 0, so f(S) ≤ 0) and f(S) = -∞ otherwise. Algorithm 1's loop guard is 'while i < maxiterations ∧ fitness(current) > 0', which is false for every possible snippet: passing snippets have non-positive fitness and failing snippets have -∞. Thus the algorithm as written cannot execute even one mutation, making the reported 10-iteration improvements impossible to reproduce. The sentence following Eq. (2) — 'In cases where both an initial and an improved snippet fail correctness tests, the snippet with the lower total severity is selected' — is also not representable under Eq. (2), since all failing snippets are equally -∞. Please provide the actual fitness/selection rule (e.g., a lexicographic objective: first pass tests, then minimize
  2. [Table I and Section VII] The paper's headline claims about readability and reliability improvements are based entirely on Pylint message categories, but the mapping from these categories to quality dimensions is not validated. For example, W0641 is treated as a reliability warning, yet the authors themselves note in Section VII that 'possibly unused variable' is not clearly a reliability issue. Similarly, Pylint Convention messages include trivial style issues such as trailing whitespace and line length, which are weak proxies for readability. Since the same static-analysis flags are used both as the feedback signal and as the measurement instrument, the reported reductions may simply reflect the model learning to suppress those flags rather than genuine improvements in the intended quality attributes. At minimum, a manual validation of a sample of issues and a sensitivity analysis of the mapping are needed to s
  3. [Section IV.B, Tables V–VII, and Figure 6] The quantitative comparison of IssuesSelected configurations is presented without any variance information. With a stochastic LLM, a single run per configuration cannot establish that differences such as 13.41% vs. 13.79% insecure snippets are meaningful, and the paper provides no confidence intervals, repeated trials, or statistical tests beyond Spearman correlations at the snippet level. This weakens RQ1.2 conclusions, including the statement that 'selecting 3, 4, or 5 issues per iteration... performs best' — notably, IssuesSelected=4 was never tested (only 1, 2, 3, 5, All). Please provide repeated-run statistics or temper the claims accordingly; at a minimum, the untested value of 4 should not be part of the recommendation.
minor comments (5)
  1. [Algorithm 1] Typographical inconsistency: 'fitness(Current)' uses an uppercase C in the acceptance condition. Also, the guard's interaction with the fitness definition should be clarified once Eq. (2) is corrected.
  2. [Table IV] The row 'Security N/A' is not an issue code and should be clarified or removed; if it aggregates multiple Bandit findings, a representative code or a separate note is needed.
  3. [Section IV.B, Figure 7] The 'probability of improvement' is not defined. Please state how improvement is measured (e.g., any decrease in total severity? a change in passing status?) and how the probability is estimated over the benchmark tasks.
  4. [Tables V–VII] Percentages are given to two decimal places without sample sizes. Since PythonSecurityEval has 470 problems, add the number of snippets that actually executed the refinement loop, as some tasks may fail all iterations or time out.
  5. [Section VI] The contribution sentence 'evolving LLMs’ for high-quality code generation' has a grammatical error ('LLMs’ for' should be 'LLMs for'). Also, 'reliability warnings reduced by up to 44%' is consistent with Table V but the abstract says '>50% to 11%'; clarify which figure is intended.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the feedback loop is self-contained and the reported reductions are empirical outcomes, not derived from the fitted objective.

full rationale

The paper's claims are empirical results from an iterative mutation/repair loop, not predictions derived from fitted inputs. Equation (2) defines the optimization objective as the negative weighted total severity for passing snippets, and Algorithm 1 accepts only proposals that do not decrease that objective; SelectIssues also injects the exact static-analysis issue descriptions into the prompt. This means the final severity is non-increasing by construction, and the same static-analysis tools serve as both feedback signal and measurement instrument. However, the specific reported numbers (e.g., insecure snippets dropping from >40% to 13%, convention issues from >80% to 11%) are not numerically forced by Eq. (2): they depend on whether GPT-4o actually generates proposals that reduce severity, which is an empirical contingency that the paper observes and reports with variation across configurations. The authors explicitly acknowledge in Section VII that the mapping from Pylint categories to quality dimensions (e.g., W0641 as a reliability issue) is imperfect, further indicating that they treat the mapping as an empirical choice rather than a derived theorem. Self-citations appear in background and related-work contexts only and are not load-bearing for the central claim. A separate, non-circularity issue is that the published Algorithm 1 and Eq. (2) are internally inconsistent: passing snippets have fitness <= 0, failing snippets have -infinity, so the loop guard fitness(current)>0 is never true and the described selection among failing snippets by severity is unrepresentable; this is a reproducibility/correctness defect, not a circular derivation.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the benchmark's test suites as a correctness oracle, the static analysis tools as valid quality measures, the mapping from Pylint categories to quality dimensions, and the hand-chosen severity weights. The authors explicitly flag all of these as imperfect in Section VII.

free parameters (2)
  • Issue severity weights = Security: HIGH=30, MEDIUM=20, LOW=10, UNDEFINED=10; Convention/Error/Warning/Refactor=3
    Hand-chosen via expert consensus (Section III-B, Table III); they define the fitness objective and the priority for issue selection, and the paper notes they are preliminary with no sensitivity analysis.
  • MaxIterations = 10
    Set because 'negligible improvement beyond this threshold based on preliminary experiments'; the preliminary experiments are not shown, and Figure 7 still shows a 20% improvement probability at iteration 10 for Select 1.
assumptions (4)
  • ad hoc to paper Pylint Convention messages measure readability, Warning messages measure reliability, Refactor messages measure maintainability, and Error messages measure functionality.
    Defined in Table I and used throughout the fitness function; the paper's own limitations section (Section VII) concedes the mapping is not clear-cut, e.g., W0641 'possibly unused variable' may not indicate a reliability problem.
  • domain assumption The PythonSecurityEval test suites are sufficient to determine functional correctness.
    Used in Eq. (2): any test failure makes fitness -infinity. The paper acknowledges in Section VII that passing tests does not guarantee functional correctness (test suite overfitting).
  • domain assumption Bandit and Pylint reports are valid and sufficient proxies for security and broader code quality.
    All quality measurements are based on these tools; the paper acknowledges false positives, and related work (Wadhwa et al.) found insecure code that passes Bandit and CodeQL checks.
  • domain assumption GPT-4o is representative of LLM code generation performance.
    Experiments use a single model; the title and abstract generalize to LLMs, but no other model is tested.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Static Analysis as a Feedback Loop: Enhancing LLM-Generated Code Beyond Correctness." pith.science (2026). https://pith.science/paper/CPMJC2HA

@misc{pith2026250814419,
  author       = {Pith},
  title        = {Pith review of: Static Analysis as a Feedback Loop: Enhancing LLM-Generated Code Beyond Correctness},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CPMJC2HA}},
  note         = {Machine review of arXiv:2508.14419}
}
read the original abstract

Large language models (LLMs) have demonstrated impressive capabilities in code generation, achieving high scores on benchmarks such as HumanEval and MBPP. However, these benchmarks primarily assess functional correctness and neglect broader dimensions of code quality, including security, reliability, readability, and maintainability. In this work, we systematically evaluate the ability of LLMs to generate high-quality code across multiple dimensions using the PythonSecurityEval benchmark. We introduce an iterative static analysis-driven prompting algorithm that leverages Bandit and Pylint to identify and resolve code quality issues. Our experiments with GPT-4o show substantial improvements: security issues reduced from >40% to 13%, readability violations from >80% to 11%, and reliability warnings from >50% to 11% within ten iterations. These results demonstrate that LLMs, when guided by static analysis feedback, can significantly enhance code quality beyond functional correctness.

Figures

Figures reproduced from arXiv: 2508.14419 by the authors.

Figure 1
Figure 1. Sample code generated by GPT-4o, illustrating readability issues [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Distribution of issues in GPT-4o-generated code snippets before [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Example prompt for Problem 3 in the PythonSecurityEval dataset, [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Proposed snippet for Problem 3 in the PythonSecurityEval [6] dataset. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 6
Figure 6. Figure 6: Mean total severity across different issue selection strategies. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 8
Figure 8. Figure 8: Comparison of initial vs. post-improvement issue distributions. [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Learning Project-wise Subsequent Code Edits via Interleaving Neural-based Induction and Tool-based Deduction

    cs.SE 2026-04 unverdicted novelty 5.0 of 10

    TRACE improves project-wise subsequent code editing by interleaving neural-based induction for semantic edits and tool-based deduction for syntactic edits.

Reference graph

Works this paper leans on

35 extracted references · 25 canonical work pages · cited by 1 Pith paper

  1. [1]

    Evaluating large language models trained on code,

    M. Chen, J. Tworek, H. Jun, Q. Yuan, and H. P. de Oliveira Pinto et al., “Evaluating large language models trained on code,” arXiv preprint arXiv:2107.03374, 2021

    arXiv 2021

  2. [2]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al. , “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023

    arXiv 2023

  3. [3]

    Gemini: a family of highly capable multimodal models,

    G. Team, R. Anil, S. Borgeaud, Y . Wu, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth et al., “Gemini: a family of highly capable multimodal models,” arXiv preprint arXiv:2312.11805 , 2023

    arXiv 2023

  4. [4]

    ISO/IEC 25010

    International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC), “ISO/IEC 25010.” [Online]. Available: https://iso25000.com/index.php/en/ iso-25000-standards/iso-25010

    work page

  5. [5]

    How have views on software quality differed over time? research and practice viewpoints,

    I. G. Ndukwe, S. A. Licorish, A. Tahir, and S. G. MacDonell, “How have views on software quality differed over time? research and practice viewpoints,” Journal of Systems and Software, vol. 195, p. 111524, 2023

    work page 2023

  6. [6]

    Can llms patch security issues?

    K. Alrashedy and A. Aljasser, “Can llms patch security issues?” arXiv preprint arXiv:2312.00024, 2023

    arXiv 2023

  7. [7]

    Lost at c: A user study on the security implications of large language model code assistants,

    G. Sandoval, H. Pearce, T. Nys, R. Karri, S. Garg, and B. Dolan-Gavitt, “Lost at c: A user study on the security implications of large language model code assistants,” in 32nd USENIX Security Symposium (USENIX Security), 2023, pp. 2205–2222

    work page 2023

  8. [8]

    Do users write more insecure code with ai assistants?

    N. Perry, M. Srivastava, D. Kumar, and D. Boneh, “Do users write more insecure code with ai assistants?” in ACM SIGSAC Conference on Computer and Communications Security , 2023, pp. 2785–2799

    work page 2023

Show all 35 references
  1. [9]

    Can llm replace stack overflow? a study on robustness and reliability of large language model code generation,

    L. Zhong and Z. Wang, “Can llm replace stack overflow? a study on robustness and reliability of large language model code generation,” in AAAI Conference on Artificial Intelligence , vol. 38, no. 19, 2024, pp. 21 841–21 849

    2024

  2. [10]

    Beyond correctness: Benchmarking multi-dimensional code generation for large language models,

    J. Zheng, B. Cao, Z. Ma, R. Pan, H. Lin, Y . Lu, X. Han, and L. Sun, “Beyond correctness: Benchmarking multi-dimensional code generation for large language models,” arXiv preprint arXiv:2407.11470 , 2024

    2024 arXiv

  3. [11]

    Understanding stack overflow code quality: A recommendation of caution,

    S. Meldrum, S. A. Licorish, C. A. Owen, and B. T. R. Savarimuthu, “Understanding stack overflow code quality: A recommendation of caution,” Science of Computer Programming, vol. 199, p. 102516, 2020

    2020

  4. [12]

    Common weakness enumeration

    “Common weakness enumeration.” [Online]. Available: https://cwe. mitre.org/

  5. [13]

    Large language models for software engineer- ing: A systematic literature review,

    X. Hou, Y . Zhao, Y . Liu, Z. Yang, K. Wang, L. Li, X. Luo, D. Lo, J. Grundy, and H. Wang, “Large language models for software engineer- ing: A systematic literature review,” arXiv preprint arXiv:2308.10620 , 2023

    2023 arXiv

  6. [14]

    Introducing the next generation of claude,

    Anthropic, “Introducing the next generation of claude,” https://www. anthropic.com/news/claude-3-family, accessed: 2024-09-11

    2024

  7. [15]

    Available: https://huggingface.co/datasets/ google-research-datasets/mbpp

    “mbpp.” [Online]. Available: https://huggingface.co/datasets/ google-research-datasets/mbpp

  8. [16]

    Claude 3.5 sonnet model card addendum

    Anthropic, “Claude 3.5 sonnet model card addendum.” [Online]. Available: https://www.cdn.anthropic.com/ fed9cc193a14b84131812372d8d5857f8f304c52/Model\ Card\ Claude\ 3\ Addendum.pdf

  9. [17]

    A survey on in-context learning,

    Q. Dong, L. Li, D. Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, J. Xu, and Z. Sui, “A survey on in-context learning,” arXiv preprint arXiv:2301.00234, 2022

    2022 arXiv

  10. [18]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhou et al. , “Chain-of-thought prompting elicits reasoning in large language models,” Advances in Neural Information Processing Systems , vol. 35, pp. 24 824–24 837, 2022

    2022

  11. [19]

    Debug like a human: A large language model debugger via verifying runtime execution step by step,

    L. Zhong, Z. Wang, and J. Shang, “Debug like a human: A large language model debugger via verifying runtime execution step by step,” in Findings of the Association for Computational Linguistics ACL 2024 , 2024, pp. 851–870

    2024

  12. [20]

    [Online]

    OpenAI. [Online]. Available: https://platform.openai.com/docs/ api-reference/chat/create

  13. [21]

    Optimizing large language model hyperparameters for code generation,

    C. Arora, A. I. Sayeed, S. Licorish, F. Wang, and C. Treude, “Optimizing large language model hyperparameters for code generation,” arXiv preprint arXiv:2408.10577, 2024

    2024 arXiv

  14. [22]

    On the capability of static code analysis to detect security vulnerabilities,

    K. Goseva-Popstojanova and A. Perhinschi, “On the capability of static code analysis to detect security vulnerabilities,” Information and Software Technology, vol. 68, pp. 18–33, 2015

    2015

  15. [23]

    [Online]

    Pylint. [Online]. Available: https://www.pylint.org/

  16. [24]

    Using static analysis to find bugs,

    N. Ayewah, W. Pugh, D. Hovemeyer, J. D. Morgenthaler, and J. Penix, “Using static analysis to find bugs,” IEEE Software, vol. 25, no. 5, pp. 22–29, 2008

    2008

  17. [25]

    How developers engage with static analysis tools in different contexts,

    C. Vassallo, S. Panichella, F. Palomba, S. Proksch, H. C. Gall, and A. Zaidman, “How developers engage with static analysis tools in different contexts,” Empirical Software Engineering, vol. 25, pp. 1419– 1457, 2020

    2020

  18. [26]

    Analyzing false positive source code vulnerabilities using static analysis tools,

    F. Cheirdari and G. Karabatis, “Analyzing false positive source code vulnerabilities using static analysis tools,” in IEEE International Con- ference on Big Data (Big Data) , 2018, pp. 4782–4788

    2018

  19. [27]

    Identifying and documenting false positive patterns generated by static code analysis tools,

    Z. P. Reynolds, A. B. Jayanth, U. Koc, A. A. Porter, R. R. Raje, and J. H. Hill, “Identifying and documenting false positive patterns generated by static code analysis tools,” in IEEE/ACM 4th International Workshop on Software Engineering Research and Industrial Practice (SER&...

    2017

  20. [28]

    Frustrated with code quality issues? llms can help!

    N. Wadhwa, J. Pradhan, A. Sonwane, S. P. Sahu, N. Natarajan, A. Kanade, S. Parthasarathy, and S. Rajamani, “Frustrated with code quality issues? llms can help!” arXiv preprint arXiv:2309.12938 , 2023

    2023 arXiv

  21. [29]

    Securityeval dataset: mining vulnerabil- ity examples to evaluate machine learning-based code generation tech- niques,

    M. L. Siddiq and J. C. Santos, “Securityeval dataset: mining vulnerabil- ity examples to evaluate machine learning-based code generation tech- niques,” in 1st International Workshop on Mining Software Repositories Applications for Privacy and Security (MSR4P&S) , 2022, pp. 29–33

    2022

  22. [30]

    Using metrics to evaluate software system maintainability,

    D. Coleman, D. Ash, B. Lowther, and P. Oman, “Using metrics to evaluate software system maintainability,” Computer, vol. 27, no. 8, pp. 44–49, 1994

    1994

  23. [31]

    Boswell and T

    D. Boswell and T. Foucher, The art of readable code. ” O’Reilly Media, Inc.”, 2011

    2011

  24. [32]

    Available: https://bandit.readthedocs.io/en/latest/man/bandit

    [Online]. Available: https://bandit.readthedocs.io/en/latest/man/bandit. html

  25. [33]

    [Online]

    OpenAI. [Online]. Available: https://github.com/openai/human-eval

  26. [34]

    Identifying test-suite-overfitted patches through test case generation,

    Q. Xin and S. P. Reiss, “Identifying test-suite-overfitted patches through test case generation,” in 26th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) . ACM, 2017, p. 226–236. [Online]. Available: https://doi.org/10.1145/3092703.3092718

    2017

  27. [35]

    Program transformation landscapes for automated program modification using gin,

    J. Petke, B. Alexander, E. T. Barr, A. E. Brownlee, M. Wagner, and D. R. White, “Program transformation landscapes for automated program modification using gin,” Empirical Software Engineering, vol. 28, no. 4, p. 104, 2023. 10

    2023

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.