Pith. sign in

REVIEW 3 major objections 5 minor 91 references

Agentic code attracts more bug fixes and introduces more security flaws after merge than human code, especially where projects skip review.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 14:42 UTC pith:7JXJCUA5

load-bearing objection Solid first longitudinal post-merge study of agentic code: overall maintenance looks similar, but corrective risk, security findings, and no-review association are the real signal. the 3 major comments →

arxiv 2607.09902 v1 pith:7JXJCUA5 submitted 2026-07-10 cs.SE cs.AI

Do These Violent Delights Have Violent Ends? Measuring the Post-Merge Fate of Agentic Code

Chunqiu Steven Xia , Courtney Miller This is my paper
classification cs.SE cs.AI
keywords agentic codingpost-merge maintenancecode survival analysiscorrective maintenancesoftware securitycode reviewLLM coding agentsempirical software engineering
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks what happens to AI-written (agentic) code after it is merged into real open-source projects, rather than stopping at whether the pull request was accepted. Across 182 repositories, the authors track individual lines of code over time, label why later commits change them, and measure the security and dependency findings those contributions introduce. Overall post-merge change rates for agentic and human lines look similar, but the mix is different: agentic lines are substantially more likely to be rewritten for corrective reasons, especially bug fixes, and agentic commits introduce more security weaknesses and dependency vulnerabilities per line changed. Within projects, a higher share of agentic code one month predicts more corrective work the next month, including corrective work done by humans. Across projects, the burden is largest where more pull requests merge without review. The authors argue that coding agents should be judged and designed not only for mergeable output, but for code that stays secure and maintainable after it ships.

Core claim

While overall post-merge maintenance rates of agentic and human lines are statistically similar, agentic contributions require significantly higher rates of corrective maintenance—driven mainly by bug fixes—and introduce more security weaknesses and dependency vulnerabilities. Agentic maintenance burden is associated with project characteristics: each 10 percentage-point increase in a project’s no-review merge rate is associated with roughly a 6% increase in agentic maintenance burden, and higher agentic code share predicts higher corrective commit rates the following month, including human corrective work.

What carries the argument

Line-level lifecycle tracking (each added line survives or is terminated by a later commit) paired with survival and competing-risks models for maintenance intent, commit-level static analysis (Semgrep, OSV-Scanner), and project-level meta-regression of agentic hazard ratios on repository characteristics.

Load-bearing premise

Authorship can be cleanly split into agentic versus human from commit name, email, and signature heuristics, even though unobserved AI help in the human baseline may shrink the measured gap.

What would settle it

A longitudinal comparison with ground-truth authorship (for example explicit agent co-author tags or developer-reported AI use) that found no elevated corrective termination risk or static-finding rates for agentic lines after the same repository and file-role controls would overturn the central claim.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Agent evaluations must measure post-merge durability, security, and corrective cost, not only merge rates or generation volume.
  • Protecting human review capacity matters more as agentic volume rises; high no-review rates amplify later agentic burden.
  • Tool builders should optimize for code that survives and needs fewer bug fixes, not only for mergeable diffs.
  • Growing agentic share can raise next-month corrective load for human developers as well as overall.
  • Weaker engineering safeguards (review, tests, technical debt) are the settings where agentic post-merge cost is most visible.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If unobserved AI assistance already contaminates the human baseline, the reported corrective and security gaps are likely lower bounds on the true agentic-human difference.
  • Metrics that reward generation volume without post-merge quality will systematically overstate productivity and undervalue review.
  • The generation-review velocity mismatch described in the discussion suggests hybrid AI-generation plus human-only review may become unstable as agentic share grows.
  • The same line-level survival and intent pipeline could test whether closed-source or safety-critical projects show larger or smaller post-merge gaps.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper presents a longitudinal empirical study of the post-merge fate of agentic versus human code across 182 open-source repositories (May 2025–May 2026). Using line-level survival analysis (Kaplan–Meier/Cox), Fine-Gray competing-risks models for maintenance intents, Poisson GLMMs for Semgrep/OSV findings, binomial panel models for lagged agentic code share, and a two-stage meta-regression of project-level hazard ratios, the authors report that overall termination rates are statistically similar (HR ≈ 1.11, n.s.), while agentic lines show elevated corrective and bug-fix risk (sHR/HR ≈ 1.45–1.51), agentic commits introduce more security and dependency findings per changed line (RR ≈ 1.10–1.51), corrective burden rises with prior agentic share, and higher no-review merge rates are associated with higher agentic maintenance burden (~6% per 10 pp). The central claim is that merge success is an incomplete quality signal and that agentic tools should be evaluated for post-merge durability, security, and maintainability.

Significance. If the results hold, the paper supplies the first large-scale, multi-outcome post-merge comparison of agentic and human contributions and shifts evaluation attention from pre-merge acceptance to lifecycle durability. Strengths include an appropriate suite of survival and mixed models for the data structure, repository stratification/clustering, exposure offsets, a validated intent classifier (κ = 0.94 on a 100-commit sample), public artifact, and explicit discussion of generation–review asymmetry with concrete implications for researchers, practitioners, and tool builders. The no-review association and the accumulation of human corrective work as agentic share grows are particularly actionable. The work is timely for cs.SE given rapid agentic adoption.

major comments (3)
  1. §III (Commit & PR Collection & Classification) and RQ1 Limitations: Authorship is assigned by name/email/signature heuristics against known agents, with all other commits treated as human. The paper correctly notes that unobserved AI assistance (IDE completions, pasted agent output) contaminates the human baseline and biases contrasts toward the null. Because the overall termination HR is already non-significant (1.11, CI includes 1) while corrective/bug-fix effects are the load-bearing positive claims, residual misclassification remains the softest point. A sensitivity analysis (e.g., restricting the human baseline to pre-agent eras or high-confidence pure-human subsets, or quantifying differential misclassification by intent/file role) is needed to secure the magnitude of the reported 1.45–1.51 effects.
  2. §VI (RQ3 meta-regression) and Table VI: The random-effects meta-regression reports I² = 100% and R² = 6.0%, with only no-review rate reaching significance among six covariates. With essentially all residual heterogeneity unexplained, the claim that project characteristics (especially no-review) explain cross-project variation in agentic maintenance burden is only weakly supported. The manuscript should either strengthen the stage-2 model (additional covariates, alternative operationalizations of review quality/test infrastructure) or substantially temper the interpretation that no-review is a robust structural driver rather than one of several weak correlates.
  3. §IV–V (RQ1c static findings and RQ2 panel): Semgrep/OSV findings and corrective commit rates are treated as defect/burden proxies, yet the paper acknowledges false positives and that not every termination is a quality problem. For the security claim in particular, a short validation (sample of high-severity findings, or comparison against a second scanner) would make the elevated RRs more credible as evidence of introduced weaknesses rather than tool noise. Likewise, the one-month lag in the panel models is coarse; a brief robustness check with alternative lags would better support the accumulation claim.
minor comments (5)
  1. Figure 2 caption contains residual drafting notes (“TODO: repo the same as the commit”, spacing instructions) that should be removed.
  2. Table I and the project-filter description: the star-percentile and size cutoffs are reasonable but should be justified more explicitly (or sensitivity-checked) so readers can assess selection toward mid-sized active projects.
  3. Clarify how “mixed” and “unknown” commits are handled after LLM reclassification and whether they are dropped or reassigned in the Fine-Gray and Cox analyses.
  4. In the Discussion, the generation–review asymmetry argument is insightful but occasionally moves beyond the measured results; a clearer separation of empirical findings from mechanistic interpretation would help.
  5. Minor consistency: abstract says “46% higher corrective” while body tables report HR/sHR ≈ 1.45–1.51; align the rounded percentages with the model estimates.

Circularity Check

0 steps flagged

No circularity: observational empirical study whose outcomes are measured independently of the authorship covariate and whose coefficients are estimated from data, not forced by definition or self-citation.

full rationale

This paper is a longitudinal observational study of post-merge line survival, maintenance intent, and static-analysis findings across 182 repositories. The focal contrast (agentic vs. human authorship) is obtained from commit-level heuristics (author name/email/signature matching known agents); the outcomes (line termination events, Fine-Gray/Cox hazards by maintenance class, Semgrep/OSV counts, panel corrective rates, meta-regression of project-level HRs) are measured by independent pipelines (git blame lifecycle tracking, rule+LLM intent classification, static scanners, SonarQube). Model coefficients (HR, sHR, RR, OR, meta-regression multipliers) are estimated from the observed data under standard survival/GLMM/meta-regression specifications; none is obtained by fitting a parameter to a quantity and then re-presenting that quantity as a prediction, nor by defining the outcome in terms of the predictor. There are no uniqueness theorems, ansatzes imported via self-citation, or renamings of known results that load-bear the central claims. Self-citations, if any, are ordinary related-work references and do not close a definitional loop. Measurement limitations (possible unobserved AI assistance in the human baseline, intent-label noise) affect validity of the contrast but do not constitute circular derivation. Score 0 is therefore the correct finding.

Axiom & Free-Parameter Ledger

3 free parameters · 6 axioms · 2 invented entities

Load-bearing content is empirical measurement under standard SE and biostatistics machinery, not free-parameter physics. The claim rests on domain assumptions about how authorship, maintenance, and defects are operationalized, plus sample construction choices. No new physical entities are introduced; invented constructs are measurement definitions (agentic line, terminal commit, maintenance class mapping).

free parameters (3)
  • Project inclusion thresholds (≥10 agentic and human PRs, ≥3 months agentic activity, ≥10 commits/month, star filters, si
    These hand-chosen filters define the 182-repo analysis population from 2,860 candidates and can change estimated burdens if relaxed; they are selection design parameters, not fitted physical constants.
  • One-month lag for agentic_code_share in panel models
    Temporal association strength depends on this chosen lag window; shorter/longer response windows are unmodeled by design.
  • Stage-1 HR stability filters (<100 tracked lines or <10 terminations dropped)
    Drops 5 projects before meta-regression; alters which project-level HRs enter the second stage.
axioms (6)
  • domain assumption Agentic vs human authorship is adequately recovered from commit author name, email, and known agent co-author signatures.
    Central contrast in all survival and static models; Methods §Commit & PR Collection and RQ1 limitations.
  • domain assumption Line modification/removal by a later commit is a valid event for post-merge maintenance burden (with intent disaggregation mitigating non-defect terminations).
    Defines the survival endpoint in lifecycle analysis and Cox/Fine-Gray models.
  • domain assumption Extended Swanson taxonomy plus operational-intent mapping (rules + MiniMax-M2.7 for mixed/unknown) correctly labels maintenance purpose.
    Underpins RQ1b/RQ1c composition and corrective claims; validated only on 100 manual labels (κ=0.94).
  • domain assumption Semgrep and OSV-Scanner findings are informative proxies for introduced security weaknesses and dependency vulnerabilities.
    Supports defect-introduction claims; tool false positives/negatives remain.
  • standard math Standard survival/competing-risks/GLMM/meta-regression assumptions hold after stated adjustments (stratification, clustering, OLRE, Knapp-Hartung).
    Inference machinery for HRs, sHRs, RRs, ORs, and project-characteristic associations.
  • ad hoc to paper No-review merge rate and related project metrics proxy engineering safeguards that affect agentic post-merge burden.
    RQ3 operationalization of six characteristics after exploratory/LASSO screening; associations not claimed as full causal explanation.
invented entities (2)
  • Agentic maintenance burden (project-level agentic-vs-human termination HR) independent evidence
    purpose: Summarize per-project excess post-merge maintenance of agentic lines for meta-regression against project characteristics.
    Defined as stage-1 Cox HR; standard statistical construct, not a new physical object.
  • Terminal commit / line termination event independent evidence
    purpose: Operationalize when an introduced line is modified or removed after merge.
    Measurement definition for lifecycle analysis via diff tracking and git blame.

pith-pipeline@v1.1.0-grok45 · 25324 in / 3744 out tokens · 35303 ms · 2026-07-14T14:42:22.712571+00:00 · methodology

0 comments
read the original abstract

Agentic coding tools are increasingly used to make autonomous repository-level changes to real-world projects. Prior work has largely evaluated these contributions at the pre-merge stage, through outcomes such as pull request acceptance and review effort. Far less is known about what happens to agentic code post-merge. Yet merge success alone does not reveal whether a contribution will remain stable or require bug fixes and other corrective maintenance downstream. We conduct a longitudinal empirical analysis of agentic and human contributions across 182 repositories, tracking their post-merge fate over time, characterizing the intent of subsequent modifications, and analyzing the defects and vulnerabilities they introduce. While the overall maintenance rates are similar, agentic contributions require significantly higher rates of corrective maintenance and introduce more security weaknesses and dependency vulnerabilities. We also find statistically significant evidence that agentic maintenance burden is associated with repository characteristics. In particular, each 10 percentage-point increase in a project's no-review rate is associated with roughly a 6% increase in agentic maintenance burden on average. As coding agents become pervasive in software development, our findings highlight the need to evaluate and design agentic tools not only to produce mergeable changes, but to produce contributions that remain secure and maintainable.

Figures

Figures reproduced from arXiv: 2607.09902 by Chunqiu Steven Xia, Courtney Miller.

Figure 1
Figure 1. Figure 1: CIF for corrective terminations. Dots represent time [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the data collection process. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Line-level termination maintenance class proportion. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 3
Figure 3. Figure 3: Post-merge maintenance rate. higher overall maintenance burden. Agentic lines are not significantly more or less likely to be terminated than human lines (HR = 1.11, 95% CI: 0.85–1.45, p = 0.45). Although the HR suggests an ∼11% higher termination rate for agentic lines, the confidence interval includes 1, so the difference is not statistically significant. This null result is not due to sparse data: in th… view at source ↗
Figure 5
Figure 5. Figure 5: CIF for bug fix. Dots represent time points of [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Static analysis findings per 100 commits. [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Predicted corrective maintenance rates as a function [PITH_FULL_IMAGE:figures/full_fig_p008_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

91 extracted references · 1 canonical work pages

  1. [1]

    Novet and J

    J. Novet and J. Vanian. (2025) Satya nadella says as much as 30% of Microsoft code is written by AI. CNBC. [Online]. Available: https://www.cnbc.com/2025/04/29/satya-nadella-says-as-much-as-30p ercent-of-microsoft-code-is-written-by-ai.html

    2025

  2. [2]

    C. Mauran. (2025) Mark zuckerberg wants AI to do half of Meta’s coding by 2026. Mashable. [Online]. Available: https: //mashable.com/article/llamacon-mark-zuckerberg-ai-writes-meta-code

    2025

  3. [3]

    H. Langley. (2026) Google says 75% of the company’s new code is AI- generated. Business Insider. [Online]. Available: https://www.businessin sider.com/google-ai-generated-code-75-gemini-agents-software-2026-4

    2026

  4. [4]

    J. Young. (2026) Introducing the state of AI coding 2026. New Relic. [Online]. Available: https://newrelic.com/blog/ai/state-of-ai-coding-202 6

    2026

  5. [5]

    E. Glover. (2026) What is tokenmaxxing? the AI workplace trend explained. Built In. [Online]. Available: https://builtin.com/articles/ai-t okenmaxxing

    2026

  6. [6]

    (2026) GitLab research reveals organizations are generating AI code faster than they can control it

    GitLab Inc. (2026) GitLab research reveals organizations are generating AI code faster than they can control it. GitLab. [Online]. Available: https://ir.gitlab.com/news/news-details/2026/GitLab-Research-Reveals -Organizations-Are-Generating-AI-Code-Faster-Than-They-Can-Contr ol-It/default.aspx

    2026

  7. [7]

    “an endless stream of ai slop

    S. Baltes, M. Cheong, and C. Treude, ““an endless stream of ai slop": The growing burden of ai-assisted software development,”arXiv preprint arXiv:2603.27249, 2026

    Pith/arXiv arXiv 2026

  8. [8]

    SWE-bench: Can language models resolve real-world github issues?

    C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. R. Narasimhan, “SWE-bench: Can language models resolve real-world github issues?” inThe Twelfth International Conference on Learning Representations, 2024

    2024

  9. [9]

    On the use of agentic coding: An empirical study of pull requests on github,

    M. Watanabe, H. Li, Y . Kashiwa, B. Reid, H. Iida, and A. E. Hassan, “On the use of agentic coding: An empirical study of pull requests on github,”ACM Transactions on Software Engineering and Methodology, 2025

    2025

  10. [10]

    How ai coding agents communicate: A study of pull request description characteristics and human review responses,

    K. Watanabe, R. Tsuchida, T. Monno, B. Huang, K. Yamasaki, Y . Fan, K. Shimari, and K. Matsumoto, “How ai coding agents communicate: A study of pull request description characteristics and human review responses,”arXiv preprint arXiv:2602.17084, 2026

    arXiv 2026

  11. [11]

    Evaluating large language models trained on code,

    M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y . Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Her...

    Pith/arXiv arXiv 2021

  12. [12]

    Github copilot your ai pair programmer,

    GitHub, “Github copilot your ai pair programmer,” https://github.com/f eatures/copilot, accessed Jun. 2026

    2026

  13. [13]

    Cursor: Ai coding agent,

    Cursor, “Cursor: Ai coding agent,” https://cursor.com/, accessed Jun. 2026

    2026

  14. [14]

    Claude code by anthropic|ai coding agent, terminal, ide,

    Anthropic, “Claude code by anthropic|ai coding agent, terminal, ide,” https://claude.com/product/claude-code, accessed Jun. 2026

    2026

  15. [15]

    Codex|ai coding partner from openai|openai,

    OpenAI, “Codex|ai coding partner from openai|openai,” https://open ai.com/codex/, accessed Jun. 2026

    2026

  16. [16]

    “maybe we need some more examples:

    C. Miller, R. Choudhuri, M. Ulloa, S. Haniyur, R. DeLine, M.-A. Storey, E. Murphy-Hill, C. Bird, and J. L. Butler, ““maybe we need some more examples:” individual and team drivers of developer genai tool use,” inProc. Int’l Conf. Software Engineering (ICSE), 2026. ACM Distinguished Paper Award

    2026

  17. [17]

    Octoverse: A new developer joins github every second as ai leads typescript to #1,

    GitHub, “Octoverse: A new developer joins github every second as ai leads typescript to #1,” GitHub, Tech. Rep., 2025. [Online]. Available: https://github.blog/news-insights/octoverse/octoverse-a-new-developer -joins-github-every-second-as-ai-leads-typescript-to-1/

    2025

  18. [18]

    Codex for every role, tool, and workflow,

    OpenAI, “Codex for every role, tool, and workflow,” https://openai.com /index/codex-for-every-role-tool-workflow/, 2026

    2026

  19. [19]

    Anthropic’s claude popularity with paying consumers is sky- rocketing,

    J. Bort, “Anthropic’s claude popularity with paying consumers is sky- rocketing,”TechCrunch, 2026, https://techcrunch.com/2026/03/28/anth ropics-claude-popularity-with-paying-consumers-is-skyrocketing/

    2026

  20. [20]

    An empirical evaluation of github copilot’s code suggestions,

    N. Nguyen and S. Nadi, “An empirical evaluation of github copilot’s code suggestions,” inProc. Conf. Mining Software Repositories (MSR), 2022, pp. 1–5

    2022

  21. [21]

    Assessing the quality of github copilot’s code generation,

    B. Yetistiren, I. Ozsoy, and E. Tuzun, “Assessing the quality of github copilot’s code generation,” inProceedings of the 18th international con- ference on predictive models and data analytics in software engineering, 2022, pp. 62–71

    2022

  22. [22]

    Quality evaluation of tabby coding assistant using real source code snippets,

    M. Borek and R. Nowak, “Quality evaluation of tabby coding assistant using real source code snippets,”arXiv preprint arXiv:2504.08650, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    Is github’s copilot as bad as humans at introducing vulnerabilities in code?

    O. Asare, M. Nagappan, and N. Asokan, “Is github’s copilot as bad as humans at introducing vulnerabilities in code?”Empirical Software Engineering, vol. 28, no. 6, p. 129, 2023

    2023

  24. [24]

    Asleep at the keyboard? assessing the security of github copilot’s code contribu- tions,

    H. Pearce, B. Ahmad, B. Tan, B. Dolan-Gavitt, and R. Karri, “Asleep at the keyboard? assessing the security of github copilot’s code contribu- tions,” in2022 IEEE Symposium on Security and Privacy (SP). IEEE Computer Society, 2022, pp. 754–768

    2022

  25. [25]

    Do users write more insecure code with ai assistants?

    N. Perry, M. Srivastava, D. Kumar, and D. Boneh, “Do users write more insecure code with ai assistants?” inProceedings of the 2023 ACM SIGSAC conference on computer and communications security, 2023

    2023

  26. [26]

    Echoes of ai: Investigating the downstream effects of ai assistants on software maintainability,

    M. Borg, D. Hewett, N. Hagatulah, N. Couderc, E. Söderberg, D. Gra- ham, U. Kini, and D. Farley, “Echoes of ai: Investigating the downstream effects of ai assistants on software maintainability,”Empirical Software Engineering, vol. 31, no. 6, p. 161, 2026

    2026

  27. [27]

    Investigating the smells of llm generated code,

    D. G. Paul, H. Zhu, and I. Bayley, “Investigating the smells of llm generated code,”arXiv preprint arXiv:2510.03029, 2025

    arXiv 2025

  28. [28]

    Impact of ai-tooling on the engineering workspace,

    L. Chretien and N. Albarran, “Impact of ai-tooling on the engineering workspace,”arXiv preprint arXiv:2406.07683, 2024

    Pith/arXiv arXiv 2024

  29. [29]

    Harnessing the potential of gen-ai coding assistants in public sector software development,

    K. K. Ng, L. Fauzi, L. Leow, and J. Ng, “Harnessing the potential of gen-ai coding assistants in public sector software development,”arXiv preprint arXiv:2409.17434, 2024

    Pith/arXiv arXiv 2024

  30. [30]

    How much does ai impact development speed? an enterprise-based randomized controlled trial,

    E. Paradis, K. Grey, Q. Madison, D. Nam, A. Macvean, V . Meimand, N. Zhang, B. Ferrari-Church, and S. Chandra, “How much does ai impact development speed? an enterprise-based randomized controlled trial,” in2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), 2025

    2025

  31. [31]

    The effects of github copilot on computing students’ programming effectiveness, efficiency, and processes in brownfield cod- ing tasks,

    M. I. H. Shihab, C. Hundhausen, A. Tariq, S. Haque, Y . Qiao, and B. W. Mulanda, “The effects of github copilot on computing students’ programming effectiveness, efficiency, and processes in brownfield cod- ing tasks,” inProceedings of the 2025 ACM Conference on International Computing Education Research V . 1, 2025, pp. 407–420

    2025

  32. [32]

    Productivity assessment of neural code completion,

    A. Ziegler, E. Kalliamvakou, X. A. Li, A. Rice, D. Rifkin, S. Simister, G. Sittampalam, and E. Aftandilian, “Productivity assessment of neural code completion,” inProceedings of the 6th ACM SIGPLAN interna- tional symposium on machine programming, 2022, pp. 21–29

    2022

  33. [33]

    Expectation vs. experi- ence: Evaluating the usability of code generation tools powered by large language models,

    P. Vaithilingam, T. Zhang, and E. L. Glassman, “Expectation vs. experi- ence: Evaluating the usability of code generation tools powered by large language models,” inCHI conference on human factors in computing systems extended abstracts, 2022, pp. 1–7

    2022

  34. [34]

    Is github copilot a substitute for human pair-programming? an empirical study,

    S. Imai, “Is github copilot a substitute for human pair-programming? an empirical study,” inProceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings, 2022

    2022

  35. [35]

    The impact of ai on developer productivity: Evidence from github copilot,

    S. Peng, E. Kalliamvakou, P. Cihon, and M. Demirer, “The impact of ai on developer productivity: Evidence from github copilot,”arXiv preprint arXiv:2302.06590, 2023

    Pith/arXiv arXiv 2023

  36. [36]

    The impact of generative ai on collaborative open-source software development: Evidence from github copilot,

    F. Song, A. Agarwal, and W. Wen, “The impact of generative ai on collaborative open-source software development: Evidence from github copilot,”arXiv preprint arXiv:2410.02091, 2024

    Pith/arXiv arXiv 2024

  37. [37]

    The effects of generative ai on high-skilled work: Evidence from three field experiments with software developers,

    K. Z. Cui, M. Demirer, S. Jaffe, L. Musolff, S. Peng, and T. Salz, “The effects of generative ai on high-skilled work: Evidence from three field experiments with software developers,”Management Science, 2026

    2026

  38. [38]

    Transforming software development: Evaluating the efficiency and challenges of github copilot in real-world projects,

    R. Pandey, P. Singh, R. Wei, and S. Shankar, “Transforming software development: Evaluating the efficiency and challenges of github copilot in real-world projects,”arXiv preprint arXiv:2406.17910, 2024

    Pith/arXiv arXiv 2024

  39. [39]

    Measuring the impact of early-2025 ai on experienced open-source developer productivity,

    J. Becker, N. Rush, E. Barnes, and D. Rein, “Measuring the impact of early-2025 ai on experienced open-source developer productivity,”arXiv preprint arXiv:2507.09089, 2025

    Pith/arXiv arXiv 2025

  40. [40]

    Ai-assisted programming decreases the productivity of experienced developers by increasing the technical debt and maintenance burden,

    F. Xu, P. K. Medappa, M. M. Tunc, M. Vroegindeweij, and J. C. Fransoo, “Ai-assisted programming decreases the productivity of experienced developers by increasing the technical debt and maintenance burden,” arXiv preprint arXiv:2510.10165, 2025

    arXiv 2025

  41. [41]

    Navigating (in) security of ai-generated code,

    S. H. Ambati, N. Ridley, E. Branca, and N. Stakhanova, “Navigating (in) security of ai-generated code,” in2024 IEEE international conference on cyber security and resilience (CSR). IEEE, 2024, pp. 1–8

    2024

  42. [42]

    Assessing the performance of ai-generated code: A case study on github copilot,

    S. Li, Y . Cheng, J. Chen, J. Xuan, S. He, and W. Shang, “Assessing the performance of ai-generated code: A case study on github copilot,” inProc. Int’l Symp. Software Reliability Engineering (ISSRE). IEEE, 2024, pp. 216–227

    2024

  43. [43]

    An empirical study of code smells in transformer-based code generation techniques,

    M. L. Siddiq, S. H. Majumder, M. R. Mim, S. Jajodia, and J. C. Santos, “An empirical study of code smells in transformer-based code generation techniques,” in2022 IEEE 22nd International Working Conference on Source Code Analysis and Manipulation (SCAM). IEEE, 2022

    2022

  44. [44]

    Security and quality in llm-generated code: A multi-language, multi-model analysis,

    M. F. Kharma, S. Choi, M. Alkhanafseh, and D. Mohaisen, “Security and quality in llm-generated code: A multi-language, multi-model analysis,” IEEE Transactions on Dependable and Secure Computing, 2026

    2026

  45. [45]

    Patchtrack: A comprehensive analysis of chatgpt’s influence on pull request outcomes,

    D. Ogenrwot and J. Businge, “Patchtrack: A comprehensive analysis of chatgpt’s influence on pull request outcomes,”Empirical Software Engineering, vol. 31, no. 5, p. 136, 2026

    2026

  46. [46]

    Debt behind the ai boom: A large-scale empirical study of ai-generated code in the wild,

    Y . Liu, R. Widyasari, Y . Zhao, I. C. Irsan, J. Chen, and D. Lo, “Debt behind the ai boom: A large-scale empirical study of ai-generated code in the wild,”arXiv preprint arXiv:2603.28592, 2026

    Pith/arXiv arXiv 2026

  47. [47]

    Speed at the cost of quality: How cursor ai increases short-term velocity and long-term complexity in open-source projects,

    H. He, C. Miller, S. Agarwal, C. Kästner, and B. Vasilescu, “Speed at the cost of quality: How cursor ai increases short-term velocity and long-term complexity in open-source projects,” inProc. Conf. Mining Software Repositories (MSR), 2026

    2026

  48. [48]

    The rise of ai teammates in software engineering (se) 3.0: How autonomous coding agents are reshaping software engineering,

    H. Li, H. Zhang, and A. E. Hassan, “The rise of ai teammates in software engineering (se) 3.0: How autonomous coding agents are reshaping software engineering,”arXiv preprint arXiv:2507.15003, 2025

    Pith/arXiv arXiv 2025

  49. [49]

    Agents in the wild - dashboard,

    C. Mürtz and M. N. Müller, “Agents in the wild - dashboard,” https://insights.logicstar.ai, 2025, interactive web dashboard. Code available at https://github.com/logic-star-ai/insights. [Online]. Available: https://doi.org/10.5281/zenodo.15846865

    work page doi:10.5281/zenodo.15846865 2025

  50. [50]

    Why are agentic pull requests merged or rejected? an empirical study,

    S. R. O. Peralta, F. Hoshi, H. Washizaki, N. Ubayashi, I. Kondo, Y . Higo, H. Mukai, N. Yoshida, K. Kusama, H. Tanakaet al., “Why are agentic pull requests merged or rejected? an empirical study,”arXiv preprint arXiv:2605.22534, 2026

    Pith/arXiv arXiv 2026

  51. [51]

    Why agentic-prs get rejected: A comparative study of coding agents,

    S. Nakashima, Y . Ishimoto, M. Kondo, S. Mclntosh, and Y . Kamei, “Why agentic-prs get rejected: A comparative study of coding agents,” arXiv preprint arXiv:2602.04226, 2026

    arXiv 2026

  52. [52]

    Understanding dominant themes in reviewing agentic ai-authored code,

    M. A. Haider and T. Zimmermann, “Understanding dominant themes in reviewing agentic ai-authored code,”arXiv preprint arXiv:2601.19287, 2026

    arXiv 2026

  53. [53]

    Early-stage prediction of review effort in ai-generated pull requests,

    D. S. D. Minh, H. T. Kiet, N. L. P. Quy, P. P. Hoa, T. C. Nguyen, N. D. H. Duong, and T. B. Tran, “Early-stage prediction of review effort in ai-generated pull requests,”arXiv preprint arXiv:2601.00753, 2026

    arXiv 2026

  54. [54]

    Do autonomous agents contribute test code? a study of tests in agentic pull requests,

    S. Haque, S. Ingale, and C. Csallner, “Do autonomous agents contribute test code? a study of tests in agentic pull requests,”arXiv preprint arXiv:2601.03556, 2026

    arXiv 2026

  55. [55]

    Agentic refactoring: An empirical study of ai coding agents,

    K. Horikawa, H. Li, Y . Kashiwa, B. Adams, H. Iida, and A. E. Hassan, “Agentic refactoring: An empirical study of ai coding agents,”arXiv preprint arXiv:2511.04824, 2025

    arXiv 2025

  56. [56]

    Ai ides or autonomous agents? measuring the impact of coding agents on software development,

    S. Agarwal, H. He, and B. Vasilescu, “Ai ides or autonomous agents? measuring the impact of coding agents on software development,”arXiv preprint arXiv:2601.13597, 2026

    arXiv 2026

  57. [57]

    Beyond bug fixes: An empirical investigation of post-merge code quality issues in agent-generated pull requests,

    S. T. Cynthia, A. Muttakin, and B. Roy, “Beyond bug fixes: An empirical investigation of post-merge code quality issues in agent-generated pull requests,”arXiv preprint arXiv:2601.20109, 2026

    arXiv 2026

  58. [58]

    Will it survive? deciphering the fate of ai- generated code in open source,

    M. Rahman and E. Shihab, “Will it survive? deciphering the fate of ai- generated code in open source,”arXiv preprint arXiv:2601.16809, 2026

    arXiv 2026

  59. [59]

    The dimensions of maintenance,

    E. B. Swanson, “The dimensions of maintenance,” inProc. Int’l Conf. Software Engineering (ICSE), 1976

    1976

  60. [60]

    Varga,Unraveling Software Maintenance and Evolution

    E. Varga,Unraveling Software Maintenance and Evolution. Springer, 2018

    2018

  61. [61]

    What do large commits tell us? a taxonomical study of large commits,

    A. Hindle, D. M. German, and R. Holt, “What do large commits tell us? a taxonomical study of large commits,” inProc. Conf. Mining Software Repositories (MSR), 2008

    2008

  62. [62]

    Large-scale intent analysis for identifying large-review-effort code changes,

    S. Wang, C. Bansal, and N. Nagappan, “Large-scale intent analysis for identifying large-review-effort code changes,”Information and Software Technology, vol. 130, p. 106408, 2021

    2021

  63. [63]

    A first look at conventional com- mits classification,

    Q. Zeng, Y . Zhang, Z. Qiu, and H. Liu, “A first look at conventional com- mits classification,” inProc. Int’l Conf. Software Engineering (ICSE), 2025

    2025

  64. [64]

    MiniMax M2.7,

    “MiniMax M2.7,” https://www.minimax.io/models/text/m27/, accessed Jun. 2026

    2026

  65. [65]

    A coefficient of agreement for nominal scales,

    J. Cohen, “A coefficient of agreement for nominal scales,”Educational and psychological measurement, vol. 20, no. 1, pp. 37–46, 1960

    1960

  66. [66]

    Semgrep app security platform,

    Semgrep, “Semgrep app security platform,” https://semgrep.dev/, accessed Jun. 2026

    2026

  67. [67]

    Osv-scanner,

    OSV-Scanner, “Osv-scanner,” https://google.github.io/osv-scanner/, accessed Jun. 2026

    2026

  68. [68]

    Sonarqube: Fight ai slop & verify ai code|sonar,

    SonarQube, “Sonarqube: Fight ai slop & verify ai code|sonar,” https: //www.sonarsource.com/products/sonarqube/, accessed Jun. 2026

    2026

  69. [69]

    Survival analysis,

    S. P. Jenkins, “Survival analysis,”Unpublished manuscript, Institute for Social and Economic Research, University of Essex, Colchester, UK, vol. 42, pp. 54–56, 2005

    2005

  70. [70]

    Nonparametric estimation from incomplete observations,

    E. L. Kaplan and P. Meier, “Nonparametric estimation from incomplete observations,”Journal of the American statistical association, 1958

    1958

  71. [71]

    D. R. Cox and D. Oakes,Analysis of survival data. CRC press, 1984

    1984

  72. [72]

    The analysis of categorical data from complex sample surveys: chi-squared tests for goodness of fit and independence in two-way tables,

    J. N. Rao and A. J. Scott, “The analysis of categorical data from complex sample surveys: chi-squared tests for goodness of fit and independence in two-way tables,”Journal of the American statistical association, vol. 76, no. 374, pp. 221–230, 1981

    1981

  73. [73]

    A proportional hazards model for the subdistribution of a competing risk,

    J. P. Fine and R. J. Gray, “A proportional hazards model for the subdistribution of a competing risk,”Journal of the American statistical association, vol. 94, no. 446, pp. 496–509, 1999

    1999

  74. [74]

    C. E. McCulloch, S. R. Searle, and J. M. Neuhaus,Generalized, linear, and mixed models. John Wiley & Sons, 2008

    2008

  75. [75]

    Principles of confounder selection,

    T. J. VanderWeele, “Principles of confounder selection,”European journal of epidemiology, vol. 34, no. 3, pp. 211–219, 2019

    2019

  76. [76]

    Competing risks regres- sion for stratified data,

    B. Zhou, A. Latouche, V . Rocha, and J. Fine, “Competing risks regres- sion for stratified data,”Biometrics, vol. 67, no. 2, pp. 661–670, 2011

    2011

  77. [77]

    Generalized linear mixed models: a practical guide for ecology and evolution,

    B. M. Bolker, M. E. Brooks, C. J. Clark, S. W. Geange, J. R. Poulsen, M. H. H. Stevens, and J.-S. S. White, “Generalized linear mixed models: a practical guide for ecology and evolution,”Trends in ecology & evolution, vol. 24, no. 3, pp. 127–135, 2009

    2009

  78. [78]

    Proportional hazards tests and diagnostics based on weighted residuals,

    P. M. Grambsch and T. M. Therneau, “Proportional hazards tests and diagnostics based on weighted residuals,”Biometrika, 1994

    1994

  79. [79]

    Importance of events per independent variable in proportional hazards regression analysis ii. accuracy and precision of regression estimates,

    P. Peduzzi, J. Concato, A. R. Feinstein, and T. R. Holford, “Importance of events per independent variable in proportional hazards regression analysis ii. accuracy and precision of regression estimates,”Journal of clinical epidemiology, vol. 48, no. 12, pp. 1503–1510, 1995

    1995

  80. [80]

    Dharma: residual diagnostics for hierarchical (multi- level/mixed) regression models,

    F. Hartig, “Dharma: residual diagnostics for hierarchical (multi- level/mixed) regression models,”CRAN: contributed packages, 2016

    2016

Showing first 80 references.