Pith. sign in

REVIEW 5 major objections 5 minor 43 references

The Impact of Generative AI on Code Expertise Models: An Exploratory Study

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Copied ChatGPT code, credited to AI instead of developers, changes GitHub truck-factor results in 73% of simulated runs.

desk verdict A useful first measurement of ChatGPT code copying, but the simulation that drives the reliability claim stretches that measurement too far. read the letter →

arxiv 2507.08160 v1 pith:IPNQQULE submitted 2025-07-10 cs.SE

Otávio Cury , Guilherme Avelino This is my paper
classification cs.SE
keywords generativeAIcodeexpertisetruckfactordegreeofauthorshipattributionGitHubminingChatGPTsharedlinkssoftwarerepositoryanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether the spread of code written by generative AI quietly undermines the models that identify who really knows a codebase. Using ChatGPT shared links embedded in GitHub source files, the authors measured how much generated code is pasted in verbatim (a mean of 39% of matched lines) and then simulated crediting that code to the AI rather than the developer. They find that Degree of Expertise values drop consistently, and that a Truck Factor algorithm built on those values changes in 73% of simulated runs, mostly by shrinking the factor. The central claim is that authorship-based expertise metrics, including the widely used Truck Factor, become less reliable as GenAI-written code enters development workflows.

What carries the argument

The carrying object is the Degree of Expertise (DOE) formula, which scores a developer's knowledge of a file from added lines, file authorship, file size, and days since last commit, plus Avelino's Truck Factor algorithm operating on those scores. The simulation's lever is the exact-match detector: code lines from ChatGPT conversations that appear verbatim in a shared-link commit are counted as copied, and a mean copy rate of 39% is then subtracted from the Adds variable of affected developers in simulated commits. DOE's line-level granularity is what makes the attribution deduction possible, since models based only on commit counts cannot be discounted this way. The argument runs from measured copy statistics to simulated authorship loss to altered expert rankings and Truck Factor values.

What would settle it

Track a group of developers who have pasted ChatGPT-generated code verbatim, later test their ability to fix or extend that exact code without AI help, and compare against developers who wrote equivalent code themselves; if comprehension is equal, the 39% authorship deduction is unjustified and the simulated impact would not materialize.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the Truck Factor is sensitive to expertise attribution changes caused by GenAI code integration, even when the underlying expertise score changes are small. The authors show that when a uniform 39% copy rate is deducted from affected developers' added lines across 10%, 20%, 30%, 40%, and 50% of files in 24 large GitHub projects, 87 of 120 computed Truck Factor values (73%) change, with a median decrease of two developers, and 85 runs (71%) reshuffle the ranking of key developers. Because these changes appear in the first 10% impact scenario for 12 of 21 affected projects, the paper argues that even limited GenAI adoption can disturb expertise metrics. The paper frames this as a reliability concern, not a demonstrated real-world failure: the study is exploratory and simulation-based.

Load-bearing premise

The load-bearing premise is Assumption 2: that developers who paste ChatGPT code verbatim learn less from it, so those lines should be stripped from their authorship; if pasted code still produces real learning, the simulated drop in expertise and the resulting Truck Factor changes would shrink.

Editorial extensions

If this is right

  • Even a 10% rate of affected files changes Truck Factor values or rankings in 12 of the 24 studied projects, so the effect is not confined to unrealistic levels of GenAI use.
  • Truck Factor reductions imply the algorithm sees knowledge as more concentrated in fewer developers when copied AI code is recognized, meaning projects appear more fragile.
  • Core developers lose slightly less expertise on average than peripheral developers (0.124 vs 0.126 DOE units), a statistically significant disparity.
  • Because authorship metrics feed reviewer recommendation, recruitment, and productivity measurement, those downstream uses inherit the same sensitivity.
  • The effect appears in small and large projects: among runs with original Truck Factor at or below 7, 49% changed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 39% mean copy rate is measured only for files that already contain a ChatGPT shared link; if developers also paste AI code without leaving links, real-world contamination could be higher than simulated, while if they adapt code more than exact matching captures, it could be lower.
  • A direct testable extension would apply the same simulation to other expertise models, since the paper argues that line-based models are needed and predicts commit-count-based models may be even more vulnerable because a single commit can contain thousands of AI-written lines.
  • Learning studies suggest pasted code may still produce some comprehension through reading and debugging, so the honest discount might be smaller than the full 39%; the reliability concern would weaken but not disappear.
  • If GenAI integration keeps growing, these findings imply that expertise metrics should be recalibrated or augmented with interaction signals, not just authorship counts.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This exploratory study investigates how GenAI-generated code, particularly code copied from ChatGPT shared links, might affect authorship-based developer expertise models and the Truck Factor. The authors mine GitHub source files containing ChatGPT shared links, retrieve the linked conversations, and measure exact-match lines between generated snippets and lines added in the associated commits. They report a mean copy rate of 39% across file–link pairs. They then simulate the impact of GenAI by deducting a uniform 39% copy rate from the added lines of developers in randomly selected subsets (10–50%) of files in 24 large open-source repositories, recomputing Degree of Expertise (DOE) values and Truck Factors. Their results show a small, consistent reduction in DOE values and changes in the Truck Factor in 73% of the simulated scenarios, leading them to conclude that current expertise metrics may become less reliable as GenAI integration grows.

Significance. If the simulation is accepted as a plausible model of GenAI-related authorship attribution, the paper makes a useful contribution by drawing attention to a concrete risk in widely used software-engineering metrics. The study is clearly described, uses a novel dataset of ChatGPT shared links, and provides a public artifact (Zenodo DOI), which supports reproducibility. The measured per-language copy rates (Table 2) are a useful empirical baseline. However, the strength of the central claim depends on an extrapolation from sparse, single-commit observations to uniform historical attribution across large repositories; this extrapolation is not yet validated, and the paper would benefit from additional sensitivity analyses and control conditions.

major comments (5)
  1. [§4.5, §5.1] The simulation applies a uniform 39% copy rate, which is the mean of the per-file–link-pair distribution, to every commit made by a developer on randomly selected files in large repositories. However, the empirical measurement in §4.4 is computed for a single commit where a shared link appears, in mostly small repositories with a median of one contributor (§4.2). This extrapolation is load-bearing for the reported 73% Truck Factor change (§5.3). Please justify why a per-file-link statistic obtained from small, mostly single-contributor projects should apply uniformly to every historical commit in large projects, or re-run the simulation using the per-language rates from Table 2 (which range from 32% to 59%) and possibly a distribution rather than a point value.
  2. [§4.5, Table 4] The random selection of affected files is not described with enough detail to assess stability: no random seed, no number of repetitions, and no variance or confidence intervals are reported. The Truck Factor is a deterministic function of which files are selected, so the numbers in Table 4 may be highly dependent on one particular random draw. Please report results over multiple runs (e.g., 100 repetitions with controlled seeds) and provide the distribution of Truck Factor changes and Kendall tau values, not just single point estimates.
  3. [§4.3, Assumption 1] Assumption 1 equates exact-match lines with code directly copied from ChatGPT. No validation of this matching heuristic is reported, such as a manual inspection sample or a comparison with lines from non-ChatGPT sources. Given that repositories contain boilerplate, common idioms, and code copied from other projects, exact matches may include false positives. The authors should quantify the precision of their matching approach or at least discuss this threat to the measured copy rate.
  4. [§6] The central conclusion that 'the Truck Factor is sensitive to changes in developers' expertise, raising concerns about its reliability when GenAI-generated code is involved' is partly tautological: reducing the Adds variable in Eq. (1) by a fixed proportion necessarily changes DOE and can change the Truck Factor. The unsupported step is equating that reduction with realistic GenAI use. To make the claim more convincing, the authors should add a control condition that removes a comparable amount of authorship noise (e.g., the same 39% deduction from randomly selected developers and files with no GenAI association) and show that the observed sensitivity is specific to the GenAI-copied-code scenario rather than a generic property of the metrics under any authorship perturbation.
  5. [§4.3, Assumption 2] Assumption 2 states that developers do not acquire the same level of knowledge from integrating copied ChatGPT-generated code. This assumption is essential because it justifies deducting the copied lines from authorship. The cited evidence comes largely from student and novice populations, and may not transfer to experienced open-source developers. The paper should explicitly discuss this external-validity threat and, ideally, model a range of knowledge-acquisition levels (e.g., partial deduction) to show how sensitive the conclusions are to this assumption.
minor comments (5)
  1. [§4.4] The manuscript reports 2,235 file–shared link pairs, while Table 1 lists 2,094 files and 1,666 links after filtering. Please clarify how pairs are counted (e.g., one pair per link occurrence in a file) to avoid apparent inconsistency.
  2. [§5.2] The Wilcoxon Signed-Rank Test results are reported only as p-values. Please include the test statistic and sample size, and note that p < 0.005 is a threshold, not a precise value.
  3. [§4.1] The phrase 'we manually filtered the results' would benefit from a brief description of the manual process, such as the number of researchers involved and whether there was any cross-checking, to support reproducibility.
  4. [§5.1] The statement 'the 100th percentile of our distribution is 326' is equivalent to the maximum; using 'maximum' would be clearer and more standard.
  5. [References] Reference [5] is missing a publication year; please complete the bibliographic information.

Circularity Check

0 steps flagged · score 2.0 of 10

Low circularity burden: DOE and Truck Factor are the authors' own prior tools used as fixed inputs, and the simulated impact is transparently imposed by the simulation design rather than presented as an empirical prediction.

full rationale

The paper's derivation chain has no load-bearing circular reduction. The empirical input is a copy-rate distribution measured from real ChatGPT shared links (Section 4.4; mean 39%, Table 2). This rate is then used in a deliberately defined simulation (Section 4.5): "in every commit made by that developer on these files, we deducted the uniform code copy rate of the added lines." The resulting DOE decrease is a mechanical consequence of Eq. (1), which contains ln(1+Adds); the paper itself acknowledges this in Section 6: "This outcome is expected, given that the simulation removed 39% of only one variable in the DOE model." Reporting the consequence of one's own simulation as a simulated consequence is not circularity, and the paper does not relabel the simulation as an empirical prediction. The Truck Factor results (73% of values changed, median decrease of 2) are nontrivial outputs of a deterministic algorithm under injected perturbation, not identities. The only self-citation usage is the selection of DOE [13] and the Truck Factor implementation [10,12]; these are treated as fixed tools with prior empirical support, not as results being proven here, so they do not constitute a self-citation chain that forces the conclusion. The weakest link, assuming the per-file-link 39% statistic applies uniformly to every historical commit in randomly selected files, is a construct-validity and extrapolation threat, not a circular derivation. No uniqueness theorem or ansatz is imported from the authors' prior work. The central claim is therefore not reduced to its inputs by construction; it is a scenario-based sensitivity exploration whose validity depends on the realism of the scenario, which is an external question rather than a circularity defect.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on two domain assumptions about copying and knowledge acquisition, an imported expertise model, and a simulation simplification that uses one fitted copy rate. No new entities are postulated.

free parameters (2)
  • uniform code copy rate = 0.39 (mean copied-code percentage)
    Computed from the 1,672 file-link pairs in Section 5.1; used as the fraction of added lines reattributed to GenAI in every simulation scenario (Sections 4.5 and 5.2).
  • DOE regression coefficients = intercept 5.28223, ln(1+Adds) 0.23173, FA 0.36151, ln(Size) -0.28761, ln(1+NumDays) -0.19421
    Fitted in prior work [13] and imported here; the simulated DOE reduction is computed directly from the Adds coefficient, so the exact magnitudes in Table 3 depend on these external fitted values.
assumptions (4)
  • domain assumption Exact matching lines between ChatGPT snippets and added lines are instances of code copied from ChatGPT.
    Stated as Assumption 1 in Section 4.3; if this fails, the copy rates and the simulated attribution loss are overestimated.
  • domain assumption Developers do not acquire the same level of knowledge from integrating copied ChatGPT-generated code as from writing it themselves.
    Stated as Assumption 2 in Section 4.3; this justifies subtracting authorship from developers and is the conceptual basis for interpreting the results as a reliability threat.
  • domain assumption The DOE model correctly measures developer expertise.
    The simulation uses DOE [13] and the modified Truck Factor [10,12] as ground truth; the study does not validate these models but takes them from prior work.
  • ad hoc to paper A single uniform copy rate and random file selection represent realistic GenAI usage patterns.
    Section 4.5 applies the mean 39% copy rate to all languages and randomly selects affected files; actual language-specific rates range from 32% to 59% (Table 2), and real usage is likely concentrated in certain files and developers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Impact of Generative AI on Code Expertise Models: An Exploratory Study." pith.science (2026). https://pith.science/paper/IPNQQULE

@misc{pith2026250708160,
  author       = {Pith},
  title        = {Pith review of: The Impact of Generative AI on Code Expertise Models: An Exploratory Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IPNQQULE}},
  note         = {Machine review of arXiv:2507.08160}
}
read the original abstract

Generative Artificial Intelligence (GenAI) tools for source code generation have significantly boosted productivity in software development. However, they also raise concerns, particularly the risk that developers may rely heavily on these tools, reducing their understanding of the generated code. We hypothesize that this loss of understanding may be reflected in source code knowledge models, which are used to identify developer expertise. In this work, we present an exploratory analysis of how a knowledge model and a Truck Factor algorithm built upon it can be affected by GenAI usage. To investigate this, we collected statistical data on the integration of ChatGPT-generated code into GitHub projects and simulated various scenarios by adjusting the degree of GenAI contribution. Our findings reveal that most scenarios led to measurable impacts, indicating the sensitivity of current expertise metrics. This suggests that as GenAI becomes more integrated into development workflows, the reliability of such metrics may decrease.

Figures

Figures reproduced from arXiv: 2507.08160 by the authors.

Figure 1
Figure 1. Overview of the methodology used to assess the impact of GenAI on a knowledge model. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Overview of the approach used to simulate the im [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 34 canonical work pages

  1. [1]

    Hervé Abdi. 2007. The Kendall rank correlation coefficient. Encyclopedia of measurement and statistics 2 (2007), 508–510

    work page 2007

  2. [2]

    Nuri Almarimi, Ali Ouni, Moataz Chouchen, and Mohamed Wiem Mkaouer. 2021. csDetector: an open source tool for community smells detection. In 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering . 1560–1564

    work page 2021

  3. [3]

    Daniel Atzberger, Nico Scordialo, Tim Cech, Willy Scheibel, Matthias Trapp, and Jürgen Döllner. 2022. CodeCV: Mining expertise of GitHub users from coding activities. In 2022 IEEE 22nd International Working Conference on Source Code Analysis and Manipulation (SCAM). IEEE, 143–147

    work page 2022

  4. [4]

    Guilherme Avelino, Leonardo Passos, Andre Hora, and Marco Tulio Valente. 2016. A novel approach for estimating truck factors. In 2016 IEEE 24th International Conference on Program Comprehension (ICPC) . IEEE, 1–10

    work page 2016

  5. [5]

    Guilherme Avelino, Marco Tulio Valente, and Andre Hora. [n. d.]. What is the Truck Factor of popular GitHub applications? A first assessment. PeerJ PrePrints 3 ([n. d.]), e1233v1

    work page

  6. [6]

    Moritz Beller, Amanda Park, Karim Nakad, Akshay Patel, Sarita Mohanty, Ford Garberson, Ian G Malone, Vaishali Garg, Henri Verroken, Andrew Kennedy, et al

    work page

  7. [7]

    Christian Bird, Denae Ford, Thomas Zimmermann, Nicole Forsgren, Eirini Kalliamvakou, Travis Lowdermilk, and Idan Gazit. 2023. Taking flight with copilot. Commun. ACM 66, 6 (2023), 56–62

    work page 2023

  8. [8]

    Michelle Brachman, Amina El-Ashry, Casey Dugan, and Werner Geyer. 2025. Current and Future Use of Large Language Models for Knowledge Work. arXiv preprint arXiv:2503.16774 (2025)

    arXiv 2025

Show all 43 references
  1. [9]

    Fabio Calefato, Marco Aurelio Gerosa, Giuseppe Iaffaldano, Filippo Lanubile, and Igor Steinmacher. 2022. Will you come back to contribute? Investigating the inactivity of OSS core developers in GitHub. Empirical Software Engineering 27, 3 (2022), 1–41

    2022

  2. [10]

    Otávio Cury and Guilherme Avelino. 2024. Knowledge Islands: Visualizing Developers Knowledge Concentration. In Simpósio Brasileiro de Engenharia de Software (SBES). SBC, 789–795

    2024

  3. [11]

    Otávio Cury and Guilherme Avelino. 2025. The Impact of Generative AI on Code Expertise Models: An Exploratory Study. doi:10.5281/zenodo.15334705

    2025 doi

  4. [12]

    Otávio Cury, Guilherme Avelino, Pedro Santos Neto, Marco Túlio Valente, and Ricardo Britto. 2024. Source code expert identification: Models and application. Information and Software Technology (2024), 107445

    2024

  5. [13]

    Otávio Cury, Guilherme Avelino, Pedro Santos Neto, Ricardo Britto, and Marco Túlio Valente. 2022. Identifying source code file experts. In 16th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement . 125–136

    2022

  6. [14]

    Paul Denny, James Prather, Brett A Becker, James Finnie-Ansley, Arto Hellas, Juho Leinonen, Andrew Luxton-Reilly, Brent N Reeves, Eddie Antonio Santos, and Sami Sarsa. 2024. Computing education in the era of generative AI.Commun. ACM 67, 2 (2024), 56–67

    2024

  7. [15]

    Thomas Dohmke, Marco Iansiti, and Greg Richards. 2023. Sea change in software development: Economic and productivity analysis of the ai-powered developer lifecycle. arXiv preprint arXiv:2306.15033 (2023)

    2023 arXiv

  8. [16]

    Neil A Ernst and Gabriele Bavota. 2022. Ai-driven development is here: Should you worry? IEEE Software 39, 2 (2022), 106–110

    2022

  9. [17]

    Mívian Ferreira, Marco Tulio Valente, and Kecia Ferreira. 2017. A comparison of three algorithms for computing truck factors. In2017 IEEE/ACM 25th International Conference on Program Comprehension (ICPC) . IEEE, 207–217

    2017

  10. [18]

    Thomas Fritz, Gail C Murphy, Emerson Murphy-Hill, Jingwen Ou, and Emily Hill. 2014. Degree-of-knowledge: Modeling a developer’s knowledge of code. ACM Transactions on Software Engineering and Methodology (TOSEM) 23, 2 (2014), 1–42

    2014

  11. [19]

    Adam M Gaweda, Collin F Lynch, Nathan Seamon, Gabriel Silva de Oliveira, and Alay Deliwa. 2020. Typing exercises as interactive worked examples for deliberate practice in cs courses. In Proceedings of the Twenty-Second Australasian Computing Education Conference. 105–113

    2020

  12. [20]

    Roberto Gozalo-Brizuela and Eduardo C Garrido-Merchán. 2023. A survey of Generative AI Applications. arXiv preprint arXiv:2306.02781 (2023)

    2023 arXiv

  13. [21]

    Balreet Grewal, Wentao Lu, Sarah Nadi, and Cor-Paul Bezemer. 2024. Analyzing Developer Use of ChatGPT Generated Code in Open Source GitHub Projects. In 2024 IEEE/ACM 21st International Conference on Mining Software Repositories (MSR). IEEE, 157–161

    2024

  14. [22]

    Christoph Hannebauer, Michael Patalas, Sebastian Stünkel, and Volker Gruhn

  15. [23]

    Huizi Hao, Kazi Amit Hasan, Hong Qin, Marcos Macedo, Yuan Tian, Steven HH Ding, and Ahmed E Hassan. 2024. An Empirical Study on Developers Shared Conversations with ChatGPT in GitHub Pull Requests and Issues. arXiv preprint arXiv:2403.10468 (2024)

    2024 arXiv

  16. [24]

    William Harding. 2025. AI Copilot Code Quality: Evaluating 2024’s Increased Defect Rate via Code Quality Metrics . White Paper. GitClear

    2025

  17. [25]

    Elgun Jabrayilzade, Mikhail Evtikhiev, Eray Tüzün, and Vladimir Kovalenko. 2022. Bus factor in practice. In 44th International Conference on Software Engineering: Software Engineering in Practice . 97–106

    2022

  18. [26]

    Kailun Jin, Chung-Yu Wang, Hung Viet Pham, and Hadi Hemmati. 2024. Can ChatGPT Support Developers? An Empirical Evaluation of Large Language Models for Code Generation. arXiv preprint arXiv:2402.11702 (2024)

    2024 arXiv

  19. [27]

    Majeed Kazemitabaar, Oliver Huang, Sangho Suh, Austin Z Henley, and Tovi Grossman. 2024. Exploring the Design Space of Cognitive Engagement Techniques with AI-Generated Code for Enhanced Learning. arXiv preprint arXiv:2410.08922 (2024)

    2024 arXiv

  20. [28]

    Boxuan Ma, Li Chen, and Shin’ichi Konomi. 2024. Enhancing Programming Edu- cation with ChatGPT: A Case Study on Student Perceptions and Interactions in a Python Course. In International Conference on Artificial Intelligence in Education . Springer, 113–126

    2024

  21. [29]

    Anh Nguyen-Duc, Beatriz Cabrero-Daniel, Adam Przybylek, Chetan Arora, Dron Khanna, Tomas Herda, Usman Rafiq, Jorge Melegati, Eduardo Guerra, Kai-Kristian Kemell, et al. 2023. Generative Artificial Intelligence for Software Engineering–A Research Agenda. arXiv preprint arXiv:23...

    2023 arXiv

  22. [30]

    Stack Overflow. 2024. Overflow: 2024 State of Development Survey. https://survey.stackoverflow.co/2024/ (2024)

    2024

  23. [31]

    Alan Peslak and Lisa Kovalchick. 2024. AI for coders: An analysis of the usage of ChatGPT and GitHub CoPilot. Issues in Information Systems 25, 4 (2024), 252–260

    2024

  24. [32]

    It’s Weird That it Knows What I Want

    James Prather, Brent N Reeves, Paul Denny, Brett A Becker, Juho Leinonen, Andrew Luxton-Reilly, Garrett Powell, James Finnie-Ansley, and Eddie Antonio Santos. 2023. “It’s Weird That it Knows What I Want”: Usability and Interactions with Copilot for Novice Programmers. ACM Tran...

    2023

  25. [33]

    James Prather, Brent N Reeves, Juho Leinonen, Stephen MacNeil, Arisoa S Randri- anasolo, Brett A Becker, Bailey Kimmel, Jared Wright, and Ben Briggs. 2024. The widening gap: The benefits and harms of generative ai for novice programmers. In Proceedings of the 2024 ACM Conferen...

    2024

  26. [34]

    Martin P Robillard. 2021. Turnover-induced knowledge loss in practice. InProceed- ings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering . 1292–1302

    2021

  27. [35]

    Daniel Russo. 2024. Navigating the complexity of generative ai adoption in software engineering. ACM Transactions on Software Engineering and Methodology (2024)

    2024

  28. [36]

    James Skripchuk, Neil Bennett, Jeffrey Zhang, Eric Li, and Thomas Price. 2023. Analysis of Novices’ Web-Based Help-Seeking Behavior While Programming. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1. 945–951

    2023

  29. [37]

    Tao Xiao, Christoph Treude, Hideaki Hata, and Kenichi Matsumoto. 2024. Devgpt: Studying developer-chatgpt conversations. In Proceedings of the 21st International Conference on Mining Software Repositories . 227–230

    2024

  30. [38]

    Ramazan Yilmaz and Fatma Gizem Karaoglan Yilmaz. 2023. Augmented intelli- gence in programming learning: Examining student views on the use of ChatGPT for programming learning. Computers in Human Behavior: Artificial Humans 1, 2 (2023), 100005

    2023

  31. [39]

    Beiqi Zhang, Peng Liang, Xiyu Zhou, Aakash Ahmad, and Muhammad Waseem

  32. [40]

    Albert Ziegler, Eirini Kalliamvakou, X Alice Li, Andrew Rice, Devon Rifkin, Shawn Simister, Ganesh Sittampalam, and Edward Aftandilian. 2024. Measuring GitHub Copilot’s Impact on Productivity. Commun. ACM 67, 3 (2024), 54–63

    2024

  33. [2016]

    In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering

    Automatically recommending code reviewers based on their expertise: An empirical comparison. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering . 99–110

  34. [2023]

    arXiv preprint arXiv:2309.05687 (2023)

    Demystifying practices, challenges and expected features of using github copilot. arXiv preprint arXiv:2309.05687 (2023)

    2023 arXiv

  35. [2025]

    arXiv preprint arXiv:2503.10977 (2025)

    What’s DAT? Three Case Studies of Measuring Software Development Productivity at Meta With Diff Authoring Time. arXiv preprint arXiv:2503.10977 (2025)

    2025 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.