Pith. sign in

REVIEW 5 major objections 4 minor 1 cited by

Developers' Experience with Generative AI -- First Insights from an Empirical Mixed-Methods Field Study

T0 review · 5 major / 4 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The way developers use an AI coding assistant—single mode, moderate, or combined—changes their speed, accuracy, and workload.

desk verdict Well-run field study with a genuinely new interaction-type comparison, but the headline causal claim is confounded by task difficulty and should not be read as causal. read the letter →

arxiv 2512.19926 v1 pith:QDOTC6TN submitted 2025-12-22 cs.HC

Charlotte Brandebusemeyer , Tobias Schimmer , Bert Arnrich This is my paper
classification cs.HC
keywords generativeAIcodingassistantsdeveloperexperiencehuman-computerinteractionmixed-methodsfieldstudyperceivedworkloadproductivitytypes
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the way developers interact with an AI coding assistant called Copilot—not just whether they use one—changes how efficiently they work, how accurate their output is, and how much workload they feel. In controlled sessions with professional developers at a firm, moderate use of either in-code suggestions or chat prompts shortened task time and reduced perceived workload compared to not using Copilot, while excessive use or combining both modes erased those benefits. The paper also claims that using only chat prompts improved task completion, and that developers' subjective workload ratings lined up with the objective behavioral data. These findings matter because they suggest that optimizing AI-assisted development means choosing the right interaction pattern, and that switching between patterns has its own cost. The study is presented as a demonstration that this kind of mixed-methods field design is feasible in a real company setting.

What carries the argument

The central machinery is the classification of each controlled-session task into one of four interaction types—no assistant use, in-code suggestions only, chat prompts only, or both—paired with counts of in-code and chat interactions as intensity measures. These are entered as fixed effects in mixed-effects models (with participants and tasks as random effects) that predict log task duration, task completion, and workload scores. The no-assistant condition serves as the baseline for pairwise comparisons. This structure is what carries the paper's claim that single-mode moderate use is beneficial while combined or excessive use is not, because it separates the effect of interaction type from

What would settle it

Add an expert-rated or no-assistant baseline difficulty score for each task to the mixed-effects model for task duration and task completion; if the 'both' category's null effect and the chat-only accuracy benefit survive after controlling for difficulty, the causal claim stands. A cleaner test: assign developers to identical tasks with randomly forced single-mode vs. forced-switch conditions and compare durations and completion rates.

Watch

Extended reading notes

Core claim

On its own terms, the central claim is that interaction type and interaction intensity causally affect efficiency, accuracy, and perceived workload: moderate use of either in-code suggestions or chat prompts improves efficiency and reduces workload compared to no Copilot use, while excessive or combined use lessens these benefits, and chat-only interaction improves task completion. The authors reach this through a four-day field study with controlled task sessions on the first and last days and an uncontrolled work period in between, combining screen, keyboard, and mouse recordings with questionnaires. They manually labeled the screen recordings to classify each task by interaction type and

Load-bearing premise

The load-bearing premise is that a task's interaction-type category is independent of its difficulty; if harder tasks push developers to combine modes or prompt more, then the reported penalties for combined or heavy use may be effects of difficulty rather than of the interaction mode itself.

Editorial extensions

If this is right

  • Developers and teams should treat AI usage as a policy variable: moderate single-mode use is the sweet spot, and encouraging simultaneous use of suggestions and chat may add switching overhead rather than help.
  • Efficiency gains from chat prompts are fragile—beyond roughly six chat prompts per task, average duration rises above the no-assistant baseline; in-code suggestions tolerate more interactions (around thirteen) before diminishing.
  • Chat-only interaction appears to help task completion, possibly because chat draws on wider code context; in-code suggestions alone did not significantly improve accuracy.
  • Perceived workload tracks objective task time in this data, so lightweight self-report instruments could serve as a proxy for behavioral efficiency in future firm-level evaluations.
  • Task category is a confound that cannot be ignored: debugging and summary tasks take longer, and coding and brainstorming tasks are more often completed, so AI-assistant evaluations should always be stratified by task type.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves the mechanism behind the combined-use penalty open; a natural extension is to analyze screen recordings for actual mode-switch events and test whether switch count, not mode combination, predicts longer duration and higher workload.
  • The interaction-type category may be endogenous: developers probably switch to a second mode when the first one fails, making 'both' a marker for harder tasks. A randomized design that fixes task difficulty, or a model with a difficulty covariate, would test whether combining modes is intrinsically costly or merely associated with difficulty.
  • The uncontrolled-period result—higher cognitive load along with higher perceived productivity—suggests a 'productive but effortful' pattern worth tracing with the recorded physiological signals, which the paper does not analyze here.
  • The numerical thresholds (six chat prompts, thirteen suggestions) come from a small, all-Java, same-IDE sample; a straightforward replication with another language or with junior developers would show whether the sweet spot is a general property of AI-assisted interaction or a local artifact.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper reports a field study with 22 professional developers at SAP, combining controlled sessions (with and without GitHub Copilot) and an uncontrolled workday period, using screen/mouse/keyboard recordings, wristband physiology, and questionnaires. The controlled-session analysis uses (generalized) linear mixed-effects models to relate Copilot interaction type (no use, in-code suggestions, chat, both) and interaction intensity (number of in-code suggestions/chat prompts) to log task duration, task completion, and raw NASA-TLX workload, with task category and participant/task random effects. The central claims are that moderate use of either in-code suggestions or chat improves efficiency and reduces perceived workload relative to no Copilot, while excessive or combined use lessens these benefits, and that chat-only use improves task completion. The uncontrolled period is analyzed descriptively, showing higher self-reported cognitive load and productivity on AI-involved tasks.

Significance. If the causal interpretation were supported, the paper would provide a practically important nuance to the productivity debate around AI coding assistants: interaction type and intensity matter, not just binary use. The study's strengths are real: it is a rare in-firm mixed-methods field study with professional developers, randomized task order, ~66 hours of manually labeled screen recordings, multimodal behavioral and subjective data, and a supplemental material with analysis details. The single-mode versus no-use comparisons are plausible and broadly consistent with prior work such as Peng et al. The paper is also explicit about several limitations, including task-ordering effects, keyboard differences, and the NASA-TLX scoring issue. However, the headline causal claims are not supported by the identification strategy, and several reported statistical results are weaker than the abstract suggests.

major comments (5)
  1. [Abstract; §5.1.1–5.1.2; §6] The headline claim that moderate use of a single interaction type improves efficiency/workload while excessive or combined use lessens benefits is causal, but interaction type and intensity are endogenous. §5.1.2 states that 'combined use of interaction types mostly indicated unsatisfying results from one mode, leading to a switch,' making the 'both' category a marker of encountered difficulty rather than an independent treatment. The mixed-effects model in §5.1 controls only for task category and post-task NASA-TLX; the latter is itself affected by the interaction and by difficulty. No independent task-difficulty measure is included, so the 'both' and high-count coefficients can reflect reverse causality even if the true causal effect is zero. In particular, the non-significant 'both' vs. no-Copilot duration contrast (estimate=0.223, SE=0.170, p=0.5559) is consistent with a difficulty-d
  2. [§5.1.2; Fig. 3] The 'moderate vs. excessive' thresholds (up to 13 in-code suggestions; up to 6 chat prompts) are read from raw per-count means in the same data used to state the claim. No confidence intervals, spline/quadratic term, or model comparison is reported for these cutoffs. As presented, the green/red split in Fig. 3 is post hoc and does not support the abstract's claim that excessive use 'lessens' benefits. Either fit a formal nonlinear interaction-intensity model or label these cutoffs clearly as exploratory and remove the causal wording.
  3. [§4.4.3; §5] Pooling both controlled sessions and both participant groups after a non-significant pre-test is not sufficient justification: with 22 participants, a null result does not establish absence of session or group effects. The mixed-effects models should include session (first/last day) and group (A/B) as fixed or random effects, or report equivalence bounds for their effects. Without this, the interaction-type estimates may absorb learning/fatigue across sessions or any baseline difference between groups.
  4. [§5.2.2] The task-completion GLMM is fitted despite complete separation in the data (all brainstorming tasks completed; §5.2.2). The resulting χ² and p-values for task categories—and any category-adjusted estimates of the chat effect—are unreliable, as the paper itself notes ('model instabilities'). Please use a bias-corrected (e.g., Firth) or exact method, or exclude the completely separated category and re-fit before drawing conclusions about chat improving accuracy.
  5. [§5.2.1] The accuracy benefit of chat rests on a marginal overall effect (χ²(3)=7.44, p_adjusted=0.0591) and a single Tukey contrast (p=0.0389). More importantly, the manuscript itself states that brainstorming and summary tasks could only be completed with either no Copilot or chat, not with in-code suggestions. Since task category strongly predicts completion (§5.2.2), the chat coefficient may be confounded by task type. Please present a sensitivity analysis restricted to code-related tasks (coding, debugging, testing, documentation) or otherwise address this category–interaction-type confounding.
minor comments (4)
  1. [§5.1] Including raw NASA-TLX as a covariate in the task-duration model is questionable because workload is measured after the task and is plausibly a mediator or a post-treatment outcome, not a pre-treatment confounder. Please clarify the assumed causal ordering or run the duration model without this covariate.
  2. [§5.2.1] Please clarify which p-values are multiplicity-adjusted. The chat estimate is reported as p=0.0078 and later as p=0.0389 from an exploratory Tukey test; the reader needs to know whether the first value is unadjusted and how it relates to the reported model-level p_adjusted=0.0591.
  3. [§4.2.1; §7] The paper mentions the German QWERTZ keyboard as a potential issue but argues it affects all conditions equally. This is plausible, but the argument would be stronger if keyboard-related metrics (e.g., key count) were reported or controlled for.
  4. [§5.3.3] The uncontrolled-period comparison of cognitive load and productivity uses paired non-parametric tests across tasks. This is acceptable as descriptive, but the effect sizes r=-0.8760 and r=-0.6788 seem unusually large for such noisy self-report data; please report the median ratings and the number of participants contributing tasks.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the paper is an empirical field study whose claims are statistical summaries of observed data, not derivations; the two self-citations are non-load-bearing.

full rationale

The paper contains no derivation chain that reduces to its own inputs. Its central claims (Abstract; §5.1.2; §5.2.1; §5.3.1) are empirical findings from mixed-effects models fit to recorded behavior and questionnaires: task duration, task completion, and NASA-TLX are outcome measures, while interaction type, interaction counts, task category, and workload are observed predictors. No quantity is defined in terms of another and then 'predicted'; no fitted parameter is renamed as an independent prediction. The 'moderate vs. excessive' thresholds in §5.1.2 ('with in-code showing consistent efficiency gains up to 13 interactions, and chat suggestions up to six interactions... Beyond these levels, task durations mostly exceeded the no-Copilot baseline') are descriptive readings of Figure 3 from the same data, presented as within-sample summaries rather than as out-of-sample predictions; this limits inferential strength but is not a constructional equivalence. The cited self-references are not load-bearing: [5] (the authors' Zenodo supplement) merely points to details of the model fits and post hoc tests, and [6] (the authors' prior wearables paper) appears only in a related-work sentence alongside [17, 36] to motivate mixed-methods designs. Neither is invoked as an external theorem or as the justification of the headline causal claim, so no uniqueness-imported or ansatz-smuggled-via-citation pattern exists. No known result is renamed as an organizing framework. The most substantive concern is validity, not circularity: interaction type and intensity are endogenous to task difficulty (the paper itself writes that 'Combined use of interaction types mostly indicated unsatisfying results from one mode, leading to a switch,' §5.1.2), which threatens the causal reading; Section 7 acknowledges this only in general terms ('one cannot control for all confounding factors'). Per the review rules, confounding and reverse causality are correctness risks and are weighed here, but they do not constitute the circularity patterns defined for this pass. Given the token presence of minor, non-load-bearing self-citation, a score of 1 is appropriate; the empirical claims stand on the collected data and on comparisons with independent prior work (e.g., Peng et al. [37]).

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

This is an empirical field study, so the ledger contains no invented entities. The main free parameters are the post-hoc efficiency cutoffs read off Figure 3. The axioms are domain assumptions about pooling, task comparability, labeling reliability, NASA-TLX correction, and hardware neutrality, all of which the central causal claims depend on.

free parameters (2)
  • In-code efficiency cutoff = 13 interactions
    Threshold in Figure 3 at which average task duration with in-code suggestions crosses the no-Copilot baseline (342.6s). This cutoff is read off the plot post hoc and is not a pre-registered parameter; no confidence interval is attached.
  • Chat efficiency cutoff = 6 prompts
    Threshold in Figure 3 at which average duration with chat prompts crosses the no-Copilot baseline. Post-hoc, descriptive.
assumptions (5)
  • ad hoc to paper Pooling of both controlled sessions and participant groups is valid for the main models; the pre-test found no significant differences in task duration between sessions or groups.
    Section 4.4.3. A null result is treated as equivalence; the main LMEs do not include session or group as fixed effects, so learning/order effects within Group A's no-Copilot then Copilot sequence may be absorbed into the interaction-type effect.
  • domain assumption The six task categories drawn from HumanEval-X are comparable in cognitive demand, as selected by code metrics (cognitive complexity, cyclomatic, Halstead, LOC, NBD).
    Section 4.1.5. The validation is that task categories did not differ in perceived workload, but they did differ in duration and completion, so comparability is only partially supported.
  • domain assumption Manual labeling of screen recordings (task duration, completion, Copilot interaction counts/types) is accurate and inter-rater consistent.
    Section 4.4.1. No inter-rater reliability statistic is reported; edge cases were reviewed by authors.
  • domain assumption The raw NASA-TLX average is a valid measure of perceived workload and the manual correction of the inversely scaled performance subscale did not bias results.
    Section 4.1.6.2 and Section 7. Participants noticed the inversion after several tasks; scores were corrected manually, possibly introducing inconsistency across tasks.
  • domain assumption The unfamiliar German QWERTZ keyboard affected all interaction-type conditions equally.
    Section 7. Authors argue no task or interaction-type bias; this is assumed, not tested.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Developers' Experience with Generative AI -- First Insights from an Empirical Mixed-Methods Field Study." pith.science (2026). https://pith.science/paper/QDOTC6TN

@misc{pith2026251219926,
  author       = {Pith},
  title        = {Pith review of: Developers' Experience with Generative AI -- First Insights from an Empirical Mixed-Methods Field Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QDOTC6TN}},
  note         = {Machine review of arXiv:2512.19926}
}
read the original abstract

With the rise of AI-powered coding assistants, firms and programmers are exploring how to optimize their interaction with them. Research has so far mainly focused on evaluating output quality and productivity gains, leaving aside the developers' experience during the interaction. In this study, we take a multimodal, developer-centered approach to gain insights into how professional developers experience the interaction with Generative AI (GenAI) in their natural work environment in a firm. The aim of this paper is (1) to demonstrate a feasible mixed-method study design with controlled and uncontrolled study periods within a firm setting, (2) to give first insights from complementary behavioral and subjective experience data on developers' interaction with GitHub Copilot and (3) to compare the impact of interaction types (no Copilot use, in-code suggestions, chat prompts or both in-code suggestions and chat prompts) on efficiency, accuracy and perceived workload whilst working on different task categories. Results of the controlled sessions in this study indicate that moderate use of either in-code suggestions or chat prompts improves efficiency (task duration) and reduces perceived workload compared to not using Copilot, while excessive or combined use lessens these benefits. Accuracy (task completion) profits from chat interaction. In general, subjective perception of workload aligns with objective behavioral data in this study. During the uncontrolled period of the study, both higher cognitive load and productivity were perceived when interacting with AI during everyday working tasks. This study motivates the use of comparable study designs, in e.g. workshop or hackathon settings, to evaluate GenAI tools holistically and realistically with a focus on the developers' experience.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Developer Experience with AI Coding Agents: HTTP Behavioral Signatures in Documentation Portals

    cs.SE 2026-04 unverdicted novelty 6.0 of 10

    AI coding agents produce identifiable HTTP behavioral signatures and compress multi-page navigation into one or two requests, rendering standard engagement metrics unreliable.

Reference graph

Works this paper leans on

52 extracted references · 1 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gal Bakal, Ali Dasdan, Yaniv Katz, Michael Kaufman, and Guy Levin. 2025. Experience with GitHub Copilot for Developer Productivity at Zoominfo. ArXiv (January 2025). https://doi.org/10.48550/arXiv.2501.13282

    arXiv doi:10.48550/arxiv.2501.13282 2025

  2. [2]

    James, and Nadia Polikarpova

    Shraddha Barke, Michael B. James, and Nadia Polikarpova. 2023. Grounded Copilot: How Programmers Interact with Code -Generating Models. Proceedings of the ACM on Programming Languages 7, OOPSLA1 (April 2023), 85–111. https://doi.org/10.1145/3586030

    doi:10.1145/3586030 2023

  3. [3]

    Alexander Barker. 2022. JNativeHook: Global Keyb oard and Mouse Listener for Java. Retrieved July 29, 2025 from https://github.com/kwhat/jnativehook

    2022

  4. [4]

    Berman, John Jonides, and Stephen Kaplan

    Marc G. Berman, John Jonides, and Stephen Kaplan. 2008. The Cognitive Benefits of Interacting With Nature. Psychol Sci 19, 12 (December 2008), 1207–1212. https://doi.org/10.1111/j.1467-9280.2008.02225.x

    arXiv 2008

  5. [5]

    Charlotte Brandebusemeyer, Tobias Schimmer, and Bert Arnrich. 2025. Supplemental Material for Developers’ Experience with Generative AI - First Insights from an Empirical Mixed -Methods Field Study. Zenodo. https://doi.org/10.5281/zenodo.17818081

    doi:10.5281/zenodo.17818081 2025

  6. [6]

    Charlotte Brandebusemeyer, Tobias Schimmer, and Bert Arnrich. 2025. Wearables to Measure Developer Experience at Work. In 2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE -SEIP), April 27, 2025. IEEE, 23 –33. https://doi.org/10.1109/ICSE-SEIP66354.2025.00008

    arXiv 2025

  7. [7]

    Ann Campbell

    G. Ann Campbell. 2023. {Cognitive Complexity} a new way of measuring understandability. SonarSource S.A

    2023

  8. [8]

    Mariana Coutinho, Lorena Marques, Anderson Santos, Marcio Dahia, Cesar França, and Ronnie de Souza Santos. 2024. The Role of Generative AI in Software Development Productivity: A Pilot Case Study. In Proceedings of the 1st ACM International Conference on AI -Powered Software (AIware 2024), July 10, 2024. ACM, New York, NY, USA, 131 –138. https://doi.org/1...

    arXiv 2024

Show all 52 references
  1. [9]

    Empatica Inc. 2025. EmbracePlus. Retrieved July 29, 2025 from https://www.empatica.com/en-eu/embraceplus/

    2025

  2. [10]

    Stefan Feuerriegel, Jochen Hartmann, Christian Janies ch, and Patrick Zschech. 2024. Generative AI. Business & Information Systems Engineering 66, 1 (February 2024), 111 –126. https://doi.org/10.1007/s12599 -023- 00834-7

    2024 doi

  3. [11]

    Nicole Forsgren, Eirini Kalliamvakou, Abi Noda, Michaela Greiler, Brian Houck, and Marg aret Anne Storey. 2023. DevEx in Action: A study of its tangible impacts. Queue 21, 6 (December 2023), 47 –77. https://doi.org/10.1145/3639443

    2023 doi

  4. [12]

    Nicole Forsgren and Abi Noda. 2025. Frictionless: 7 Steps to Remove Barriers, Unlock Value, and Outpace Your Competition in the AI Era . Shift Key Press

    2025

  5. [13]

    Nicole Forsgren, Margaret -Anne Storey, Chandra Maddila, Thomas Zimmermann, Brian Houck, and Jenna Butler. 2021. The SPACE of Developer Productivity: There’s more to it than you think. Queue 19, 1 (February 2021), 20–48. https://doi.org/10.1145/3454122.3454124

    2021

  6. [14]

    Free HD videos - no copyright. 2019. [4K] Autumn Leaves | Drone Aerial View | Free Stock Footage | Free HD Videos - No Copyright. Retrieved July 29, 2025 from https://www.youtube.com/watch?v=o - 7fsuJtEhk&list=PL4Gr5tOAPttKUXrXjulSCYa-L4xIwDyTi&index=13

    2019

  7. [15]

    Free HD videos - no copyright. 2019. [4K] Beauty Of Nature | Drone Aerial View | Free stock footage | Free HD Videos - No Copyright. Retrieved July 29, 2025 from https://www.youtube.com/watch?v=RK1RRVR9A2g&t=934s

    2019

  8. [16]

    Free HD videos - no copyright. 2020. Sunset Stock Footage | Royalty Free | Free HD Videos - no copyright. Retrieved July 29, 2025 from https://www.youtube.com/watch?v=TGLV2Fw -k5I&t=527s

    2020

  9. [17]

    Mül ler, Serap Yigit -Elliott, and Manuela Züger

    Thomas Fritz, Andrew Begel, Sebastian C. Mül ler, Serap Yigit -Elliott, and Manuela Züger. 2014. Using psycho -physiological measures to assess task difficulty in software development. In Proceedings - International Conference on Software Engineering, May 31, 2014. IEEE Comput...

    2014

  10. [18]

    Lewis R Goldberg. 1992. The development of markers for the Big -Five factor structure. Psychol Assess 4, 1 (1992), 26–42

    1992

  11. [19]

    Lewis R Goldberg. 2024. International Personality Item Pool: A Scientific Collaboratory fo r the Development of Advanced Measures of Personality Traits and Other Individual Differences. Retrieved July 29, 2025 from https://ipip.ori.org/new_ipip-50-item-scale.htm

    2024

  12. [20]

    Halstead

    Maurice H. Halstead. 1977. Elements of Software Science. Elsevier

    1977

  13. [21]

    Sandra G. Hart. 2006. Nasa -Task Load Index (NASA -TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 50, 9 (October 2006), 904 –908. https://doi.org/10.1177/154193120605000909

    2006 doi

  14. [22]

    Headfulness - Luke Horton. 2024. Calming 4 -7-8 Breathing (5 Minutes). Retrieved July 29, 2025 from https://www.youtube.com/watch?v=DAp3aiC57ZQ&t=17s

    2024

  15. [23]

    Haritha Khandabattu. 2025. The 2025 Hype Cycle for Artificial Intelligence Goes Beyond GenAI. Gartner. Retrieved September 13, 2025 from https://www.gartner.com/en/articles/hype-cycle-for-artificial- intelligence

    2025

  16. [24]

    Mansi Khemka and Brian Houck. 2024. Toward Effective AI Support for Developers. Commun ACM 67, 11 (November 2024), 42 –49. https://doi.org/10.1145/3690928

    2024 doi

  17. [25]

    Bas Leijdekkers. 2021. M etricsReloaded. Retrieved July 29, 2025 from https://github.com/BasLeijdekkers/MetricsReloaded

    2021

  18. [26]

    Gloria Mark, Daniela Gudith, and Ulrich Klocke. 2008. The cost of interrupted work: more speed and stress. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , April 06, 2008. ACM, New York, NY, USA, 107–110. https://doi.org/10.1145/1357054.1357072

    2008

  19. [27]

    Gloria Mark, Shamsi Iqbal, Mary Czerwinski, and Paul Johns. 2015. Focused, Aroused, but so Distractible: Temporal Perspectives on Mu ltitasking and Communications. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing , February 28...

    2015

  20. [28]

    T.J. McCabe. 1976. A Comple xity Measure. IEEE Transactions on Software Engineering SE-2, 4 (December 1976), 308 –320. https://doi.org/10.1109/TSE.1976.233837

    1976

  21. [29]

    LLC Meltytech. 2025. Shotcut Video Editor. Retrieved July 29, 2025 from https://www.shotcut.org/

    2025

  22. [30]

    Microsoft and LinkedIn. 2024. 2024 Work Trend Index Annual Report - AI at Work is Here. Now Comes the Hard Part. Retrieved July 29, 2025 from https://www.microsoft.com/en-us/worklab/work-trend-index/ai-at- work-is-here-now-comes-the-hard-part

    2024

  23. [31]

    Giovanni B. Moneta. 2021. On the conceptualization and measurement of flow. In Advances in Flow Research . Springer Science, New York, 31 –69. https://doi.org/10.1007/978-3-030-53468-4_2

    2021 doi

  24. [32]

    Desmarais, and Zhen Ming (Jack) Jiang

    Arghavan Moradi Dakhel, Vahid Majdinasab, Amin Nikanjam, Foutse Khomh, Michel C. Desmarais, and Zhen Ming (Jack) Jiang. 2023. GitHub Copilot AI pair programmer: Asset or Liability? Journal of Systems and Software 203, (September 2023). https://doi.org/10.1016/j.jss.2023.111734

    2023

  25. [33]

    Hussein Mozannar, Gagan Bansal, Adam Fourney, and Eric Horvitz. 2024. Reading Between the Lines: Modeling User Behavior and Costs in AI - Assisted Programming. In Proceedings of the CHI Conference on Huma n Factors in Computing Systems (CHI ’24) , May 11, 2024. ACM, Honolulu, ...

    2024

  26. [34]

    Nhan Nguyen and Sarah Nadi. 2022. An empirical evaluation of GitHub copilot’s code suggestions. In Proceedings of the 19th In ternational Conference on Mining Software Repositories (MSR ’22) , May 23, 2022. ACM, Pittsburgh, PA, USA, 1–5. https://doi.org/10.1145/3524842.3528470

    2022

  27. [35]

    OBS Project. 2025. OBS - Open Broadcaster Software. Retrieved July 29, 2025 from https://obsproject.com/

    2025

  28. [36]

    Norman Peitek, Annabelle Bergum, Maurice Rekrut, Jonas Mucke, Matthias Nadig, Chris Parnin, Janet Siegmund, and Sven Apel. 2022. Correlates of programmer efficacy and their link to experience: a combined EEG and eye-tracking study. In Proceedings of the 30th ACM Joint European...

    2022

  29. [37]

    Sida Peng, Eirini Kalliamvakou, Peter Cihon, and Mert Demirer. 2023. The Impact of AI on Developer Productivity: Evidence from GitHub Copilot. ArXiv (February 2023). https://doi.org/10.48550/arXiv.2302.06590

    2023 doi arXiv

  30. [38]

    Paige S Rutner, Bill C Hardgrave, and D Harrison Mcknight. 2008. Emotional Disso nance and the Information Technology Professional. MIS Quarterly 32, 3 (September 2008), 635 –652. https://doi.org/10.2307/25148859

    2008 doi

  31. [39]

    Safari Realms. 2024. Waterfalls | Nature | Relaxation | Free HD Videos - No Copyright Footages. Retrieved July 29, 2025 from https://www.youtube.com/watch?v=t3LA4Zx7p_I

    2024

  32. [40]

    Gordon, Carina Negreanu, Christian Poelitz, Sruti Srinivasa Ragavan, and Ben Zorn

    Advait Sarkar, Andrew D. Gordon, Carina Negreanu, Christian Poelitz, Sruti Srinivasa Ragavan, and Ben Zorn. 2022. What is it like to program with artificial intelligence? ArXiv (2022), 1 –27. https://doi.org/10.48550/arXiv.2208.06213

    2022 doi arXiv

  33. [41]

    Margaret-Anne Storey, T Zimmermann, C Bird, J Czerwonka, B Murphy, and E Kalliamvakou. 2019. Supplemental material for towards a theory of software developer job satisfaction and perceived productivity. Zenodo. Retrieved from https://zenodo.org/records/3451354#.XYUr -OdKjOQ

    2019

  34. [42]

    Viktoria Stray, Nils Brede Moe, Nivethika Ganeshan, and Simon Kobbenes

  35. [43]

    Laura Tacho. 2024. Introducing Core 4: The best way to measure and improve your product velocity. Retri eved July 29, 2025 from https://www.lennysnewsletter.com/p/introducing -core-4-the-best-way- to

    2024

  36. [44]

    https://doi.org/10.24251/HICSS.2025.883

    7381–7390. https://doi.org/10.24251/HICSS.2025.883

    2025 doi

  37. [45]

    Glassman

    Priyan Vaithilingam, Tianyi Zhang, and Elena L . Glassman. 2022. Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI ’22 Extended Abstrac...

    2022

  38. [46]

    Ningzhi Tang, Meng Chen, Zheng Ning, Aakash Bansal, Yu Huang, Collin McMillan, and Toby Jia -Jun Li. 2024. Developer Behaviors in Validating and Repairing LLM -Generated Code Using IDE and Eye Tracking. In 2024 IEEE Symposium on Visual Languages and Human -Centric Computing (V...

    2024

  39. [47]

    Burak Yetiştiren, Işık Özsoy, Miray Ayerdem, and Eray Tüzün. 2023. Evaluating the Code Quality of AI -Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT. ArXiv (October 2023). https://doi.org/10.48550/arXiv.2304.10778

    2023 doi arXiv

  40. [48]

    Burak Yetistiren, Isik Ozsoy, and Eray Tuzun. 2022. Assessing the quality of GitHub copilot’s code generation. In Proceedings of the 18th International Conference on Predictive Models and Data Analytics in Software Engineering (PROMISE ’22) , November 07, 2022. ACM, Singapore,...

    2022

  41. [49]

    Alice Li, Andrew Rice, Devon Rifkin, Shawn Simister, Ganesh Sittampalam, and Edward Aftandilian

    Albert Ziegler, Eirini Kalliamvakou, X. Alice Li, Andrew Rice, Devon Rifkin, Shawn Simister, Ganesh Sittampalam, and Edward Aftandilian. 2022. Productivity assessment of neural code completion. In MAPS 2022: Proceedings of the 6th ACM SIGPLAN International Symposium on Machine...

    2022

  42. [50]

    Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Lei Shen, Zihan W ang, Andi Wang, Yang Li, Teng Su, Zhilin Yang, and Jie Tang

  43. [2023]

    In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’23), August 06, 2023

    CodeGeeX: A Pre -Trained Model for Code Generation with Multilingual Benchmarking on HumanEval -X. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’23), August 06, 2023. ACM, Long Beach, CA, USA, 5673 –5684. https://doi.org/10.1145/...

    2023

  44. [2025]

    In Proceedings of the 58th ICSE SEIP 2026, Rio de Janeiro, Brazil Brandebusemeyer et al

    Generative AI and Developer Workflows: How GitHub Copilot and ChatGPT Influence Solo and Pair Programming. In Proceedings of the 58th ICSE SEIP 2026, Rio de Janeiro, Brazil Brandebusemeyer et al. Hawaii International Conference on System Sciences (HICSS ’25) , January 07,

    2026

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.