Human and Machine: How Software Engineers Perceive and Engage with AI-Assisted Code Reviews Compared to Their Peers

Adam Alami; Neil A. Ernst

REVIEW 3 major objections 4 minor 1 cited by

Human and Machine: How Software Engineers Perceive and Engage with AI-Assisted Code Reviews Compared to Their Peers

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LLM-assisted code reviews reduce emotional strain but increase cognitive load for engineers.

desk verdict A well-run interview study with a real finding, but the LLM condition is a single context-free prompt, so the headline 'lack of context / excessive detail' results need scoping. read the letter →

arxiv 2501.02092 v1 pith:ZBZO3GVX submitted 2025-01-03 cs.SE

Adam Alami , Neil A. Ernst This is my paper · ORCID

classification cs.SE

keywords codereviewlargelanguagemodelshuman-AIcollaborationengagementemotionalregulationcognitiveloadsense-makingsoftwareengineering

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that code review engagement is multi-dimensional—cognitive, emotional, and behavioral—and that swapping human peers for a large language model changes the balance among those dimensions. It claims that LLM-generated feedback is less emotionally taxing because its tone is consistently polite, so engineers need fewer coping and emotional-regulation strategies, but that the same feedback is more cognitively demanding because it is verbose and overloaded with details. It further claims that engineers evaluate and adopt feedback through the same sense-making process whether the reviewer is a peer or an LLM, yet adoption of LLM suggestions is gated by trust in the model and by the review's lack of codebase-specific context. The study matters because code review is a human-centric socio-technical process; knowing how AI shifts its emotional and cognitive costs guides how to design AI-human workflows that are actually adopted.

What carries the argument

The load-bearing framework is the three-dimensional engagement model (cognitive, emotional, and behavioral), coupled with a sense-making process that links feedback to adoption. A review has two components—content and delivery—and delivery style triggers emotional and behavioral responses, while content and clarity determine cognitive effort. The model adds a 'reviewer context' input: for peers it is seniority and familiarity; for LLMs it is the engineer's trust in the model and its known limitations, such as missing codebase context. This framework organizes the interview data and locates the study's central finding, the reversal between emotional ease and cognitive effort when the reviewer is a machine.

What would settle it

Run the same study design but feed the LLM the full repository context—surrounding modules, coding standards, and change history—before it reviews the code, and measure engineers' trust, adoption, and cognitive load; if the 'lack of context' complaint and the trust deficit vanish, those constraints are artifacts of the prompt rather than general properties of LLM reviews. A simpler check is to replicate with prompt variants to see whether verbosity, and hence cognitive load, is model- and prompt-dependent.

Watch

Extended reading notes

Core claim

The paper's central claim is that engagement in code review comprises cognitive, emotional, and behavioral dimensions, and that LLM-assisted review alters each dimension in a distinct way. Compared with peer review, LLM review lowers the emotional cost: engineers report less need for emotional regulation, coping mechanisms, and personal-value-guarding behaviors because the machine's delivery is uniformly positive and never 'picky' or 'brutal.' The trade-off is a higher cognitive load, since the LLM's lengthy, detailed output takes more time and mental effort to analyze than the more straightforward comments of human reviewers. Engineers still work through the same reflective sense-making process—going suggestion by suggestion, deciding what makes sense and what to adopt—but for LLM output that process is constrained by two reviewer-context factors: trust in the model's capabilities and the absence of codebase-specific context. The paper concludes that the introduction of LLM review brings an emotional-cognitive trade-off rather than a strict improvement or worsening.

Load-bearing premise

The whole comparison rests on one generic LLM prompt with no codebase background standing in for LLM-assisted code review; if the model had been given the surrounding repository context, the trust and context barriers the paper reports might shrink or disappear.

Editorial extensions

If this is right

Organizations adopting LLM-assisted code review can expect fewer emotionally charged interactions and less need for emotional regulation, but engineers may need help managing the heavier cognitive load of verbose AI feedback.
Because engineers use the same sense-making process for both sources, LLM suggestions will not be adopted on authority; they must survive the same line-by-line scrutiny as peer comments.
Trust and missing codebase context are the binding constraints on LLM feedback adoption, so tools that supply repository context or clearly flag their confidence levels could directly raise adoption.
The divergence in delivery-style preferences means a one-size-fits-all LLM review format will leave some engineers dissatisfied; personalization of tone and detail is a design requirement.
The emotional-cognitive trade-off implies that summarization or filtering of LLM output into concise, actionable comments could shift the balance substantially in favor of AI-assisted review.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The LLM condition in the study used a single generic prompt with no codebase context, so the reported 'lack of context' barrier is at least partly an artifact of prompt design; providing the LLM with repository context, coding standards, and change history would likely weaken or erase that constraint—an untested but direct consequence of the paper's own reviewer-context model.
A natural field experiment follows: hold the engineer and code fixed, vary only the amount of context given to the LLM, and measure trust, adoption rate, and perceived cognitive load; the prediction from the paper's model is that context-rich prompts raise trust and lower the perceived verbosity penalty.
The model may generalize beyond code review to other human-AI writing and analysis feedback loops, where content volume and delivery style trade off against emotional safety and cognitive effort.
Several interviewees already proposed a division of labor—LLM as exhaustive first-pass reviewer, human as contextual and relational judge—which the paper's trade-off structure supports as a likely steady-state arrangement for AI-human collaboration.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's the short version. The paper does something genuinely useful: it gives us a compact framework (cognitive, emotional, behavioral engagement plus sense-making) and a concrete, counterintuitive trade-off from 20 engineers who each got both peer reviews and a ChatGPT-4 review of their own code. The finding that LLM feedback reduces emotional regulation needs but increases cognitive load is clearly supported by the interview data. Methodologically it's careful: prescreening, saturation tracking, member checking with 19 of 20 participants, peer debriefing, and a shared package that includes the actual code and reviews. The claims are hedged appropriately.

The soft spot is the one the stress-test note points to, and it's real. The LLM condition used a single context-free prompt ("You are an expert of [language]. Provide a thorough review..."). So the two headline differences—excessive detail and lack of context—are exactly what that prompt would generate. The participants' human reviewers also had no codebase context, yet the engineers complained about missing context only from the LLM; that asymmetry suggests the finding is partly about expectations of AI rather than a real informational gap. The paper itself even acknowledges (Section III.B, citing [49]) that a prompt without context can generate unrealistic feedback, then proceeds to use one. That's a validity threat to the abstract's general claims about 'LLM-assisted review,' though not to the internal coherence of the qualitative analysis.

There are two smaller issues: the interview transcripts are not yet available (promised on acceptance), which limits independent audit; and the anonymous Prolific setting is a reasonable trade-off but weakens comparison to real workplace reviews. Neither is disqualifying.

Net: this is an honest, well-executed qualitative study that needs to be scoped. The engagement framework and the emotional/cognitive trade-off deserve a serious referee. I'd push the authors to either add a context-aware LLM condition or clearly frame the findings as about zero-context LLM reviews, and to make transcripts available. I'd accept it with major revision, not desk reject. I'd cite it if I'm writing about human-AI interaction in SE; it's a solid empirical anchor. I'd probably bring it to reading group, mostly because the prompt-artifact issue is a good methodological lesson.

Referee Report

3 major / 4 minor

Summary. The paper reports a qualitative interview study (20 software engineers recruited via Prolific with a two-phase pre-screening) in which each participant authored a code sample, received written peer reviews from two to three other participants, and then, during the interview, viewed a ChatGPT-4.0-generated review of the same code. The authors use these artifacts to investigate how engineers perceive and engage with LLM-assisted code reviews compared with human peer reviews. The central findings are that engagement in code review is multi-dimensional (cognitive, emotional, behavioral), that LLM-assisted reviews reduce the need for emotional regulation because of their consistently polite tone, that LLM-generated feedback sometimes imposes higher cognitive load because of its excessive detail, that engineers apply a similar sense-making process to feedback from both sources, and that adoption of LLM feedback is constrained by trust and by the LLM's lack of codebase context. The paper also derives practical implications about emotional intelligence training, feedback constructiveness, and personalization of AI feedback delivery.

Significance. If the findings hold, the paper makes a useful contribution to the emerging literature on human-AI collaboration in software engineering by moving beyond technical evaluations of LLM code review to the socio-emotional experience of engineers. The qualitative methodology is careful in several respects: iterative pre-screening using task-oriented questions, saturation monitoring, member checking with 19 of 20 participants, peer debriefing, and a shared documents package that includes code, reviews, prompts, and the interview guide. The claims in the findings are generally hedged with terms such as 'sometimes' and 'some engineers,' which is appropriate for an interview study. However, the comparison underpinning the headline claims rests on a single generic, context-free LLM prompt, and this operationalization substantially limits the external validity of the central comparative conclusions.

major comments (3)

[III.B] The LLM-assisted review condition is operationalized with the single prompt 'You are an expert of [the programming language]. Provide a thorough review of the attached code.' This prompt gives the model no repository context, no pull-request description, no tests, and no project conventions. The two headline differences in the findings—higher cognitive load from 'excessive details' (Section IV.A) and adoption constrained by the LLM's 'lack of context' (Section IV.C)—are exactly what such a zero-context prompt would be expected to produce. The abstract and Section IV frame these as properties of LLM-assisted review in general, but they may be properties of this particular prompt configuration. Because this is load-bearing for the central comparative claim, the paper should either temper the generalization or add robustness evidence, for example by varying the prompt to include repository context and showing whether the same cognitive-load and lack-of-context findings persist.
[IV.A, IV.C, and VII] The human peer reviewers in the study also lacked access to the author's full codebase, tests, and project history, yet participants are reported as attributing missing context specifically to the LLM ('ChatGPT is not completely aware of what the context of the code you are writing...' per P18). The paper does not address this asymmetry. It is possible that the 'lack of context' finding reflects participants' expectations about AI tools rather than an actual difference between the two review conditions. The limitations section (Section VII) acknowledges the artificial anonymous setting but does not grapple with this specific confound. The authors should discuss whether they probed participants about the peers' equally limited context, or should otherwise present the finding as an expectation-driven perception rather than a verified property of LLM reviews.
[VII] The limitations and trade-offs section is candid about anonymity and the artificial research setting, but it omits the most consequential limitation: the LLM review was produced with a single generic prompt and no codebase context, and this choice is what drives the 'lack of context' and 'excessive details' findings. Adding a statement that the results characterize one particular zero-context LLM review configuration, and that modern LLM review tools may operate with repository and PR metadata, is necessary to prevent over-generalization of the abstract's claims.

minor comments (4)

[Table I] In Table I, a few rows list the participant as their own reviewer (for example, P6 lists reviewers 'P6, P11, & P14' and P12 lists 'P4, P12, & P19'). This appears to be a typographical error in participant IDs and should be corrected, since it affects the transparency of the review-assignment description.
[Abstract and Section IV] The phrase 'the cognitive load sometimes is higher' is appropriately hedged, but the only direct supporting quotation in Section IV.A is from P14. Consider adding one or two additional participant excerpts to strengthen the evidence for the cognitive-load finding beyond a single voice.
[Throughout] There are several typos and formatting issues: 'hese engagement attributes' in the Introduction should be 'these engagement attributes'; 'whic' in Section VII should be 'which'; the table caption 'CHARACHTERESTICS' should be 'CHARACTERISTICS'. A careful proofreading pass is needed.
[I] The Introduction states that the authors used ChatGPT 'for its wide accessibility and familiarity among a diverse audience,' which is reasonable, but the choice of ChatGPT 4.0 over other LLM-based review tools should be mentioned in the limitations section, since different models and tools may differ substantially in verbosity, tone, and context handling.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the findings are induced from interview data; self-citations concern recruitment and background only, and the LLM-prompt limitation is an external-validity concern, not a circular one.

full rationale

This is an empirical interview study with no derivation chain. The central claims—engagement spans cognitive, emotional, and behavioral dimensions; LLM-assisted review reduces emotional load but can increase cognitive load; adoption is constrained by trust and lack of context—are supported by quoted participant accounts (e.g., P14, P8, P20) and by inductive coding (Sect. III.C), not by equations or fitted parameters. The authors' self-citations ([6], [7], [8], [26]) are to prior methods and background on pull-request governance and bot interaction; none is invoked to establish the target findings. The one potentially self-referential design choice is the zero-context ChatGPT prompt in Sect. III.B; the paper itself notes that 'a prompt that lacks specificity or context may generate unrealistic and unrelatable feedback,' and any resulting bias in the 'lack of context' or 'excessive detail' themes is a threat to external validity, not circularity. Hard rule 5 applies: concerns about the adequacy of the LLM configuration belong under correctness risk, not circularity. No prediction is fitted to its own outcome, no claimed result reduces by construction to an input, and no uniqueness theorem is imported from self-citation. Therefore score 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

This is a qualitative interview study, so the ledger captures domain assumptions rather than mathematical axioms. No free parameters or invented entities apply. The central findings depend on assumptions about participant genuineness, the representativeness of the LLM review, and self-report validity.

assumptions (4)

domain assumption The pre-screening process (programming task, critical-thinking question, and problem-solving scenario, plus ChatGPT- and manual evaluation) identifies genuine software engineers.
This relies on Alami et al. [8], a self-cited method paper; if the screening failed, the sample would not represent engineers.
domain assumption The ChatGPT-4.0 review generated with the generic prompt is representative of LLM-assisted code review as practitioners would encounter it.
The prompt (Section III.B) provides no codebase context, which may predetermine the 'lack of context' finding and does not reflect context-aware uses of LLMs.
domain assumption Participants' self-reports during interviews accurately reflect their real engagement and behavior in code review.
Qualitative interview data are retrospective and socially desirable; the authors use member checking (Section VI) to mitigate this.
domain assumption Data saturation was reached with 20 interviewees.
The authors monitored saturation iteratively (Section III.C), but the sample size is small and still limited to Prolific-recruited engineers.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Human and Machine: How Software Engineers Perceive and Engage with AI-Assisted Code Reviews Compared to Their Peers." pith.science (2026). https://pith.science/paper/ZBZO3GVX

@misc{pith2026250102092,
  author       = {Pith},
  title        = {Pith review of: Human and Machine: How Software Engineers Perceive and Engage with AI-Assisted Code Reviews Compared to Their Peers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZBZO3GVX}},
  note         = {Machine review of arXiv:2501.02092}
}

read the original abstract

The integration of artificial intelligence (AI) continues to increase and evolve, including in software engineering (SE). This integration involves processes traditionally entrusted to humans, such as coding. However, the impact on socio-technical processes like code review remains underexplored. In this interview-based study (20 interviewees), we investigate how software engineers perceive and engage with Large Language Model (LLM)-assisted code reviews compared to human peer-led reviews. In this inherently human-centric process, we aim to understand how software engineers navigate the introduction of AI into collaborative workflows. We found that engagement in code review is multi-dimensional, spanning cognitive, emotional, and behavioral dimensions. The introduction of LLM-assisted review impacts some of these attributes. For example, there is less need for emotional regulation and coping mechanisms when dealing with an LLM compared to peers. However, the cognitive load sometimes is higher in dealing with LLM-generated feedback due to its excessive details. Software engineers use a similar sense-making process to evaluate and adopt feedback suggestions from their peers and the LLM. However, the LLM feedback adoption is constrained by trust and lack of context in the review. Our findings contribute to a deeper understanding of how AI tools are impacting SE socio-technical processes and provide insights into the future of AI-human collaboration in SE practices.

Figures

Figures reproduced from arXiv: 2501.02092 by the authors.

**Figure 1.** An abstract presentation of the findings, capturing engagement and how engineers respond to feedback. The line from [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Rethinking Code Review Workflows with LLM Assistance: An Empirical Study
cs.SE 2025-05 conditional novelty 5.0 of 10

In a real-world experiment, developers preferred an AI-led code review mode over an interactive assistant, especially for large or unfamiliar pull requests.

Reference graph

Works this paper leans on

82 extracted references · 64 canonical work pages · cited by 1 Pith paper

[49]

Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing,

P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, “Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing,” ACM Computing Surveys, vol. 55, no. 9, pp. 1–35, 2023

2023
[1]

Github copilot · your ai pair programmer · github,

“Github copilot · your ai pair programmer · github,” https://github.com/ features/copilot/, (Accessed on 08/27/2024)

2024
[2]

Openai codex | openai,

“Openai codex | openai,” https://openai.com/index/openai-codex/, (Ac- cessed on 08/27/2024)

2024
[3]

Tabnine ai code assistant | private, personalized, protected,

“Tabnine ai code assistant | private, personalized, protected,” https:// www.tabnine.com/, (Accessed on 08/27/2024)

work page 2024
[4]

Octoverse: The state of open source and rise of ai in 2023 - the github blog,

“Octoverse: The state of open source and rise of ai in 2023 - the github blog,” https://github.blog/news-insights/research/ the-state-of-open-source-and-ai/, 2023, (Accessed on 08/27/2024)

work page 2023
[5]

Information seeking using AI assistants,

E. Al Haque, C. Brown, T. D. LaToza, and B. Johnson, “Information seeking using AI assistants,” 2024. [Online]. Available: https: //arxiv.org/abs/2408.04032

arXiv 2024
[6]

How do foss communities decide to accept pull requests?

A. Alami, M. L. Cohn, and A. W ˛ aisowski, “How do foss communities decide to accept pull requests?” in Proceedings of the 24th International Conference on Evaluation and Assessment in Software Engineering , 2020, pp. 220–229

work page 2020
[7]

Pull request gov- ernance in open source communities,

A. Alami, R. Pardo, M. L. Cohn, and A. W ˛ asowski, “Pull request gov- ernance in open source communities,” IEEE Transactions on Software Engineering, vol. 48, no. 12, pp. 4838–4856, 2021

work page 2021

Show all 82 references

[8]

Are you a real software engineer? best practices in online recruitment for software engineering studies,

A. Alami, M. Zahedi, and N. Ernst, “Are you a real software engineer? best practices in online recruitment for software engineering studies,” in Proceedings of the 1st IEEE/ACM International Workshop on Method- ological Issues with Empirical Studies in Software Engineering , 2...

2024
[9]

Data saturation: The mysterious step in grounded theory methodology,

K. M. Aldiabat and C.-L. Le Navenec, “Data saturation: The mysterious step in grounded theory methodology,” The qualitative report , vol. 23, no. 1, pp. 245–261, 2018

2018
[10]

“collaborating

C. Anthony, B. A. Bechky, and A.-L. Fayard, ““collaborating” with ai: Taking a system view to explore the future of work,” Organization Science, vol. 34, no. 5, pp. 1672–1694, 2023

2023
[11]

The bar-on model of emotional-social intelligence (esi) 1,

R. Bar-On, “The bar-on model of emotional-social intelligence (esi) 1,” Psicothema, pp. 13–25, 2006

2006
[12]

Socio-technical systems: From design methods to systems engineering,

G. Baxter and I. Sommerville, “Socio-technical systems: From design methods to systems engineering,” Interacting with computers , vol. 23, no. 1, pp. 4–17, 2011

2011
[13]

Min- ing email social networks,

C. Bird, A. Gourley, P. Devanbu, M. Gertz, and A. Swaminathan, “Min- ing email social networks,” in Proceedings of the 2006 international workshop on Mining software repositories , 2006, pp. 137–143

2006
[14]

Member check- ing: a tool to enhance trustworthiness or merely a nod to validation?

L. Birt, S. Scott, D. Cavers, C. Campbell, and F. Walter, “Member check- ing: a tool to enhance trustworthiness or merely a nod to validation?” Qualitative health research, vol. 26, no. 13, pp. 1802–1811, 2016

2016
[15]

Naturalistic inquiry and the saturation concept: a research note,

G. A. Bowen, “Naturalistic inquiry and the saturation concept: a research note,” Qualitative research, vol. 8, no. 1, pp. 137–152, 2008

2008
[16]

Language models are few-shot learners,

T. B. Brown, “Language models are few-shot learners,” arXiv preprint arXiv:2005.14165, 2020

2005 arXiv
[17]

Differing perceptions in the feedback process,

D. Carless, “Differing perceptions in the feedback process,” Studies in higher education, vol. 31, no. 2, pp. 219–233, 2006

2006
[18]

A study of emotions in requirements engineering,

R. Colomo-Palacios, A. Hernández-López, Á. García-Crespo, and P. Soto-Acosta, “A study of emotions in requirements engineering,” in Organizational, Business, and Technological Aspects of the Knowledge Society: Third World Summit on the Knowledge Society, WSKS 2010, Corfu, Gree...

2010
[19]

Development of a new resilience scale: The connor-davidson resilience scale (cd-risc),

K. M. Connor and J. R. Davidson, “Development of a new resilience scale: The connor-davidson resilience scale (cd-risc),” Depression and anxiety, vol. 18, no. 2, pp. 76–82, 2003

2003
[20]

J. W. Creswell and C. N. Poth, Qualitative inquiry and research design: Choosing among five approaches . Sage publications, 2016

2016
[21]

Code generation using machine learning: A systematic review,

E. Dehaerne, B. Dey, S. Halder, S. De Gendt, and W. Meert, “Code generation using machine learning: A systematic review,” IEEE Access, vol. 10, pp. 82 434–82 455, 2022

2022
[22]

Multi-objective ant colony optimization for requirements selection,

J. Del Sagrado, I. M. Del Águila, and F. J. Orellana, “Multi-objective ant colony optimization for requirements selection,” Empirical Software Engineering, vol. 20, pp. 577–610, 2015

2015
[23]

On designing programming error messages for novices: Readability and its constituent factors,

P. Denny, J. Prather, B. A. Becker, C. Mooney, J. Homer, Z. C. Albrecht, and G. B. Powell, “On designing programming error messages for novices: Readability and its constituent factors,” in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems , ser. CHI...

2021
[24]

Large language models for software engineering: Sur- vey and open problems,

A. Fan, B. Gokkaya, M. Harman, M. Lyubarskiy, S. Sengupta, S. Yoo, and J. M. Zhang, “Large language models for software engineering: Sur- vey and open problems,” in 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE). IEEE...

2023
[25]

School engagement: Potential of the concept, state of the evidence,

J. A. Fredricks, P. C. Blumenfeld, and A. H. Paris, “School engagement: Potential of the concept, state of the evidence,” Review of educational research, vol. 74, no. 1, pp. 59–109, 2004

2004
[26]

Autonomy is an acquired taste: Exploring developer preferences for github bots,

A. Ghorbani, N. Cassee, D. Robinson, A. Alami, N. A. Ernst, A. Sere- brenik, and A. W ˛ asowski, “Autonomy is an acquired taste: Exploring developer preferences for github bots,” in 2023 IEEE/ACM 45th Inter- national Conference on Software Engineering (ICSE) . IEEE, 2023, pp. ...

2023
[27]

Emotions and perceived productivity of software developers at the workplace,

D. Girardi, F. Lanubile, N. Novielli, and A. Serebrenik, “Emotions and perceived productivity of software developers at the workplace,” IEEE Transactions on Software Engineering , vol. 48, no. 9, pp. 3326–3341, 2021

2021
[28]

Goleman, Emotional intelligence: Why it can matter more than IQ

D. Goleman, Emotional intelligence: Why it can matter more than IQ . Bloomsbury Publishing, 2020

2020
[29]

The emerging field of emotion regulation: An integrative review,

J. J. Gross, “The emerging field of emotion regulation: An integrative review,”Review of general psychology, vol. 2, no. 3, pp. 271–299, 1998

1998
[30]

Human-ai collaboration in software engineering: Lessons learned from a hands-on workshop,

M. Hamza, D. Siemon, M. A. Akbar, and T. Rahman, “Human-ai collaboration in software engineering: Lessons learned from a hands-on workshop,” in Proceedings of the 7th ACM/IEEE International Workshop on Software-intensive Business , 2024, pp. 7–14

2024
[31]

The power of feedback,

J. Hattie and H. Timperley, “The power of feedback,” Review of educational research, vol. 77, no. 1, pp. 81–112, 2007

2007
[32]

How efficient are emotional intelligence trainings: A meta-analysis,

S. Hodzic, J. Scharfen, P. Ripoll, H. Holling, and F. Zenasni, “How efficient are emotional intelligence trainings: A meta-analysis,” Emotion Review, vol. 10, no. 2, pp. 138–148, 2018

2018
[33]

“it’s the only thing i can trust

J. Jang, S. Moharana, P. Carrington, and A. Begel, ““it’s the only thing i can trust”: Envisioning large language model use by autistic workers for communication assistance,” in Proceedings of the CHI Conference on Human Factors in Computing Systems , 2024, pp. 1–18

2024
[34]

Automated adaptations to dynamic software architectures by using autonomous agents,

W. Jiao and H. Mei, “Automated adaptations to dynamic software architectures by using autonomous agents,” Engineering Applications of Artificial Intelligence , vol. 17, no. 7, pp. 749–770, 2004

2004
[35]

Why don’t software developers use static analysis tools to find bugs?

B. Johnson, Y . Song, E. Murphy-Hill, and R. Bowdidge, “Why don’t software developers use static analysis tools to find bugs?” in Proceed- ings of the 2013 International Conference on Software Engineering , ser. ICSE ’13. IEEE Press, 2013, p. 672–681

2013
[36]

Qualitative inquiry and research design,

C. Jw, “Qualitative inquiry and research design,” Choosing Among Five Traditions, 1998

1998
[37]

Framing student engagement in higher education,

E. R. Kahu, “Framing student engagement in higher education,” Studies in higher education , vol. 38, no. 5, pp. 758–773, 2013

2013
[38]

What makes a great manager of software engineers?

E. Kalliamvakou, C. Bird, T. Zimmermann, A. Begel, R. DeLine, and D. M. German, “What makes a great manager of software engineers?” IEEE Transactions on Software Engineering, vol. 45, no. 1, pp. 87–106, 2017

2017
[39]

Understanding the impact of emotions on the quality of software artifacts,

K. M. Khan and M. Saleh, “Understanding the impact of emotions on the quality of software artifacts,” IEEE Access, vol. 9, pp. 110 194–110 208, 2021

2021
[40]

The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory

A. N. Kluger and A. DeNisi, “The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory.” Psychological bulletin, vol. 119, no. 2, p. 254, 1996

1996
[41]

Code review quality: How developers see it,

O. Kononenko, O. Baysal, and M. W. Godfrey, “Code review quality: How developers see it,” in Proceedings of the 38th international con- ference on software engineering , 2016, pp. 1028–1038

2016
[42]

Personality, emotional intel- ligence and work preferences in software engineering: An empirical study,

M. V . Kosti, R. Feldt, and L. Angelis, “Personality, emotional intel- ligence and work preferences in software engineering: An empirical study,” Information and Software Technology , vol. 56, no. 8, pp. 973– 990, 2014

2014
[43]

Kvale and S

S. Kvale and S. Brinkmann, Interviews: Learning the craft of qualitative research interviewing. sage, 2009

2009
[44]

Larkin, P

M. Larkin, P. Flowers, and J. A. Smith, Interpretative phenomenological analysis: Theory, method and research . sAgE Publications ltd, 2021

2021
[45]

R. S. Lazarus, Stress, appraisal, and coping . Springer, 1984, vol. 464

1984
[46]

Automating code review activities by large-scale pre-training,

Z. Li, S. Lu, D. Guo, N. Duan, S. Jannu, G. Jenks, D. Majumder, J. Green, A. Svyatkovskiy, S. Fu, and N. Sundaresan, “Automating code review activities by large-scale pre-training,” in Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on ...

2022
[47]

A large-scale survey on the usability of ai programming assistants: Successes and challenges,

J. T. Liang, C. Yang, and B. A. Myers, “A large-scale survey on the usability of ai programming assistants: Successes and challenges,” in Proceedings of the IEEE/ACM 46th International Conference on Software Engineering. ACM, Feb. 2024, p. 1–13

2024
[48]

Latent predictor networks for code generation,

W. Ling, P. Blunsom, E. Grefenstette, K. M. Hermann, T. Ko ˇciský, F. Wang, and A. Senior, “Latent predictor networks for code generation,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , K. Erk and N. A. Smi...

2016
[50]

Llama-reviewer: Advancing code review automation with large language models through parameter- efficient fine-tuning,

J. Lu, L. Yu, X. Li, L. Yang, and C. Zuo, “Llama-reviewer: Advancing code review automation with large language models through parameter- efficient fine-tuning,” in 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE). IEEE, Oct. 2023, p. 647–658

2023
[51]

A framework for emotion- oriented requirements change handling in agile software engineering,

K. Madampe, R. Hoda, and J. Grundy, “A framework for emotion- oriented requirements change handling in agile software engineering,” IEEE Transactions on Software Engineering , vol. 49, no. 5, pp. 3325– 3343, 2023

2023
[52]

Artificial intelligence and the future of work,

T. W. Malone, D. Rus, and R. Laubacher, “Artificial intelligence and the future of work,” A report prepared by MIT Task Force on the work of the future, Research Brief , vol. 17, pp. 1–39, 2020

2020
[53]

Can emotional intelligence be trained? a meta-analytical investigation,

V . Mattingly and K. Kraiger, “Can emotional intelligence be trained? a meta-analytical investigation,” Human Resource Management Review , vol. 29, no. 2, pp. 140–155, 2019

2019
[54]

Nl-oops: A requirements analysis tool based on natural language processing,

L. Mich and R. Garigliano, “Nl-oops: A requirements analysis tool based on natural language processing,” WIT Transactions on Information and Communication Technologies, vol. 28, 2002

2002
[55]

M. B. Miles, A. M. Huberman, and J. Saldaña, Qualitative data analysis: A methods sourcebook. 3rd . Thousand Oaks, CA: Sage, 2014

2014
[56]

M. B. Miles, M. Huberman, and J. Saldana, Qualitative data analysis: A methods sourcebook . SAGE Publications, Incorporated, 2013

2013
[57]

Explainable software bot contributions: Case study of automated bug fixes,

M. Monperrus, “Explainable software bot contributions: Case study of automated bug fixes,” in 2019 IEEE/ACM 1st International Workshop on Bots in Software Engineering (BotSE) . IEEE, May 2019

2019
[58]

Repairnator patches programs automatically,

M. Monperrus, S. Urli, T. Durieux, M. Martinez, B. Baudry, and L. Seinturier, “Repairnator patches programs automatically,” Ubiquity, vol. 2019, no. July, p. 1–12, Jul. 2019

2019
[59]

Theoretical saturation,

J. M. Morse, “Theoretical saturation,” Encyclopedia of social science research methods, vol. 3, pp. 1122–3, 2004

2004
[60]

Among the machines: Human-bot interaction on social q&a websites,

A. Murgia, D. Janssens, S. Demeyer, and B. Vasilescu, “Among the machines: Human-bot interaction on social q&a websites,” in Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems , ser. CHI’16. ACM, May 2016. [Online]. Available: http:...

2016
[61]

Do developers feel emotions? an exploratory analysis of emotions in software artifacts,

A. Murgia, P. Tourani, B. Adams, and M. Ortu, “Do developers feel emotions? an exploratory analysis of emotions in software artifacts,” in Proceedings of the 11th working conference on mining software repositories, 2014, pp. 262–271

2014
[62]

Intellireq: intelligent techniques for software requirements engineering,

G. Ninaus, A. Felfernig, M. Stettinger, S. Reiterer, G. Leitner, L. Weninger, and W. Schanil, “Intellireq: intelligent techniques for software requirements engineering,” in ECAI 2014 . IOS Press, 2014, pp. 1161–1166

2014
[63]

Sentiment and emotion in software engineering,

N. Novielli and A. Serebrenik, “Sentiment and emotion in software engineering,” IEEE Software, vol. 36, no. 5, pp. 6–23, 2019

2019
[64]

An empirical investigation of relevant changes and automation needs in modern code review,

S. Panichella and N. Zaugg, “An empirical investigation of relevant changes and automation needs in modern code review,” Empirical Software Engineering , vol. 25, no. 6, p. 4833–4872, Sep. 2020. [Online]. Available: http://dx.doi.org/10.1007/s10664-020-09870-3

2020 doi
[65]

Emotional intelligence: The key to mitigating stress and fostering trust among software developers working on information system projects,

A. Rezvani and P. Khosravi, “Emotional intelligence: The key to mitigating stress and fostering trust among software developers working on information system projects,” International Journal of Information Management, vol. 48, pp. 139–150, 2019

2019
[66]

Artificial intelligence in service-oriented software design,

G. Rodríguez, Á. Soria, and M. Campo, “Artificial intelligence in service-oriented software design,” Engineering Applications of Artificial Intelligence, vol. 53, pp. 86–104, 2016

2016
[67]

Taking the emotional pulse of software engineering—a systematic literature review of empiri- cal studies,

M. Sánchez-Gordón and R. Colomo-Palacios, “Taking the emotional pulse of software engineering—a systematic literature review of empiri- cal studies,” Information and Software Technology, vol. 115, pp. 23–43, 2019

2019
[68]

Automated regression tests: A no-code approach for bpmn-based process-driven applications,

K. Schneid, L. Stapper, S. Thöne, and H. Kuchen, “Automated regression tests: A no-code approach for bpmn-based process-driven applications,” in 2021 IEEE 25th International Enterprise Distributed Object Comput- ing Conference (EDOC) , 2021, pp. 31–40

2021
[69]

Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries,

S. H. Schwartz, “Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries,” Advances in experimental social psychology/Academic Press , 1992

1992
[70]

Seidman, Interviewing as qualitative research: A guide for researchers in education and the social sciences

I. Seidman, Interviewing as qualitative research: A guide for researchers in education and the social sciences . Teachers college press, 2006

2006
[71]

Focus on formative feedback,

V . J. Shute, “Focus on formative feedback,” Review of educational research, vol. 78, no. 1, pp. 153–189, 2008

2008
[72]

What counts as qualitative research? some cautionary comments,

D. Silverman, “What counts as qualitative research? some cautionary comments,” Qualitative sociology review, vol. 9, no. 2, pp. 48–55, 2013

2013
[73]

E. A. Skinner, T. A. Kindermann, and C. J. Furrer, “A motivational perspective on engagement and disaffection: Conceptualization and assessment of children’s behavioral and emotional participation in academic activities in the classroom,” Educational and psychological measurem...

2009
[74]

Cobots in knowledge work: Human–ai collaboration in managerial professions,

K. Sowa, A. Przegalinska, and L. Ciechanowski, “Cobots in knowledge work: Human–ai collaboration in managerial professions,” Journal of Business Research, vol. 125, pp. 135–142, 2021

2021
[75]

The abc of software engineering research,

K.-J. Stol and B. Fitzgerald, “The abc of software engineering research,” ACM Transactions on Software Engineering and Methodology (TOSEM), vol. 27, no. 3, pp. 1–51, 2018

2018
[76]

Intellicode compose: Code generation using transformer,

A. Svyatkovskiy, S. K. Deng, S. Fu, and N. Sundaresan, “Intellicode compose: Code generation using transformer,” in Proceedings of the 28th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering , 2020, pp. 1433–1443

2020
[77]

Using pre-trained models to boost code review automa- tion,

R. Tufano, S. Masiero, A. Mastropaolo, L. Pascarella, D. Poshyvanyk, and G. Bavota, “Using pre-trained models to boost code review automa- tion,” in Proceedings of the 44th International Conference on Software Engineering. ACM, May 2022

2022
[78]

Attention is all you need,

A. Vaswani, “Attention is all you need,” Advances in Neural Information Processing Systems, 2017

2017
[79]

Constructive code review: Managing the impact of interpersonal conflicts in prac- tice,

P. Wurzel Goncalves, J. SV Goncalves, and A. Bacchelli, “Constructive code review: Managing the impact of interpersonal conflicts in prac- tice,” in Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice , 2024, pp. 334–345

2024
[80]

Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task,

T. Yu, R. Zhang, K. Yang, M. Yasunaga, D. Wang, Z. Li, J. Ma, I. Li, Q. Yao, S. Roman et al. , “Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task,” in 2018 Conference on Empirical Methods in Natural Language Processi...

2018
[81]

Automated generation of oracles for testing user-interaction features of mobile apps,

R. N. Zaeem, M. R. Prasad, and S. Khurshid, “Automated generation of oracles for testing user-interaction features of mobile apps,” in 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation. IEEE, 2014, pp. 183–192

2014
[82]

The future of software engineering by 2050s: Will ai replace software engineers?

L. M. A. Zohair, “The future of software engineering by 2050s: Will ai replace software engineers?” International Journal of Information Technology, vol. 2, no. 3, pp. 1–13, 2018

2018

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [49]

Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing,

P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, “Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing,” ACM Computing Surveys, vol. 55, no. 9, pp. 1–35, 2023

2023

[2] [1]

Github copilot · your ai pair programmer · github,

“Github copilot · your ai pair programmer · github,” https://github.com/ features/copilot/, (Accessed on 08/27/2024)

2024

[3] [2]

Openai codex | openai,

“Openai codex | openai,” https://openai.com/index/openai-codex/, (Ac- cessed on 08/27/2024)

2024

[4] [3]

Tabnine ai code assistant | private, personalized, protected,

“Tabnine ai code assistant | private, personalized, protected,” https:// www.tabnine.com/, (Accessed on 08/27/2024)

work page 2024

[5] [4]

Octoverse: The state of open source and rise of ai in 2023 - the github blog,

“Octoverse: The state of open source and rise of ai in 2023 - the github blog,” https://github.blog/news-insights/research/ the-state-of-open-source-and-ai/, 2023, (Accessed on 08/27/2024)

work page 2023

[6] [5]

Information seeking using AI assistants,

E. Al Haque, C. Brown, T. D. LaToza, and B. Johnson, “Information seeking using AI assistants,” 2024. [Online]. Available: https: //arxiv.org/abs/2408.04032

arXiv 2024

[7] [6]

How do foss communities decide to accept pull requests?

A. Alami, M. L. Cohn, and A. W ˛ aisowski, “How do foss communities decide to accept pull requests?” in Proceedings of the 24th International Conference on Evaluation and Assessment in Software Engineering , 2020, pp. 220–229

work page 2020

[8] [7]

Pull request gov- ernance in open source communities,

A. Alami, R. Pardo, M. L. Cohn, and A. W ˛ asowski, “Pull request gov- ernance in open source communities,” IEEE Transactions on Software Engineering, vol. 48, no. 12, pp. 4838–4856, 2021

work page 2021