How Developers Interact with AI: A Taxonomy of Human-AI Collaboration in Software Engineering

Christoph Treude; Marco A. Gerosa

REVIEW 4 major objections 4 minor 16 references

How Developers Interact with AI: A Taxonomy of Human-AI Collaboration in Software Engineering

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A taxonomy of eleven interaction types organizes how developers use AI tools

desk verdict Useful first SE-specific taxonomy of developer-AI interaction types, but the eleven categories need an operational boundary test before the 'distinct' claim holds. read the letter →

arxiv 2501.08774 v2 pith:MQUOYO6T submitted 2025-01-15 cs.SE cs.AIcs.HC

Christoph Treude , Marco A. Gerosa This is my paper · ORCID

classification cs.SEcs.AIcs.HC

keywords ArtificialIntelligenceSoftwareDevelopmentDeveloperToolsHuman-AIInteractionGenerativeAILargeLanguageModels

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Software developers do not use AI in one way: they accept ghost-text completions while typing, ask chat assistants to explain code, issue commands to generate summaries, and let CI bots report on pull requests. This paper argues that these varied behaviors are not a blur but can be sorted into eleven distinct interaction types, each characterized by its trigger, the AI's response, the developer's reaction, and the output produced. A sympathetic reader would care because the field currently lacks a shared vocabulary: without a taxonomy, studies of different AI tools cannot be compared, and designers have no systematic way to ask which interaction style fits which task. Building on the taxonomy, the paper outlines research directions on effectiveness, trust, context-awareness, developer control, cognitive load, ethics, privacy, and hallucination in AI-assisted development.

What carries the argument

The central object is Table I, a characterization grid with five columns—trigger, AI response, developer response, output type, and example—used to define each of the eleven interaction types. The trigger column records how an interaction starts (typing context, explicit command, chat question, code selection, UI click, comment, workflow event, shortcut, file type, or API call), and the output column records what artifact the AI produces (suggestion, action, explanation, or report). The grid is doing the argument's work: it is the proposed answer to 'what distinguishes one kind of developer-AI interaction from another,' with the pairs that are hardest to separate—contextual recommendations versus file-aware suggestions, and event-based triggers versus automated API responses—separated by where the trigger comes from and how much system-to-system automation is involved.

What would settle it

Record a corpus of real developer sessions in an IDE and have independent raters assign every observed AI interaction to exactly one of the eleven types. The taxonomy's central claim is falsified if a substantial share of sessions falls into no type, splits across two types at once, or produces low inter-rater agreement; it is also weakened if a search of current tools reveals a common interaction, such as a voice-command assistant or an agent that autonomously edits multiple files, that fits none of the existing rows.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that all current developer-AI interactions in software engineering can be organized by a two-level scheme: a set of interaction types and the dimensions that define them. The eleven types are Auto-Complete Code Suggestions, Command-Driven Actions, Conversational Assistance, Contextual Recommendations, Selection-Based Enhancements, Explicit UI Actions, Comment-Guided Prompts, Event-Based Triggers, Shortcut-Activated Commands, File-Aware Suggestions, and Automated API Responses. Each type is defined by a row in a characterization table whose columns are the trigger, the AI response, the developer response, the type of output, and a concrete tool example. The paper further claims that this organization gives researchers and tool builders a common conceptual model for empirical study and evaluation, and it uses the taxonomy to motivate a research agenda rather than to report empirical validation.

Load-bearing premise

The load-bearing premise is that the five-part grid—trigger, AI response, developer response, output type, and example—captures enough of a real interaction to draw clean boundaries between eleven distinct types, with no common interaction left unclassifiable and no pair of types actually being the same behavior described twice.

Editorial extensions

If this is right

Researchers gain a shared vocabulary for reporting studies of autocomplete, chat assistants, and CI-integrated AI, so results from different tools and papers can be aggregated.
A tool or feature can be classified by reading off its row in the grid, which gives evaluators a checklist of what to measure: how the interaction starts, what the AI emits, and what the developer is expected to do.
The research agenda ties each cluster of interaction types to specific open questions: passive suggestions raise cognitive-load and control questions, while commands, UI actions, and API responses raise trust, privacy, and damage-control questions.
The taxonomy can be used to audit a development environment for coverage, exposing interaction types that existing AI features do not yet support.
With the taxonomy in place, empirical studies can measure how often each interaction type occurs in real workflows and whether some types are more effective in particular tasks, which the paper explicitly leaves to future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An implication the paper leaves implicit is that the trigger dimension doubles as an autonomy spectrum: the more automatic the trigger, the more the design burden is on the AI to be unobtrusive; the more explicit the trigger, the more the design burden is on the developer to know what to ask. Tool builders could use this spectrum to align initiative with the cost of a wrong suggestion.
Not in the paper, but testable: the taxonomy can be turned into a coding scheme for IDE telemetry, so that logs of accept, dismiss, and override behavior can be labeled by interaction type and compared across tools.
The authors treat the eleven types as distinct, but adjacent categories point toward hybrids—for example, a conversational assistant that also triggers CI checks—so a plausible extension is a two-axis space rather than a flat list, with 'who initiates' and 'what artifact is produced' as continuous dimensions.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Useful, clearly written taxonomy paper that the SE community will probably cite, but the eleven categories are not yet shown to be mutually exclusive or exhaustive. The stress-test note lands: several pairs are separated mostly by input modality or context scope.

What's new and what works: the eleven-type framework is a genuine synthesis, to my knowledge the first SE-specific taxonomy of developer-AI interaction types. It brings together a diverse set of tools, and Table I is a clean organizing device. The research agenda is sensible and keeps the paper from being just a list. The prose is restrained, and Section IV honestly says validation is beyond scope.

The soft spots are real. The reader's conditional verdict is about right. Command-driven actions, explicit UI actions, and shortcut-activated commands are all the developer explicitly asking the AI to do something; the difference is click vs key vs command, a UI design choice rather than an interaction type. Event-based triggers and automated API responses both fire on workflow events; the system-to-system vs developer-environment distinction is not operational. Contextual recommendations and file-aware suggestions overlap too. That matters because the abstract says 'eleven distinct interaction types' — distinctness means an observer can classify an episode without guessing. Right now the table is filled in post hoc, with each type illustrated by a single tool example. There is no inter-rater reliability, no boundary examples, no attempt to collapse or expand the categories against observed behavior. The paper would be stronger with one or two ambiguous cases worked through, or a small pilot classification.

There is no math or data beyond the examples, so the burden falls on internal coherence and representativeness. The examples are commercial and self-selected; fine for a proposal, but it limits the comprehensiveness claim. The citation pattern is fine; self-citations are relevant.

This is for researchers studying AI-assisted development, tool designers, and anyone building a coding scheme for empirical studies. It deserves peer review: it is clearly argued, fills a gap, and will likely become a background reference. The authors should be pushed to address category boundaries, ideally with a small empirical exercise.

I'd probably bring it to a reading group and would cite it if I were writing on empirical studies of AI coding tools. Send it to review.

Referee Report

4 major / 4 minor

Summary. This paper proposes a taxonomy of eleven types of developer-AI interaction in software engineering, characterized by trigger, AI response, developer response, output, and concrete tool examples (Table I and Section II). The interaction types range from auto-complete code suggestions and command-driven actions to conversational assistance and automated API responses. Building on this taxonomy, Section III outlines a research agenda covering effectiveness, trust, context-awareness, developer control, cognitive load, ethics, privacy, and hallucination. Section IV states that empirical validation is beyond the scope of this paper and suggests future work to observe real-world tool use.

Significance. If the proposed taxonomy holds up, it provides a useful common vocabulary for studying and comparing developer-AI interactions, a topic that is currently fragmented. The paper's concrete examples from GitHub Copilot, ChatGPT, Sourcery, CodeClimate, and other tools help make each category tangible, and the authors are transparent that validation is deferred. However, the central claim of eleven distinct interaction types is not currently demonstrated, so the contribution is best seen as a promising conceptual framework rather than an empirically grounded classification. The research agenda is broad and plausible, though it lists questions rather than prioritizing them or linking them tightly to the taxonomy's dimensions.

major comments (4)

[Section II and Table I] The central claim that the eleven interaction types are distinct is not established, because the four characterization columns are illustrative rather than operational. No decision rule or coding protocol is given for assigning an observed interaction episode to exactly one type, and the examples appear to be selected after the fact to fit each category. Section IV acknowledges that validation is beyond scope; for the taxonomy to support the stated contribution, the paper should either provide explicit disambiguation criteria for each pair of similar types or report an empirical assignment/reliability study.
[Section II-f and II-i] Explicit UI Actions and Shortcut-Activated Commands appear to be the same interaction type distinguished only by input modality (button click versus keyboard shortcut), and Section II-i's own contrast refers to "more deliberate engagement" rather than any difference in AI response, developer response, or output. Similarly, Command-Driven Actions in Section II-b is also an explicit request via typed command. The paper needs to justify why input modality is a defining dimension of interaction type, or merge these categories.
[Section II-h and II-k] Event-Based Triggers and Automated API Responses are not reliably separable. Both are initiated by workflow events such as pull requests; the examples GitLab Auto DevOps (classified as Event-Based) and CodeClimate (classified as Automated API) both analyze pull requests. The stated distinction between "system-to-system" and "developer environment" is not derivable from the table's columns and is not operational. Please refine the distinguishing criterion or consolidate these categories.
[Section II-d and II-j] Contextual Recommendations and File-Aware Suggestions both depend on file and project context, and the distinction drawn in Section II-j (file-type-specific suggestions at file creation/editing versus broader project-level cues) is not encoded in Table I and is not backed by a rule for classifying borderline episodes. The paper should provide clearer boundary conditions or show that the categories are empirically discriminable.

minor comments (4)

[Table I] The table header merges "Trigger and AI Response" and "Developer Response and Output" into two wide columns; splitting the table into four separate columns would make the characterization easier to compare across the eleven types.
[Abstract] There are minor typographical/OCR artifacts such as "too ls" in the abstract; a final proofread is needed before publication.
[Introduction] The paper claims to be the first taxonomy specifically for human-AI interaction in software engineering; a brief comparison with existing general human-AI interaction taxonomies would help position the contribution and justify the SE-specific dimensions.
[Section III] The research agenda lists many worthwhile questions but does not state which ones are highest priority or how each connects to particular interaction types; adding a mapping or prioritization could make the agenda more actionable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the taxonomy is an inductive conceptual proposal whose validation is explicitly deferred to future empirical work.

full rationale

The paper proposes a taxonomy of eleven developer-AI interaction types characterized in Table I by trigger, AI response, developer response, output, and example. There is no derivation chain, no fitted parameter, and no theorem whose conclusion is equivalent to its assumptions. The categories are presented as a conceptual organization of observed tool behaviors, and the paper explicitly states that "validation is beyond the scope of this paper" and that "future studies could empirically evaluate the taxonomy by observing tool use in real-world software development workflows" (Section IV). The only self-citations are references [8] and [9], used as illustrative examples of command-driven actions and conversational assistance; they do not bear the weight of the taxonomy's validity. The distinctions between interaction types (e.g., shortcut-activated commands vs. explicit UI actions, or event-based triggers vs. automated API responses) may be arguable and the categories may overlap, but that is a validity or representativeness concern, not circular reasoning. No step in the paper reduces a claimed result to its own inputs by construction, so the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 1 invented entities

The taxonomy rests on assumptions about the adequacy of its dimensions, the distinctness of its categories, and the representativeness of its examples. There are no numeric free parameters or physical entities.

assumptions (3)

domain assumption The five characterization dimensions (trigger, AI response, developer response, output) are sufficient and non-overlapping for describing developer-AI interactions.
Introduced in Section II; the paper imposes this grid on all interaction types without prior empirical analysis.
domain assumption The eleven interaction types are distinct and cover the interaction space in software engineering.
This is the central claim; it is asserted through Table I and Section II, and validation is explicitly deferred in Section IV.
domain assumption The example tools (Copilot, ChatGPT, Cody, Sourcery, etc.) are representative of current developer-AI interactions.
Examples in Table I are selected from commercial and research tools without systematic selection criteria, and Section IV notes the absence of systematic evaluation.

invented entities (1)

Taxonomy of 11 developer-AI interaction types (auto-complete, command-driven, conversational, etc.)
purpose: Provides a vocabulary for classifying developer-AI interactions in software engineering.
The categories are asserted from examples in Table I; no external validation, inter-rater study, or systematic survey is provided. The paper states validation is beyond scope.

how reviews work

0 comments

Cite this review

Pith. "Pith review of How Developers Interact with AI: A Taxonomy of Human-AI Collaboration in Software Engineering." pith.science (2026). https://pith.science/paper/MQUOYO6T

@misc{pith2026250108774,
  author       = {Pith},
  title        = {Pith review of: How Developers Interact with AI: A Taxonomy of Human-AI Collaboration in Software Engineering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MQUOYO6T}},
  note         = {Machine review of arXiv:2501.08774}
}

read the original abstract

Artificial intelligence (AI), including large language models and generative AI, is emerging as a significant force in software development, offering developers powerful tools that span the entire development lifecycle. Although software engineering research has extensively studied AI tools in software development, the specific types of interactions between developers and these AI-powered tools have only recently begun to receive attention. Understanding and improving these interactions has the potential to enhance productivity, trust, and efficiency in AI-driven workflows. In this paper, we propose a taxonomy of interaction types between developers and AI tools, identifying eleven distinct interaction types, such as auto-complete code suggestions, command-driven actions, and conversational assistance. Building on this taxonomy, we outline a research agenda focused on optimizing AI interactions, improving developer control, and addressing trust and usability challenges in AI-assisted development. By establishing a structured foundation for studying developer-AI interactions, this paper aims to stimulate research on creating more effective, adaptive AI tools for software development.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 15 canonical work pages

[1]

Navigating the complexity of generative AI ad option in software engineering,

D. Russo, “Navigating the complexity of generative AI ad option in software engineering,” ACM Transactions on Software Engineering and Methodology, 2024

work page 2024
[2]

A decade of progress: A systematic literature review on the integration of AI in software engineering phases and a ctivities (2013-2023),

U. K. Durrani, M. Akpinar, M. F. Adak, A. T. Kabakus, M. M. O zturk, and M. Saleh, “A decade of progress: A systematic literature review on the integration of AI in software engineering phases and a ctivities (2013-2023),” IEEE Access , 2024

work page 2013
[3]

Visuali zing developer interactions,

R. Minelli, A. Mocci, M. Lanza, and L. Baracchi, “Visuali zing developer interactions,” in Proceedings of the IEEE W orking Conference on Software Visualization. IEEE, 2014, pp. 147–156

work page 2014
[4]

Iden- tifying the factors that inﬂuence trust in AI code completio n,

A. Brown, S. D’Angelo, A. Murillo, C. Jaspan, and C. Green , “Iden- tifying the factors that inﬂuence trust in AI code completio n,” in Pro- ceedings of the ACM International Conference on AI-Powered Software, 2024, pp. 1–9

work page 2024
[5]

Understanding and designing for trust in AI- powered developer tooling,

A. Murillo, A. Elizondo, S. D’Angelo, A. Brown, U. Kumar, Q. Madi- son, and A. Macvean, “Understanding and designing for trust in AI- powered developer tooling,” IEEE Software , vol. 41, no. 6, pp. 23–28, 2024

work page 2024
[6]

The programmer’s assistant: Conversational interaction with a large language model for software development,

S. I. Ross, F. Martinez, S. Houde, M. Muller, and J. D. Weis z, “The programmer’s assistant: Conversational interaction with a large language model for software development,” in Proceedings of the International Conference on Intelligent User Interfaces , 2023, pp. 491–514

work page 2023
[7]

Taking ﬂight with Copilot: Ear ly insights and opportunities of AI-powered pair-programming tools,

C. Bird, D. Ford, T. Zimmermann, N. Forsgren, E. Kalliamv akou, T. Lowdermilk, and I. Gazit, “Taking ﬂight with Copilot: Ear ly insights and opportunities of AI-powered pair-programming tools,” ACM Queue, vol. 20, no. 6, pp. 35–57, 2022

work page 2022
[8]

Generativ e AI for pull request descriptions: Adoption, impact, and developer int erventions,

T. Xiao, H. Hata, C. Treude, and K. Matsumoto, “Generativ e AI for pull request descriptions: Adoption, impact, and developer int erventions,” Proceedings of the ACM on Software Engineering , vol. 1, no. FSE, pp. 1043–1065, 2024

work page 2024

Show all 16 references

[9]

DevGPT: St udying developer-ChatGPT conversations,

T. Xiao, C. Treude, H. Hata, and K. Matsumoto, “DevGPT: St udying developer-ChatGPT conversations,” in Proceedings of the IEEE/ACM International Conference on Mining Software Repositories. IEEE, 2024, pp. 227–230

2024
[10]

AI-assisted coding with Cody: Lessons from context retrieval and evalua tion for code recommendations,

J. Hartman, H. Sagtani, J. Tibshirani, and R. Mehrotra, “AI-assisted coding with Cody: Lessons from context retrieval and evalua tion for code recommendations,” in Proceedings of the ACM Conference on Recommender Systems , 2024, pp. 748–750

2024
[11]

More on AI tools: Developer’s magic wand,

A. Dwivedi, “More on AI tools: Developer’s magic wand,” in Code- Mosaic: Learn AI-Driven Development and Modern Best Practi ces for Enterprise. Springer, 2024, pp. 505–535

2024
[12]

From Copilot to pilot: Toward s AI supported software development,

R. Pudari and N. A. Ernst, “From Copilot to pilot: Toward s AI supported software development,” arXiv preprint arXiv:2303.04142 , 2023

2023 arXiv
[13]

Greeni ng large language models of code,

J. Shi, Z. Y ang, H. J. Kang, B. Xu, J. He, and D. Lo, “Greeni ng large language models of code,” in Proceedings of the International Conference on Software Engineering: Software Engineering in Society , 2024, pp. 142–153

2024
[14]

Dem ystifying practices, challenges and expected features of using GitHu b Copilot,

B. Zhang, P . Liang, X. Zhou, A. Ahmad, and M. Waseem, “Dem ystifying practices, challenges and expected features of using GitHu b Copilot,” International Journal of Software Engineering and Knowled ge Engi- neering, vol. 33, no. 11n12, pp. 1653–1672, 2023

2023
[15]

An analysis of generative ar- tiﬁcial intelligence tools usage to adapt and enrich softwa re development courses,

B. Mihaljevi´ c, A. Radovan, and M. ˇZagar, “An analysis of generative ar- tiﬁcial intelligence tools usage to adapt and enrich softwa re development courses,” in Proceedings of the International Conference on Education and New Developments , 2024, pp. 553–557

2024
[16]

Improving feedback on GitHub pull requests: A bots approach,

Z. Hu and E. F. Gehringer, “Improving feedback on GitHub pull requests: A bots approach,” in Proceedings of the IEEE Frontiers in Education Conference. IEEE, 2019, pp. 1–9

2019

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Navigating the complexity of generative AI ad option in software engineering,

D. Russo, “Navigating the complexity of generative AI ad option in software engineering,” ACM Transactions on Software Engineering and Methodology, 2024

work page 2024

[2] [2]

A decade of progress: A systematic literature review on the integration of AI in software engineering phases and a ctivities (2013-2023),

U. K. Durrani, M. Akpinar, M. F. Adak, A. T. Kabakus, M. M. O zturk, and M. Saleh, “A decade of progress: A systematic literature review on the integration of AI in software engineering phases and a ctivities (2013-2023),” IEEE Access , 2024

work page 2013

[3] [3]

Visuali zing developer interactions,

R. Minelli, A. Mocci, M. Lanza, and L. Baracchi, “Visuali zing developer interactions,” in Proceedings of the IEEE W orking Conference on Software Visualization. IEEE, 2014, pp. 147–156

work page 2014

[4] [4]

Iden- tifying the factors that inﬂuence trust in AI code completio n,

A. Brown, S. D’Angelo, A. Murillo, C. Jaspan, and C. Green , “Iden- tifying the factors that inﬂuence trust in AI code completio n,” in Pro- ceedings of the ACM International Conference on AI-Powered Software, 2024, pp. 1–9

work page 2024

[5] [5]

Understanding and designing for trust in AI- powered developer tooling,

A. Murillo, A. Elizondo, S. D’Angelo, A. Brown, U. Kumar, Q. Madi- son, and A. Macvean, “Understanding and designing for trust in AI- powered developer tooling,” IEEE Software , vol. 41, no. 6, pp. 23–28, 2024

work page 2024

[6] [6]

The programmer’s assistant: Conversational interaction with a large language model for software development,

S. I. Ross, F. Martinez, S. Houde, M. Muller, and J. D. Weis z, “The programmer’s assistant: Conversational interaction with a large language model for software development,” in Proceedings of the International Conference on Intelligent User Interfaces , 2023, pp. 491–514

work page 2023

[7] [7]

Taking ﬂight with Copilot: Ear ly insights and opportunities of AI-powered pair-programming tools,

C. Bird, D. Ford, T. Zimmermann, N. Forsgren, E. Kalliamv akou, T. Lowdermilk, and I. Gazit, “Taking ﬂight with Copilot: Ear ly insights and opportunities of AI-powered pair-programming tools,” ACM Queue, vol. 20, no. 6, pp. 35–57, 2022

work page 2022

[8] [8]

Generativ e AI for pull request descriptions: Adoption, impact, and developer int erventions,

T. Xiao, H. Hata, C. Treude, and K. Matsumoto, “Generativ e AI for pull request descriptions: Adoption, impact, and developer int erventions,” Proceedings of the ACM on Software Engineering , vol. 1, no. FSE, pp. 1043–1065, 2024

work page 2024