RECORD
LLMs' Reshaping of People, Processes, Products, and Society in Software Development: A Comprehensive Exploration with Early Adopters
Software Engineering · Submitted 6 Mar 2025
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Our findings, later echoed in large-scale quantitative studies, offer actionable implications for developers, organizations, educators, and tool designers seeking to integrate LLMs responsibly into software practice today.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Large language models (LLMs) are rapidly reshaping software development, but their impact across the software development lifecycle is underexplored. Existing work focuses on isolated activities such as code generation or testing, leaving open questions about how LLMs affect developers, processes, products, and the software ecosystem. We address this gap through semi-structured interviews with sixteen early-adopter software professionals who integrated LLM-based tools into their day-to-day work in early to mid-2023. We treat these interviews as early empirical evidence and compare participants' accounts with recent work on LLMs in software engineering, noting which early patterns persist or shift. Using thematic analysis, we organize findings around four dimensions: people, process, product, and society. Developers reported substantial productivity gains from reducing routine tasks, streamlining search, and accelerating debugging, but also described a productivity-quality paradox: they often discarded generated code and shifted effort from writing to critically evaluating and integrating it. LLM use was highly phase-dependent, with strong uptake in implementation and debugging but limited influence on requirements gathering and other collaborative work. Participants developed new competencies to use LLMs effectively, including prompt engineering strategies, layered verification, and secure integration to protect proprietary data. They anticipated changes in hiring expectations, team practices, and computing education, while emphasizing that human judgment and foundational software engineering skills remain essential. Our findings, later echoed in large-scale quantitative studies, offer actionable implications for developers, organizations, educators, and tool designers seeking to integrate LLMs responsibly into software practice today.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Large language models (LLMs) are rapidly reshaping software development, but their impact across the software development lifecycle is underexplored. | background 0.72 | 0.05 | 0.03 | asserted 1.00 |
| s1 | Existing work focuses on isolated activities such as code generation or testing, leaving open questions about how LLMs affect developers, processes, products, and the software ecosystem. | gap 0.96 | 0.04 | 0.03 | asserted 1.00 |
| s2 | We address this gap through semi-structured interviews with sixteen early-adopter software professionals who integrated LLM-based tools into their day-to-day work in early to mid-2023. | method 0.93 | 0.97 | 0.02 | asserted 0.99 |
| s3 | We treat these interviews as early empirical evidence and compare participants' accounts with recent work on LLMs in software engineering, noting which early patterns persist or shift. | method 0.75 | 0.92 | 0.03 | asserted 0.88 |
| s4 | Using thematic analysis, we organize findings around four dimensions: people, process, product, and society. | method 0.89 | 0.93 | 0.04 | asserted 0.99 |
| s5 | Developers reported substantial productivity gains from reducing routine tasks, streamlining search, and accelerating debugging, but also described a productivity-quality paradox: they often discarded generated code and shifted effort from writing to critically evaluating and integrating it. | result 1.00 | 0.89 | 0.25 | asserted 1.00 |
| s6 | LLM use was highly phase-dependent, with strong uptake in implementation and debugging but limited influence on requirements gathering and other collaborative work. | result 1.00 | 0.90 | 0.08 | asserted 1.00 |
| s7 | Participants developed new competencies to use LLMs effectively, including prompt engineering strategies, layered verification, and secure integration to protect proprietary data. | result 1.00 | 0.86 | 0.05 | asserted 1.00 |
| s8 | They anticipated changes in hiring expectations, team practices, and computing education, while emphasizing that human judgment and foundational software engineering skills remain essential. | result 0.76 | 0.66 | 0.16 | asserted 0.82 |
| s9 | Our findings, later echoed in large-scale quantitative studies, offer actionable implications for developers, organizations, educators, and tool designers seeking to integrate LLMs responsibly into software practice today. | interpretation 0.97 | 0.61 | 0.36 | asserted 0.99 |
What was examined, and what was not
- arXiv listing abstract v2
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Large language models (LLMs) are rapidly reshaping software development, but their impact across the software development lifecycle is underexplored. Existing work focuses on isolated activities such as code generation or testing, leaving open questions about how LLMs affect developers, processes, products, and the software ecosystem. We address this gap through semi-structured interviews with sixteen early-adopter software professionals who integrated LLM-based tools into their day-to-day work in early to mid-2023. We treat these interviews as early empirical evidence and compare participants' accounts with recent work on LLMs in software engineering, noting which early patterns persist or shift. Using thematic analysis, we organize findings around four dimensions: people, process, product, and society. Developers reported substantial productivity gains from reducing routine tasks, streamlining search, and accelerating debugging, but also described a productivity-quality paradox: they often discarded generated code and shifted effort from writing to critically evaluating and integrating it. LLM use was highly phase-dependent, with strong uptake in implementation and debugging but limited influence on requirements gathering and other collaborative work. Participants developed new competencies to use LLMs effectively, including prompt engineering strategies, layered verification, and secure integration to protect proprietary data. They anticipated changes in hiring expectations, team practices, and computing education, while emphasizing that human judgment and foundational software engineering skills remain essential. Our findings, later echoed in large-scale quantitative studies, offer actionable implications for developers, organizations, educators, and tool designers seeking to integrate LLMs responsibly into software practice today.
Human review
References
How the literature uses it
5 arXiv papers cite this in 6 sentences
Papers citing it · 5 sentences without a confident label
Cites it as background 1 paper
-
Advances in LLMs, such as ChatGPT, GitHub Copilot, and Codex, have created new opportunities and challenges for novice developers learning programming and real-world software development [ref] .
AgentForge: An Immersive Role-Playing Platform for Learning Agentic Software Engineering · AI-Assisted Programming for Novice Developers · 0.94
What it cites · 16 sentences without a confident label
Cites as support 2 papers
-
Another critical concern is that LLMs often do not provide the source of the code they generate [ref] , leaving the origins and security of the produced code uncertain.
2407.06153 · Emphasizing secure practices · 1.00
-
Developers must remain vigilant about the proprietary and security aspects of the code they share with LLMs, as emphasized by Wang et al. [ref] .
Enhancing Large Language Models for Secure Code Generation: A Dataset-driven Study on Vulnerability Mitigation · Emphasizing secure practices · 0.99
Cites as background 2 papers
-
Studies have focused on readability [ref] , complexity [ref] , correctness [ref] , and security [ref] .
Extending the Frontier of ChatGPT: Code Generation and Debugging · Products- Artifacts · 0.89
-
The most notable work on LLMs in the Software Engineering (SE) domain is a systematic review by Hou et al., which analyzed 395 papers on their impact, focusing on optimization techniques, applications, and potential use cases [ref] .
Large Language Models for Software Engineering: A Systematic Literature Review · Related Works · 0.95
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.