Pith. sign in

REVIEW 3 major objections 5 minor 2 references

What do professional software developers need to know to succeed in an age of Artificial Intelligence?

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Successful AI-enhanced software development rests on four skill domains woven into a six-step task workflow, and training must target all four, not just prompting, to prevent deskilling.

desk verdict A genuinely useful occupational taxonomy of AI-enhanced developers, weaker where it overclaims comprehensiveness. read the letter →

arxiv 2506.00202 v3 pith:OJL5PSFI submitted 2025-05-30 cs.AI

Matthew Kam , Cody Miller , Miaoxin Wang , Abey Tidwell , Irene A. Lee , Joyce Malyn-Smith , Beatriz Perez , Vikram Tiwari
show 3 more authors
Joshua Kenitzer Andrew Macvean Erin Barrar
This is my paper · ORCID
classification cs.AI
keywords DACUMGenerativeArtificialIntelligencehuman-centeredAIsoftwareengineeringeducationoccupationalprofileskillsandknowledgedeskillingAI-enhanceddevelopment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Generative AI tools are already speeding up coding, but the paper asks what developers must know to use them well rather than be undercut by them. To answer, the authors ran structured DACUM workshops in which 21 expert developers defined their own AI-augmented work, producing a profile of 12 work goals, 75 concrete tasks, and the skills and knowledge each task demands. The paper's central claim is that these skills organize into four domains — using generative AI effectively, core software engineering, adjacent engineering, and adjacent non-engineering — and that every task unfolds through a six-step workflow: identify, engage, evaluate, calibrate, tweak, finalize. Because the profile exists to guide teaching, the authors conclude that degree programs and on-the-job training must target all four domains, together with soft skills, to reskill, upskill, and protect against deskilling. The deeper claim is that AI augments rather than replaces the developer, but only for those who bring enough engineering and domain knowledge to act as a knowledgeable human in the loop.

What carries the argument

The load-bearing structure is the occupational profile produced by the DACUM method (Developing A CurriculUM), a facilitated consensus process in which expert workers define their own job as a grid of duties, tasks, and required skills and knowledge. The profile's 12 goals and 75 tasks are the paper's inventory of what AI-enhanced developers do. The argument's internal engine is the six-step task workflow — identify, engage, evaluate, calibrate, tweak, finalize — which the authors claim applies to every one of the 75 tasks, and the four-domain T-shaped skill map showing which domains each workflow step draws on. The workflow does the paper's central conceptual work: it reframes prompt engineering as only the engage step, and locates the real skill requirement in the surrounding steps, where domain knowledge in core software engineering, adjacent engineering, and adjacent non-engineering fields governs judging, steering, and finishing AI output.

What would settle it

Instrument the real AI-tool use of a larger, more diverse sample of developers — IDE telemetry, prompts, and pull requests — and check whether actual work decomposes into the 12 goals, 75 tasks, and six-step workflow; the claim falters if factor analysis fails to recover the four skill domains or if tasks completed while skipping the evaluate and calibrate steps show no quality cost. A sharper experiment would assign junior developers to training in prompt fluency alone versus training in all four domains and measure task outcomes, since the paper's position predicts a clear advantage for the four-domain group.

Watch

Extended reading notes

Core claim

Using the DACUM job-analysis methodology, the paper claims to have produced a validated occupational profile of the AI-enhanced software developer. Expert practitioners enumerated 12 work goals — from contextualizing a unit of work and exploring technical solutions to producing code, ensuring compliance, investigating production issues, and improving reliability — broken down into 75 tasks they carry out with AI assistance, each with the skills, knowledge, attributes, and tools required. Aggregating across tasks, the authors find that success hinges on four skill domains: fluency with generative AI itself; deep core software engineering; specialized adjacent engineering fields such as cybersecurity and regulation; and adjacent non-engineering knowledge of users, business, and markets. They further claim that all 75 tasks are executed through a common six-step workflow in which the developer identifies what the AI needs, engages it, evaluates the output, calibrates the interaction, tweaks the artifact, and finalizes with documentation. The conclusion the authors draw is that the effective AI-enhanced developer is the developer who can act as a knowledgeable human in the loop: the craft is not automated away, but the proficiency bar shifts upward toward today's senior-developer skills, applied across engineering and non-engineering domains.

Load-bearing premise

The study's load-bearing premise, which the paper itself acknowledges in its limitations section, is that the 21 expert developers described their AI-augmented work accurately and that this small, self-selected group of early adopters is representative enough to describe what the wider population of developers does or will do.

Editorial extensions

If this is right

  • University programs and corporate training will need to teach all four domains — including business, user, and regulatory knowledge — rather than treating prompt fluency as the whole skill.
  • Developers who use AI well will spend more of their time in the planning stage of the software lifecycle, acting as technical decision-makers who review and select among AI-generated options.
  • The typical AI-enhanced developer of tomorrow will need the system-design and technical-decision skills of today's senior developer, which raises the bar for junior-developer training.
  • Soft skills such as communication and collaboration become a competitive edge and can be taught without displacing technical content.
  • Because AI-generated content is replacing more authoritative human sources, deliberate investment in content knowledge is needed to prevent the erosion of the skills developers need to evaluate AI output.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The six-step workflow is stated as universal across all 75 tasks within software development; a natural extension the paper does not make is to test whether the same workflow describes other knowledge professions — legal drafting, data analysis, technical writing — where generative AI is entering daily practice.
  • If the planning-stage emphasis is correct, then as code generation improves, the binding constraint on software output shifts from writing code to interpreting requirements and judging options; that suggests evaluating training by the quality of planning decisions rather than by coding speed.
  • The four-domain structure could be turned into a quantitative rubric and scored against a larger, more diverse sample of developers to see whether each domain independently predicts task success with AI tools.
  • Combined with prior evidence that the least experienced developers can be slowed down by AI, the paper's senior-developer bar implies a testable asymmetry: upskilling in the core and adjacent domains may be a precondition before AI tools pay off for junior developers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper reports a DACUM-based occupational analysis of expert AI-enhanced software developers. The authors conducted a three-phase process involving 8 expert panelists from outside Google and 13 Google advisors, producing 12 work goals and 75 associated tasks, together with the skills, knowledge, and attributes for each task. From these findings, they derive four skill domains (generative AI usage, core software engineering, adjacent engineering, adjacent non-engineering) and a six-step human-AI task workflow (identify, engage, evaluate, calibrate, tweak, finalize). The paper argues that AI is shifting developers' work toward planning and decision-making, broadening their engagement with business and adjacent domains, and that future-proofing requires both technical depth and soft skills.

Significance. If interpreted as a description of the practices of the recruited expert panel, this is a valuable and unusually detailed empirical artifact: the full occupational profile with per-task skills and knowledge is included, the DACUM protocol is a recognized job-analysis method, and the recruitment criteria are described transparently. The paper's strengths include the public availability of the profile, the explicit limitation section, and the concrete, falsifiable inventory of tasks that can seed curriculum design and larger-sample studies. The central generalizing claim, however—that this profile is comprehensive for real-world industry conditions—is not yet supported on the evidence presented, because the goal inventory was seeded with a single company's taxonomy and the interpretation was validated largely within that company. An external coverage check or a more carefully scoped claim is needed before the broader conclusions can be accepted.

major comments (3)
  1. [Section 3.2 (Phase 1 and Phase 2)] The workshop's goal inventory was seeded with Google's 30 high-level developer goals, and the 8 panelists voted to keep 11 of those goals and add 2 new ones. Since the paper excludes the 13th goal (testing AI models), between 10 and 11 of the 12 goals discussed in Section 4.1 derive from a single company's internal taxonomy. The claim in Section 1 that the skills & knowledge reported are "comprehensive in accounting for what it takes to build software under a range of real-world, industry conditions" is therefore not supported by the elicitation design: any activities absent from Google's 30 goals (e.g., open-source ecosystem maintenance, regulated-industry compliance workflows beyond GDPR, or hardware-adjacent development) cannot appear in the 75 tasks, and any gap propagates to the four-domain and six-step abstractions derived from the task set. Section 7's limitations cover sample size, statistical validation, prior research, and self-report, but not this framing risk. The authors should either add an external coverage check against an independent taxonomy (e.g., SWEBOK knowledge areas and the Stack Overflow Developer Survey's AI task categories) or restrict the "comprehensive" claim to the organizational contexts represented by the panel and advisors, and explicitly discuss the possible omissions.
  2. [Section 4.3 and occupational profile, p. 8 (Essential Skills)] Section 4.3 presents a 6-step task workflow (Identify, Engage, Evaluate, Calibrate, Tweak, Finalize) as common across all 75 tasks. However, the accompanying occupational profile's "Essential Skills" list includes a seventh cross-cutting skill, Socialize ("Drive alignment within and between teams... Manage stakeholders"), which the paper's workflow omits without explanation. The statement that the workflow "applies to any of the 75 tasks" is thus in tension with the profile's own list of cross-cutting skills. The authors should justify the omission or incorporate Socialize into the workflow, and reconcile this with the soft-skills discussion in Section 5.
  3. [Abstract and Section 3.2] The abstract describes the study as "research with 21 developers," but the DACUM workshop that produced the 12 goals and 75 tasks was conducted with 8 panelists. The 13 Google advisors participated in Phase 1 (familiarization) and Phase 3 (validation), not in the consensus task-identification phase. Reporting 21 as the study sample conflates the validation/debrief role with the profile-producing panel and overstates the evidence base for the task inventory. The abstract and Section 7 should state the panel size (8) and the advisors' role separately.
minor comments (5)
  1. [Section 4.1] The sentence "the following 12 goals and pertinent 1 tasks in the profile apply widely" appears to contain a typo; "pertinent 1 tasks" should likely read "pertinent tasks" or "associated tasks."
  2. [Occupational profile, Section 03 "Locate information"] The three tasks under this goal are labeled 2A, 2B, and 2C, duplicating the task numbers already used under Goal 02 "Explore technical solutions." Renumber them as 3A-3C to avoid ambiguity when tasks are cited.
  3. [Figure 2] The stage-level task counts (Plan 58, Code 42, Build 11, Test 28, Release 8, Deploy 7, Operate 27, Monitor 10) sum to 191 for 75 non-mutually-exclusive tasks, but the paper provides no table or appendix mapping the 75 tasks to stages. Provide the mapping or a note explaining how tasks assigned to multiple stages were counted.
  4. [Section 3.2, Phase 3] The reduction from 91 to 80 tasks and the subsequent exclusion of the 5 tasks of goal 13 are not explicitly reconciled with the abstract's "75 tasks"; state the arithmetic (91 minus 11 duplicated equals 80, then 80 minus 5 equals 75) near the first mention of the counts.
  5. [Section 4.1] The numbered list of 12 goals follows a different order from the occupational profile's Section 02 numbering (e.g., "Produce high-quality code" is Goal 5 here but Goal 01 in the profile); a cross-reference table would improve usability for readers who consult the accompanying profile.

Circularity Check

1 steps flagged · score 4.0 of 10

The goal inventory is partially circular: at least 10 of the 12 reported 'uncovered' goals are selections from Google's pre-supplied 30-goal list, though the task and skills content is independently elicited.

  1. self definitional [Section 3.2 (Phase 1 and Phase 2); Section 4.1 (12 goals); Abstract]
    "We asked our 13 advisors to identify how they were already using AI to enhance business value in Google's 30 high-level software development goals. ... The 8 panelists first reviewed and edited the straw occupational definition, then voted to identify 11 of the Google developer goals and 2 new goals they added to best describe their work as AI-enabled software developers."

    The abstract's headline is '12 of their work goals we uncovered', but Phase 1 supplied Google's 30-goal list as the starting frame and Phase 2 panelists voted to keep 11 of those goals, adding only 2. After Section 4.1 excludes the 13th goal (testing AI models), at least 10 of the 12 reported goals (and possibly 11) are the same Google-supplied categories placed in front of the panelists. The 'uncovered' goal taxonomy is therefore substantially a selection and re-description of the input list rather than open-ended discovery, and the 75 tasks and four skill domains are organized under those pre-existing goals.

full rationale

The paper's main independent content is at the task and skill level: eight non-Google panelists decomposed the goals into 91 workshop tasks (later consolidated and reported as 75) and generated per-task skills, knowledge, attributes, and tools, so the four domains and the six-step workflow are inductive summaries of those data rather than quantities fitted to the outcome. The paper's self-citations to prior EDC DACUM profiles are not load-bearing because the DACUM method is separately grounded in external references (Norton; Ohio State DACUM center). Section 7 openly discloses small sample, self-report, and lack of statistical validation, which are validity limitations rather than circularity. The one genuine circular step is the goal inventory: since the workshop's goals were seeded by Google's 30-goal list and the panelists mostly voted to retain those goals, the headline '12 goals uncovered' is partly the input frame re-labeled as a finding. The task, skill, and workflow content is not forced by that frame, so the appropriate score is a moderate 4, not a higher forced-by-construction score.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new particles, forces, or mathematical objects. Its load-bearing assumptions are qualitative: expert self-report accuracy, panel representativeness, and the use of Google-seeded goals as the workshop starting point. The four domains and six-step workflow are conceptual categories, not invented entities with falsifiable handles.

assumptions (5)
  • domain assumption Expert workers can describe and define their jobs more accurately than anyone else.
    Core DACUM principle introduced in Section 3. If this is false, the entire self-report-based occupational profile is unreliable.
  • domain assumption The 8 panelists and 13 advisors are expert-level AI-enhanced developers and can comprehensively describe the occupation.
    Stated in Section 3.1 recruitment criteria. If the panel is not representative, the 12 goals and 75 tasks may miss major parts of the occupation.
  • domain assumption Self-reported AI use and work examples are accurate and not distorted by selective memory, attribution bias, or exaggeration.
    Acknowledged in Section 7 Limitations: the study relies on self-report and was not independently verified.
  • ad hoc to paper The 30 Google developer goals used as a starting instrument are a reasonable seed set for the occupation.
    Section 3.2 Phase 1: the advisors worked from Google's 30 high-level goals. This organizational frame may bias which goals and tasks appear in the profile.
  • domain assumption The 13th goal, testing AI models, is specialized enough to exclude from the main analysis.
    Section 4.1 explicitly excludes goal 13 because it requires AI/ML specialist skills. If the excluded goal is actually common among professional developers, the 75-task count underrepresents the occupation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of What do professional software developers need to know to succeed in an age of Artificial Intelligence?." pith.science (2026). https://pith.science/paper/OJL5PSFI

@misc{pith2026250600202,
  author       = {Pith},
  title        = {Pith review of: What do professional software developers need to know to succeed in an age of Artificial Intelligence?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OJL5PSFI}},
  note         = {Machine review of arXiv:2506.00202}
}
read the original abstract

Generative AI is showing early evidence of productivity gains for software developers, but concerns persist regarding workforce disruption and deskilling. We describe our research with 21 developers at the cutting edge of using AI, summarizing 12 of their work goals we uncovered, together with 75 associated tasks and the skills & knowledge for each, illustrating how developers use AI at work. From all of these, we distilled our findings in the form of 5 insights. We found that the skills & knowledge to be a successful AI-enhanced developer are organized into four domains (using Generative AI effectively, core software engineering, adjacent engineering, and adjacent non-engineering) deployed at critical junctures throughout a 6-step task workflow. In order to "future proof" developers for this age of AI, on-the-job learning initiatives and computer science degree programs will need to target both "soft" skills and the technical skills & knowledge in all four domains to reskill, upskill and safeguard against deskilling.

Figures

Figures reproduced from arXiv: 2506.00202 by the authors.

Figure 1
Figure 1. This Figure shows the portion of the occupational pro [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. One way to better understand and appreciate how [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. The T-shape is a common visualization used to rep [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The AI-enhanced developer (of tomorrow) will [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages

  1. [1]

    Example: I prompt an AI image generator to give examples of different tigers to obtain an exhaustive list of the tigers to see all styles and angles, positions, etc Human prompts the AI to generate example assets based on their needs AI generates these examples Human reviews the results and re-iterates with different prompts if needed. - Scope the project...

    work page 2025

  2. [2]

    ACM, New York, NY, USA, 12 pages

    Trondheim, Norway. ACM, New York, NY, USA, 12 pages. https://doi.org/10.1145/3696630.3727251 Google Contact Matthew Kam mattkam@google.com EDC Contact Joyce Malyn-Smith jmsmith@edc.org 50 Acknowledgments | Defining what experts at AI-enhanced software development do and know

    work page arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.