Pith. sign in

REVIEW 4 cited by

A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challenges

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2303.17125 v2 pith:FYGWRUDP submitted 2023-03-30 cs.SE cs.AIcs.HC

Jenny T. Liang , Chenyang Yang , Brad A. Myers This is my paper
classification cs.SEcs.AIcs.HC
keywords developersprogrammingassistantstoolsbecausetheyusabilitychallenges
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The software engineering community recently has witnessed widespread deployment of AI programming assistants, such as GitHub Copilot. However, in practice, developers do not accept AI programming assistants' initial suggestions at a high frequency. This leaves a number of open questions related to the usability of these tools. To understand developers' practices while using these tools and the important usability challenges they face, we administered a survey to a large population of developers and received responses from a diverse set of 410 developers. Through a mix of qualitative and quantitative analyses, we found that developers are most motivated to use AI programming assistants because they help developers reduce key-strokes, finish programming tasks quickly, and recall syntax, but resonate less with using them to help brainstorm potential solutions. We also found the most important reasons why developers do not use these tools are because these tools do not output code that addresses certain functional or non-functional requirements and because developers have trouble controlling the tool to generate the desired output. Our findings have implications for both creators and users of AI programming assistants, such as designing minimal cognitive effort interactions with these tools to reduce distractions for users while they are programming.

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Exploring the Challenges and Opportunities of AI-assisted Codebase Generation

    cs.SE 2025-08 conditional novelty 6.0 of 10

    Developers prompting codebase-level AI assistants are often dissatisfied with generated code, citing missing functionality, poor code quality, and communication gaps, despite varied prompting strategies.

  2. The Effects of GitHub Copilot on Computing Students' Programming Effectiveness, Efficiency, and Processes in Brownfield Programming Tasks

    cs.SE 2025-06 conditional novelty 6.0 of 10

    GitHub Copilot made undergraduate students faster and more test-successful on brownfield programming tasks, and shifted their workflow from manual coding and web search to prompting, reviewing, and integrating AI suggestions.

  3. OnGoal: Tracking and Visualizing Conversational Goals in Multi-Turn Dialogue with Large Language Models

    cs.HC 2025-08 reject novelty 5.0 of 10

    OnGoal is an LLM chat interface that infers, merges, and evaluates user goals in real time and visualizes their progress, tested with 20 users on a writing task.

  4. A Comprehensive Survey of Deep Research: Systems, Methodologies, and Applications

    cs.AI 2025-06 conditional novelty 4.0 of 10

    A survey of 80+ Deep Research systems that proposes a four-layer taxonomy (foundation models, tool use, planning, synthesis) and compares commercial and open-source implementations.

Pith tools