Interleaving Natural Language Prompting with Code Editing for Solving Programming Tasks with Generative AI Models

Adish Singla; Ahana Ghosh; Alkis Gotovos; Andrew Luxton-Reilly; James Prather; Juho Leinonen; Paul Denny; Victor-Alexandru P\u{a}durean

REVIEW 3 major objections 5 minor 2 cited by

Interleaving Natural Language Prompting with Code Editing for Solving Programming Tasks with Generative AI Models

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper reports that students in AI-assisted programming typically prompt for an initial solution and then enter short manual edit-run loops after generated code fails, with edits functioning as last-mile repair.

desk verdict First large-scale look at how students interleave prompting and editing; RQ1 is solid, RQ2's complexity claim is confounded by category content and optionality. read the letter →

arxiv 2509.14088 v3 pith:VUJZOB3J submitted 2025-09-17 cs.CY

Victor-Alexandru Pădurean , Alkis Gotovos , Ahana Ghosh , Paul Denny , Juho Leinonen , Andrew Luxton-Reilly , James Prather , Adish Singla This is my paper · ORCID

classification cs.CY

keywords generativeAInaturallanguagepromptingcodeeditingcomputingeducationLLM-assistedprogramminglast-milerepairstudentbehaviorpromptproblems

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports a large classroom study of how novice programmers combine natural-language prompting and direct code editing when solving tasks with generative AI. Using a custom platform that logs every prompt, code run, edit, and execution, the authors collected 13,305 interactions from 355 students over a three-day lab exercise. They find that students typically use prompts to obtain an initial solution, then react to failed test runs by entering short edit-run loops in which they make small manual corrections, often on a single line. Editing becomes more common as task difficulty rises and among lower-performing students, while higher performers more often succeed by prompting alone. Student reflections characterize prompting as fast but imprecise and editing as the way to achieve exact, working code.

What carries the argument

The platform's interaction model is the load-bearing instrument: a chat-like editor in which students can send natural-language messages, run generated code against hidden tests, load the latest code into an editor, execute edited code, and optionally attach edited code to a follow-up prompt. The analysis compresses each session into a state-transition diagram over seven states (Start/Reset, Msg, MsgAtt, RunLLM, Edit, RunEdit, Succ/Fail), then counts adjacent transitions to reveal dominant loops. A line-level Levenshtein distance, the minimum number of line insertions, deletions, and substitutions between two code versions, quantifies edit size. This machinery lets the authors distinguish edits made in reaction to a failed LLM run from edits made independently, and to measure complexity and competence effects on editing frequency, size, and placement.

What would settle it

In a replication where all students are required to attempt all three difficulty categories, if the rate of manual editing no longer increases with difficulty, the paper's complexity claim is refuted; a randomized A/B arm with the editor disabled would further test whether editing actually causes higher success.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is a behavioral pattern: AI-assisted programming in this educational setting is a mixed-initiative workflow, not a one-shot generation. Students mostly begin with a natural-language prompt, run the LLM-generated code, and only if it fails do they switch to manual editing. The edit phase is short and narrowly targeted: most edits change a single line and are followed by few further messages, so editing functions as last-mile repair. This pattern strengthens with task complexity and weakens with student competence: harder tasks draw more editing, and higher-scoring students solve more problems without any edits. The authors interpret this as evidence that stronger programmers write prompts precise enough that generated code needs little correction.

Load-bearing premise

The comparison of editing across task difficulty assumes the same kinds of students attempted all three difficulty levels, but the harder levels were optional, so students who tried them may differ in ability or motivation from those who did not.

Editorial extensions

If this is right

Curriculum design should treat both prompt writing and quick manual repair as core skills, because student workflows interleave them rather than relying on one.
AI programming tools should make the edit-run loop a first-class interaction, since small targeted edits after a failed run are a dominant pattern.
Prompt precision appears to be a teachable skill that reduces downstream need for edits, since higher-performing students succeed more often by prompting alone.
Studies of AI-assisted learning that log only prompts or final code will miss the edit-run loop where much debugging and learning occurs.
For harder tasks, educators should expect and support manual editing rather than viewing it as a failure to use AI.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If this pattern extends beyond the classroom, human value in AI-assisted programming shifts toward validation and targeted correction, and tools that accelerate edit-run iteration may matter more than improving one-shot generation.
The competence result leaves two explanations open: stronger students may write better prompts, or they may be better at deciding when a failure is cheaper to fix by editing than by re-prompting; a study that records prompt revisions separately could separate these.
The optional-task confound the paper acknowledges can be tested directly by making all difficulty levels mandatory; if the complexity gradient survives, the editing-complexity claim is robust.
The single-line edit result suggests an instrumented editor that offers syntax-aware micro-edits could be a productive intervention for novices, though the paper does not test this.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Quick take: this is a worthwhile observational study and the first dataset I know of that lets you see, at scale, how students toggle between prompting an LLM and hand-editing generated code. The RQ1 finding—students mostly prompt for the first solution and then fall into short edit-run loops after a failed execution—is well supported by the transition diagrams and by the student reflections. That alone is enough to make the paper useful.

What's new: the platform itself adds editable code to the Prompt Problems line of work, and the analysis quantifies interleaving patterns, edit sizes, and cohort differences. The edit-size result is clean: most edits touch one line, and the authors count lines with a line-level Levenshtein distance, which is a reasonable, transparent choice. The 52-student common-subset check shows the main patterns are not solely an artefact of who opted into the harder tasks.

Soft spots, in proportion. The RQ2 complexity claim is the weakest link. The three categories differ in structure and topic, not just in difficulty: multi-function problems, classes, data structures. So "editing is more frequent for harder tasks" is really "editing is more frequent in the Classes & Functions category," and the visualizations can't separate complexity from category content. The paper acknowledges the self-selection confound, but not this category-content confound. On top of that, all the RQ2 comparisons are descriptive: no effect sizes or intervals, so it's hard to know whether e.g. the 50% vs 30% editing difference in the common subset is stable. The competence split is also coarse—rounded median split on test scores—though the direction of the finding is plausible and the qualitative data support it. These are fixable with a mandatory cross-difficulty design or at least adjusted comparisons, not fatal flaws.

The citation pattern looks fine; the prior platform papers are background, not load-bearing, and the observational claims are anchored to new interaction logs.

Who it's for: computing-education researchers and people building AI-assisted programming tools. It deserves a serious referee. I'd send it out, and I'd ask for a revision that either reframes RQ2 as a comparison across problem categories or adds the inferential statistics to back the difficulty language.

Recommendation: engage with it.

Referee Report

3 major / 5 minor

Summary. This paper reports an observational study of 355 students solving nine 'Prompt Problems' on a custom platform that supports both natural-language prompting of GPT-4o mini and manual code editing. The authors collected 13,305 logged interactions over a three-day lab activity, plus 373 open-ended student reflections and course test scores. Their three research questions address how students combine prompting and editing (RQ1), how editing varies with task complexity and student competence (RQ2), and how students perceive the two modes (RQ3). The main findings are that students typically use prompts to obtain initial code and switch to short edit-run loops after failed executions, that edits are usually small (often single-line), that editing is more common on harder/optional problem categories and among lower-scoring students, and that students see prompting and editing as complementary. The paper concludes by arguing that AI-assisted programming is a mixed-initiative process in which the human's distinctive contribution is targeted 'last-mile' repair.

Significance. If the descriptive findings are taken as reported, the paper makes a useful empirical contribution: it moves beyond prompt-only studies by instrumenting a platform that explicitly supports editing, and it provides a concrete, quantified account of the edit-run loop that prior work on AI coding assistants discusses only anecdotally. The state-transition visualization of genuine interactions, the 52-student common-subset robustness check, and the triangulation with student reflections are genuine strengths. However, the RQ2 claims about task complexity and student competence carry interpretative weight in the abstract and conclusion, and those claims are currently undermined by confounded category comparisons and a lack of inferential statistics. The study is best described as a well-motivated observational characterization rather than an experimental test of the complexity or competence hypotheses.

major comments (3)

[Sec. 4.2, Fig. 3a–d; Answer to RQ2] The headline claim that 'editing was more frequent for harder tasks' is not supported by the analyses as presented. The three problem categories (Basic Functions, Advanced Functions, Classes & Functions) differ in structural demands and topic, not only in difficulty: the latter categories include multi-function problems, classes, and data structures. In addition, Advanced Functions and Classes & Functions were optional, so the populations attempting each category are self-selected; the repeated 52-student common-subset analysis controls for stable student differences but does not remove the category-content confound or the selection into attempting harder categories. The comparisons are also purely descriptive, with no effect sizes, confidence intervals, or significance tests. Please either report inferential statistics and an analysis that separates difficulty from category structure, or rephrase the RQ2 answer as a category-level descriptive finding.
[Sec. 3.3, Figs. 3e–3h] The competence split is based on median test scores rounded to 8, producing groups of 180 and 150 students, and all comparisons in Figures 3e–3h are visual/descriptive. The claim that higher-performing students 'succeed with less reliance on editing' may reflect their higher overall success rate rather than a different editing strategy; without conditioning on eventual success or reporting effect sizes and tests, this is not established. Please add inferential statistics and an analysis that separates success rate from editing tendency.
[Sec. 4.2, Figs. 3c and 3g] The definition of 'lines changed per edit' includes zero-line changes (students executed or attached code unchanged, or reverted to the original). Counting these as edits makes the distribution of edit sizes and the claim that 'most edits are single-line' difficult to interpret. Please report the proportion of recorded edit events with zero net change and re-run the distribution excluding them, or justify why they should count as edits.

minor comments (5)

[Abstract and Sec. 3.3] There are formatting typos such as 'collecting13 , 305interactions' that should be corrected.
[Sec. 4.2] The common-subset analysis on 52 students is mentioned in the text but not visualized; please include the figure or a table with exact numbers for reproducibility.
[Sec. 4.1] The term 'session' (a student attempting a problem over possibly multiple conversations) should be defined before Figure 2 is discussed.
[Sec. 3.2] The manuscript does not state whether institutional ethics approval and informed consent were obtained for collecting student interaction logs and reflections; this information should be added.
[General] No data availability statement is provided; given the observational nature of the study, making anonymized logs or analysis code available would strengthen reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's empirical claims rest on external interaction logs and test scores, not on fitted parameters or self-cited derivations.

full rationale

The paper is an observational study of 13,305 logged interactions and 373 student reflections. Its central claims—that students prompt for initial solutions, edit after failed executions, edit more on harder tasks, and that higher-scoring students edit less—are summaries of collected data, not consequences of any fitted parameter or equation. The editing feature and the interleaving analysis are new relative to the cited Prompt Problems/Prompt Programming work; citations to Denny et al. [6] and Padurean et al. [24] supply task design and platform context but do not determine the empirical results. The optional-task self-selection limitation acknowledged in Section 4.4 is a validity concern, not a circularity. No step in the paper defines a claimed result in terms of its own inputs, so no circular step is present.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the platform's behavioral logging, the LLM/test environment, the competence proxy, and the ordinal difficulty assumption. No new theoretical entities are introduced. One hand-chosen threshold (test score >= 8) defines the competence cohorts.

free parameters (1)

Competence split threshold = 8 (median 7.67 rounded to nearest whole number)
Chosen in Section 3.3 to split 330 students into 180 high and 150 low performers; the rounding affects group membership and the RQ2 cohort comparisons.

assumptions (4)

domain assumption GPT-4o mini serves as the code generator and hidden test cases define task success.
The study's outcome labels (success/failure) depend on this model and test suite; results may not transfer to other AI models or evaluation setups (Section 3.1).
domain assumption Average score on three invigilated code-writing tests is a valid proxy for programming competence.
Used to split students into high and low competence cohorts in Section 3.3; test performance may not capture all relevant skills.
domain assumption Only edits that are executed or attached to a message represent meaningful editing.
The platform logs only these edits, excluding intermediate drafts, so editing effort may be undercounted (Section 3.2).
domain assumption The three problem categories are ordered by increasing complexity.
Basic Functions, Advanced Functions, and Classes & Functions are treated as increasing difficulty without independent validation (Section 3.2).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Interleaving Natural Language Prompting with Code Editing for Solving Programming Tasks with Generative AI Models." pith.science (2026). https://pith.science/paper/VUJZOB3J

@misc{pith2026250914088,
  author       = {Pith},
  title        = {Pith review of: Interleaving Natural Language Prompting with Code Editing for Solving Programming Tasks with Generative AI Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VUJZOB3J}},
  note         = {Machine review of arXiv:2509.14088}
}

read the original abstract

Modern computing students often rely on both natural-language prompting and manual code editing to solve programming tasks. Yet we still lack a clear understanding of how these two modes are combined in practice, and how their usage varies with task complexity and student ability. In this paper, we investigate this through a large-scale study in an introductory programming course, collecting 13,305 interactions from 355 students during a three-day lab activity. Our analysis shows that students primarily use prompting to generate initial solutions, and then often enter short edit-run loops to refine their code following a failed execution. Student reflections confirm that prompting is helpful for structuring solutions, editing is effective for making targeted corrections, while both are useful for learning. We find that manual editing becomes more frequent as task complexity increases, but most edits remain concise, with many affecting a single line of code. Higher-performing students succeed with less reliance on editing and fewer overall interactions. These findings highlight the role of manual editing as a form of last-mile repair, complementing prompting in AI-assisted programming workflows.

Figures

Figures reproduced from arXiv: 2509.14088 by the authors.

**Figure 1.** Illustration of a student’s interaction combining prompting and editing. (a) presents the specification for the ‘sort sub [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 2.** Patterns of prompting and editing across tasks. (a) presents aggregate transitions between interaction states: Start/Reset, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Editing activity grouped two ways: problem category (top row, a–d) and student cohort (bottom row, e–h). (a,e) show [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Student preference for NL prompt messages vs. code [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Commenting with Copilot: A Taxonomy and Multi-Year Analysis of Student Code-Generation Specifications
cs.SE 2026-07 conditional novelty 6.0 of 10

In four years of Copilot tasks, students wrote mostly natural-language What comments, used more How comments for procedural constructs, and focused effort on verifying output rather than rewriting comments.
Say What? Examining Text and Voice Input Modalities for Prompt-Based Programming in Computing Education
cs.CY 2026-07 unverdicted novelty 6.0 of 10

Among 919 intro CS students solving Prompt Problems, typed prompts beat unedited voice on first-attempt success for two of three tasks; edited voice matched text, and most preferred text.

Reference graph

Works this paper leans on

38 extracted references · 35 canonical work pages · cited by 2 Pith papers

[1]

Cagla Acun and Ramazan Acun. 2023. GAI-Enhanced Assignment Framework: A Case Study on Generative AI Powered History Education. InNeurIPS’23 Workshop on Generative AI for Education

work page 2023
[2]

Matin Amoozadeh, Daye Nam, Daniel Prol, Ali Alfageeh, James Prather, Michael Hilton, Sruti Srinivasa Ragavan, and Amin Alipour. 2024. Student-AI Interaction: A Case Study of CS1 Students. InProceedings of the Koli Calling International Conference on Computing Education Research (Koli Calling)

work page 2024
[3]

Becker, Paul Denny, Raymond Pettit, Durell Bouchard, Dennis J

Brett A. Becker, Paul Denny, Raymond Pettit, Durell Bouchard, Dennis J. Bouvier, Brian Harrington, Amir Kamil, Amey Karkare, Chris McDonald, Peter-Michael Osera, Janice L. Pearce, and James Prather. 2019. Compiler Error Messages Considered Unhelpful: The Landscape of Text-Based Programming Error Mes- sage Research. InProceedings of the Working Group Repor...

work page 2019
[4]

Doga Cambaz and Xiaoling Zhang. 2024. Use of AI-driven Code Generation Mod- els in Teaching and Learning Programming: a Systematic Literature Review. In Proceedings of the Technical Symposium on Computer Science Education (SIGCSE)

work page 2024
[5]

Heffernan, Tanja Käser, Steven Moore, Anna N

Paul Denny, Sumit Gulwani, Neil T. Heffernan, Tanja Käser, Steven Moore, Anna N. Rafferty, and Adish Singla. 2024. Generative AI for Education (GAIED): Advances, Opportunities, and Challenges.CoRRabs/2402.01580 (2024)

arXiv 2024
[6]

Becker, and Brent N

Paul Denny, Juho Leinonen, James Prather, Andrew Luxton-Reilly, Thezyrie Amarouche, Brett A. Becker, and Brent N. Reeves. 2024. Prompt Problems: A New Programming Exercise for the Generative AI Era. InProceedings of the Technical Symposium on Computer Science Education (SIGCSE)

work page 2024
[7]

Tempero, and Jacob Hendrickx

Paul Denny, Andrew Luxton-Reilly, Ewan D. Tempero, and Jacob Hendrickx. 2011. Understanding the syntax barrier for novices. InProceedings of the Innovation and Technology in Computer Science Education Conference (ITiCSE)

work page 2011
[8]

Paul Denny, Stephen MacNeil, Jaromir Savelka, Leo Porter, and Andrew Luxton- Reilly. 2024. Desirable Characteristics for AI Teaching Assistants in Programming Education. InProceedings Companion of the Conference on Innovation and Tech- nology in Computer Science Education (ITiCSE)

work page 2024

Show all 38 references

[9]

Jacob Doughty, Zipiao Wan, Anishka Bompelli, Jubahed Qayum, Taozhi Wang, Juran Zhang, Yujia Zheng, Aidan Doyle, Pragnya Sridhar, Arav Agarwal, Christo- pher Bogart, Eric Keylor, Can Kültür, Jaromír Savelka, and Majd Sakr. 2024. A Comparative Study of AI-Generated (GPT-4) and H...

2024
[10]

John Edwards, Joseph Ditton, Dragan Trninic, Hillary Swanson, Shelsey Sulli- van, and Chad D. Mano. 2020. Syntax Exercises in CS1. InProceedings of the International Computing Education Research Conference (ICER)

2020
[11]

Hans Hüttel. 2025. On Program Synthesis and Large Language Models.Commun. ACM(2025)

2025
[12]

Henley, Barbara Jane Ericson, David Weintrop, and Tovi Grossman

Majeed Kazemitabaar, Xinying Hou, Austin Z. Henley, Barbara Jane Ericson, David Weintrop, and Tovi Grossman. 2023. How Novices Use LLM-based Code Generators to Solve CS1 Coding Tasks in a Self-Paced Learning Environment. In Proceedings of the Koli Calling International Confere...

2023
[13]

Majeed Kazemitabaar, Runlong Ye, Xiaoning Wang, Austin Zachary Henley, Paul Denny, Michelle Craig, and Tovi Grossman. 2024. CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needs. InProceedings of the Conferenc...

2024
[14]

Smith, James Prather, Juho Leinonen, Andrew Luxton-Reilly, and Stephen MacNeil

Chris Kerslake, Paul Denny, David H. Smith, James Prather, Juho Leinonen, Andrew Luxton-Reilly, and Stephen MacNeil. 2024. Integrating Natural Language Prompting Tasks in Introductory Programming Courses. InProceedings of the Virtual Global Computing Education Conference (SIGC...

2024
[15]

Juho Leinonen, Paul Denny, Stephen MacNeil, Sami Sarsa, Seth Bernstein, Joanne Kim, Andrew Tran, and Arto Hellas. 2023. Comparing Code Explanations Created by Students and Large Language Models. InProceedings of the Conference on Innovation and Technology in Computer Science E...

2023
[16]

Qianou Ma, Hua Shen, Kenneth Koedinger, and Sherry Tongshuang Wu. 2024. How to Teach Programming in the AI Era? Using LLMs as a Teachable Agent for Debugging. InProceedings of the International Conference on Artificial Intelligence in Education (AIED)

2024
[17]

Stephen MacNeil, Andrew Tran, Arto Hellas, Joanne Kim, Sami Sarsa, Paul Denny, Seth Bernstein, and Juho Leinonen. 2023. Experiences from Using Code Explanations Generated by Large Language Models in a Web Software Develop- ment E-Book. InProceedings of the Technical Symposium ...

2023
[18]

Markel, Steven G

Julia M. Markel, Steven G. Opferman, James A. Landay, and Chris Piech. 2023. GPTeach: Interactive TA Training with GPT-based Students. InProceedings of the Conference on Learning @ Scale (L@S)

2023
[19]

Manh Hung Nguyen, Sebastian Tschiatschek, and Adish Singla. 2024. Large Language Models for In-Context Student Modeling: Synthesizing Student’s Be- havior in Visual Programming from One-Shot Observation. InProceedings of the International Conference on Educational Data Mining (EDM)

2024
[20]

Manh Hung Nguyen, Alkis Gotovos Victor-Alexandru Pădurean, Sebastian Tschi- atschek, and Adish Singla. 2025. Synthesizing High-Quality Programming Tasks with LLM-based Expert and Student Agents.CoRRabs/2504.07655 (2025)

2025 arXiv
[21]

Sydney Nguyen, Hannah McLean Babe, Yangtian Zi, Arjun Guha, Carolyn Jane Anderson, and Molly Q. Feldman. 2024. How Beginning Programmers and Code LLMs (Mis)read Each Other. InProceedings of the Conference on Human Factors in Computing Systems (CHI)

2024
[22]

Olney, and Vasile Rus

Priti Oli, Rabin Banjade, Andrew M. Olney, and Vasile Rus. 2025. Can LLMs Identify Gaps and Misconceptions in Students’ Code Explanations?CoRR abs/2501.10365 (2025)

2025 arXiv
[23]

OpenAI. 2024. GPT-4o mini: Advancing Cost-efficient Intelligence. https:// openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/

2024
[24]

Victor-Alexandru Padurean, Paul Denny, Alkis Gotovos, and Adish Singla. 2025. Prompt Programming: A Platform for Dialogue-based Computational Problem Solving with Generative AI Models. InProceedings of the Innovation and Technol- ogy in Computer Science Education Conference (ITiCSE)

2025
[25]

Victor-Alexandru Pădurean, Paul Denny, and Adish Singla. 2025. BugSpotter: Au- tomated Generation of Code Debugging Exercises. InProceedings of the Technical Symposium on Computer Science Education (SIGCSE)

2025
[26]

Victor-Alexandru Pădurean, Georgios Tzannetos, and Adish Singla. 2024. Neural Task Synthesis for Visual Programming.Transactions on Machine Learning Research (TMLR)(2024)

2024
[27]

Tung Phung, José Cambronero, Sumit Gulwani, Tobias Kohn, Rupak Majumdar, Adish Singla, and Gustavo Soares. 2023. Generating High-Precision Feedback for Programming Syntax Errors using Large Language Models. InProceedings of the International Conference on Educational Data Mining (EDM)

2023
[28]

Becker, Ibrahim Albluwi, Michelle Craig, Hieke Keuning, Natalie Kiesler, Tobias Kohn, Andrew Luxton- Reilly, Stephen MacNeil, Andrew Petersen, Raymond Pettit, Brent N

James Prather, Paul Denny, Juho Leinonen, Brett A. Becker, Ibrahim Albluwi, Michelle Craig, Hieke Keuning, Natalie Kiesler, Tobias Kohn, Andrew Luxton- Reilly, Stephen MacNeil, Andrew Petersen, Raymond Pettit, Brent N. Reeves, and Jaromír Savelka. 2023. The Robots Are Here: Na...

2023
[29]

James Prather, Brent N. Reeves, Paul Denny, Juho Leinonen, Stephen MacNeil, Andrew Luxton-Reilly, João Orvalho, Amin Alipour, Ali Alfageeh, Thezyrie Amarouche, Bailey Kimmel, Jared Wright, Musa Blake, and Gweneth Barbre
[30]

Reeves, Juho Leinonen, Stephen MacNeil, Arisoa S

James Prather, Brent N. Reeves, Juho Leinonen, Stephen MacNeil, Arisoa S. Ran- drianasolo, Brett A. Becker, Bailey Kimmel, Jared Wright, and Ben Briggs. 2024. The Widening Gap: The Benefits and Harms of Generative AI for Novice Pro- grammers. InProceedings of the Conference on...

2024
[31]

Mitchell

Robin Schmucker, Meng Xia, Amos Azaria, and Tom M. Mitchell. 2024. Ruffle &Riley: Insights from Designing and Evaluating a Large Language Model-Based Conversational Tutoring System. InProceedings of the International Conference on Artificial Intelligence in Education (AIED)

2024
[32]

Griswold, and Adalbert Gerald Soosai Raj

Anshul Shah, Anya Chernova, Elena Tomson, Leo Porter, William G. Griswold, and Adalbert Gerald Soosai Raj. 2025. Students’ Use of GitHub Copilot for Working with Large Code Bases. InProceedings of the Technical Symposium on Computer Science Education (SIGCSE)

2025
[33]

Smith IV, Mounika Padala, Christine Alvarado, Jamie Gorson Benario, and Leo Porter

Annapurna Vadaparty, Daniel Zingaro, David H. Smith IV, Mounika Padala, Christine Alvarado, Jamie Gorson Benario, and Leo Porter. 2024. CS1-LLM: Integrating LLMs into CS1 Instruction. InProceedings of the Innovation and Technology in Computer Science Education Conference (ITiCSE)

2024
[34]

Mitchell, and Chris Piech

Sierra Wang, John C. Mitchell, and Chris Piech. 2024. A Large Scale RCT on Effective Error Messages in CS1. InProceedings of the Technical Symposium on Computer Science Education (SIGCSE)

2024
[35]

Daniel M. Yellin. 2023. The Premature Obituary of Programming.Commun. ACM(2023)

2023
[36]

Zamfirescu-Pereira, Laryn Qi, Bjorn Hartmann, John DeNero, and Narges Norouzi

J.D. Zamfirescu-Pereira, Laryn Qi, Bjorn Hartmann, John DeNero, and Narges Norouzi. 2023. Conversational Programming with LLM-Powered Interactive Support in an Introductory Computer Science Course. InNeurIPS’23 Workshop on Generative AI for Education

2023
[37]

Jialu Zhang, José Pablo Cambronero, Sumit Gulwani, Vu Le, Ruzica Piskac, Gus- tavo Soares, and Gust Verbruggen. 2024. PyDex: Repairing Bugs in Introductory Python Assignments using LLMs.Proceedings of the ACM on Programming Languages(2024)

2024
[2024]

Breaking the Programming Language Barrier: Multilingual Prompting to Empower Non-Native English Learners.CoRRabs/2412.12800 (2024)

2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Cagla Acun and Ramazan Acun. 2023. GAI-Enhanced Assignment Framework: A Case Study on Generative AI Powered History Education. InNeurIPS’23 Workshop on Generative AI for Education

work page 2023

[2] [2]

Matin Amoozadeh, Daye Nam, Daniel Prol, Ali Alfageeh, James Prather, Michael Hilton, Sruti Srinivasa Ragavan, and Amin Alipour. 2024. Student-AI Interaction: A Case Study of CS1 Students. InProceedings of the Koli Calling International Conference on Computing Education Research (Koli Calling)

work page 2024

[3] [3]

Becker, Paul Denny, Raymond Pettit, Durell Bouchard, Dennis J

Brett A. Becker, Paul Denny, Raymond Pettit, Durell Bouchard, Dennis J. Bouvier, Brian Harrington, Amir Kamil, Amey Karkare, Chris McDonald, Peter-Michael Osera, Janice L. Pearce, and James Prather. 2019. Compiler Error Messages Considered Unhelpful: The Landscape of Text-Based Programming Error Mes- sage Research. InProceedings of the Working Group Repor...

work page 2019

[4] [4]

Doga Cambaz and Xiaoling Zhang. 2024. Use of AI-driven Code Generation Mod- els in Teaching and Learning Programming: a Systematic Literature Review. In Proceedings of the Technical Symposium on Computer Science Education (SIGCSE)

work page 2024

[5] [5]

Heffernan, Tanja Käser, Steven Moore, Anna N

Paul Denny, Sumit Gulwani, Neil T. Heffernan, Tanja Käser, Steven Moore, Anna N. Rafferty, and Adish Singla. 2024. Generative AI for Education (GAIED): Advances, Opportunities, and Challenges.CoRRabs/2402.01580 (2024)

arXiv 2024

[6] [6]

Becker, and Brent N

Paul Denny, Juho Leinonen, James Prather, Andrew Luxton-Reilly, Thezyrie Amarouche, Brett A. Becker, and Brent N. Reeves. 2024. Prompt Problems: A New Programming Exercise for the Generative AI Era. InProceedings of the Technical Symposium on Computer Science Education (SIGCSE)

work page 2024

[7] [7]

Tempero, and Jacob Hendrickx

Paul Denny, Andrew Luxton-Reilly, Ewan D. Tempero, and Jacob Hendrickx. 2011. Understanding the syntax barrier for novices. InProceedings of the Innovation and Technology in Computer Science Education Conference (ITiCSE)

work page 2011

[8] [8]

Paul Denny, Stephen MacNeil, Jaromir Savelka, Leo Porter, and Andrew Luxton- Reilly. 2024. Desirable Characteristics for AI Teaching Assistants in Programming Education. InProceedings Companion of the Conference on Innovation and Tech- nology in Computer Science Education (ITiCSE)

work page 2024