Pith. sign in

REVIEW 4 major objections 4 minor 50 references

Students' Feedback Requests and Interactions with the SCRIPT Chatbot: Do They Get What They Ask For?

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Students solving introductory programming tasks with a purpose-built chatbot make feedback requests in a predictable sequence, and the chatbot's responses match the requested feedback type in 75% of exchanges.

desk verdict Honest, data-rich chatbot study whose headline sequence claim is not supported by the aggregated evidence—the pairwise alignment analysis is the real contribution. read the letter →

arxiv 2507.17258 v1 pith:ADFTYREC submitted 2025-07-23 cs.AI

Andreas Scholl , Natalie Kiesler This is my paper
classification cs.AI
keywords generativeAIprogrammingeducationchatbotfeedbacktypesintroductorystudentinteractionspromptdesignlearningsupport
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports on SCRIPT, a chatbot built on ChatGPT-4o-mini for novice programmers, and asks whether its responses give students the kind of feedback they request. Analysing 1,409 prompts and responses from 136 students in an introductory programming course, the authors find that feedback requests tend to arrive in a particular order: understanding task constraints, identifying relevant concepts, deciding how to proceed, finding mistakes, checking performance or results, and finally asking for the correct solution. They further report that the chatbot's output matched the requested feedback type in 75% of evaluated question-response pairs, counting cases where the bot added extra helpful feedback as matches. The authors argue this shows a purpose-built chatbot can scaffold novices through a predictable problem-solving trajectory while mostly sticking to guardrails against handing over complete solutions.

What carries the argument

The analytical machinery is a feedback-type taxonomy for programming exercises developed in prior work, whose categories (knowledge of result, knowledge of correct result, knowledge of performance, knowledge about task constraints, knowledge about concepts, knowledge about mistakes, knowledge on how to proceed, knowledge about meta-cognition) were used to code every student prompt and every chatbot response. That coding turns raw chat logs into the two main results: the flowchart of common interaction patterns that grounds the claimed request sequence, and the pair-wise comparison of requested versus generated feedback that yields the 75% alignment figure. The chatbot itself also carries part of the argument: its closed prompts, derived from earlier research on eliciting specific feedback types, channel students into known categories, while its system prompt constrains outputs to stepwise, solution-free guidance.

What would settle it

Re-code the same chat logs with an independent second coder and compute inter-rater agreement; if agreement is low or the flow changes, the sequence and alignment rest on one person's judgment. Separately, run the same tasks with a chatbot that has no predefined closed prompts; if the KTC-to-KCR sequence disappears, the sequence is produced by the menu rather than by learners.

Watch

Extended reading notes

Core claim

The authors claim that students' feedback requests in SCRIPT conversations followed a consistent sequence: they first asked about task constraints (KTC), then about required concepts (KC), then how to proceed (KH), then about their mistakes (KM), then about performance and result correctness (KP, KR), and finally requested the correct result (KCR). The chatbot's responses aligned with the requested feedback type in 47% of 891 question-response pairs, rising to 75% when 'over-matching' responses that added extra conceptual explanation or next-step guidance were included; most mismatches were solution requests (KCR) that the system was designed to refuse. The outputs were judged fully correct in 81% of task-related responses, and the system mostly followed its constraints of one step at a time, no complete solutions, and simple examples, though it occasionally provided completed templates and corrected code.

Load-bearing premise

The entire sequence and alignment results rest on one researcher's categorization of the 1,409 chat messages into feedback types, with no independent second coder to verify that the categories match student intentions.

Editorial extensions

If this is right

  • Chatbot interfaces can offer closed prompts in the discovered order, helping novices start with task constraints and move toward evaluating the correct result without struggling to formulate prompts.
  • A 75% alignment rate means the remaining quarter of exchanges are predictable targets for improvement, with solution requests being an intentional mismatch rather than a bug.
  • The stepwise, no-full-solution guardrail can be evaluated against learning outcomes; the data show partial solutions emerge during extended interactions, so 'no full solutions' is a gradual boundary, not an absolute one.
  • The observed sequence gives adaptive tutoring systems a basis for anticipating the next feedback type a student will need, potentially reducing the number of clarification questions.
  • Because closed and open prompts produced observably different conversational modes, designers can use this distinction to smooth transitions when students switch from guided prompts to free-form questions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next experiment is to test whether presenting the closed prompts in the discovered order changes learning outcomes or task completion time; the current data do not say whether the sequence is normative or merely descriptive.
  • The 75% alignment counts over-matching as success, but students may not always want the extra concept explanations; re-analysing the 151 thumbs-up/down ratings by over-match categories would show whether added KC/KH feedback is valued.
  • If the sequence is partly an artifact of the predefined prompt menu, a version of the chatbot without closed prompts would reveal the natural free-form ordering; the paper's data include open prompts but cannot fully separate menu-driven from spontaneous requests.
  • The three tasks varied in how often full solutions were granted (25% to 71%), suggesting task complexity modulates guardrail adherence; comparing constraints across task types could guide task design for AI-assisted learning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper reports an empirical study of SCRIPT, a ChatGPT-4o-mini-based chatbot for introductory programming. Data come from 136 students who generated 241 chat sessions and 1,409 prompts and responses. The authors code all prompts and responses using a feedback typology with additional inductive categories, construct an aggregated flowchart of common interaction patterns (Figure 2), and then analyze request-response alignment and adherence to system-prompt constraints. The main claims are that students' feedback requests follow a specific sequence (KTC, KC, KH, KM, KP, KR, KCR), that responses aligned with requests in 75% of pairs, and that SCRIPT adhered to its system-prompt constraints.

Significance. If the central claims held, this would be a useful empirical contribution: it would provide evidence that novice programmers request feedback in a fairly predictable order and that a guardrailed chatbot can satisfy most requests without leaking full solutions. The study's strengths include the real interaction dataset from 136 students, the open data repository, the transparent presentation of tasks and system prompt, and the candid acknowledgment of several limitations. The coding scheme is borrowed from prior work rather than newly developed, but applying it to logged chatbot-student interactions is a natural and potentially valuable extension. However, the claims as currently supported are substantially more modest than the abstract and conclusions suggest, because the sequence claim is not backed by per-session evidence and the alignment claim is inflated by counting over-matching as success.

major comments (4)
  1. [§4.1, Figure 2, §6] The claimed sequence 'KTC, KC, KH, KM, KP, KR, and finally KCR' is not derivable from the presented evidence. Figure 2 is explicitly an aggregate of patterns observed at least ten times, and the text states that it does not display individual student paths. The KH node is described as isolated because fewer than ten outgoing edges to other feedback types meet the threshold. An aggregate flowchart with a frequency cutoff cannot establish a per-session ordering, especially when the modal path may be much shorter (e.g., KTC→KC→KR). To support the sequence claim, the authors should report per-session trajectory evidence, such as the proportion of sessions that contain each transition in the claimed order, or a formal sequence analysis with confidence intervals. If the data do not support such an analysis, the Conclusion should be reworded to describe the sequence as a tentative aggregate pattern rather than an empirical finding.
  2. [§4.2, Abstract] The 75% alignment figure counts 'over-matching'—responses that provide additional unrequested feedback types—as success. The exact-match rate is 47% (417/891), overmatching adds 255/891 (≈29%), and mismatches are 22%. For a paper whose title asks 'Do They Get What They Ask For?', the primary metric should be the exact-match rate; over-matching is at best a partial success and can signal that the chatbot is not answering the specific request. The authors should report exact, over-match, partial-match, and mismatch rates separately per feedback type, and they should not headline 75% in the abstract without this decomposition.
  3. [§3.3, §5.1] All coding was performed by a single researcher (the first author), and 'coded twice by the same coder' is an intra-rater repetition, not an inter-rater reliability check. Because every result in the paper—the interaction categories, the sequence, the alignment percentages, and the adherence judgments—is derived from this coding, the absence of any second-coder agreement measure leaves the reliability of the central data unquantified. The authors should have a second coder independently code a random subsample and report an agreement statistic such as Cohen's kappa, or they should explicitly justify why this is infeasible and present this limitation much more prominently than the current one-sentence note in Section 5.1.
  4. [§4.3, Table 3, Abstract] The statement that SCRIPT 'adhered to the system prompt constraints' is inconsistent with the paper's own results: only 51% of problem-related responses contained a single step, 28% contained multiple steps without an explicit next step, complete solutions were given in 129 responses, completed templates in 95 responses, and code was corrected in 34 responses. If the intended claim is 'adhered with documented exceptions', the abstract and conclusions should say so; as written, the claim overstates RQ3's outcome. The authors should either report an explicit adherence metric with a clearly stated threshold or recast the claim as partial adherence.
minor comments (4)
  1. [Figure 2] The figure uses abbreviations (KTC, KC, KH, KM, KP, KR, KCR) without a legend; a reader unfamiliar with the typology cannot interpret the flowchart without cross-referencing the text.
  2. [§4.1 vs Table 2a] The text says sessions began with KTC in 71 cases, while Table 2a reports 158 KTC requests; please clarify whether the former counts only initial prompts of a session.
  3. [§4.2] The 891 question-response pairs should be reconciled with the total of 1,409 prompts and the 207 closed prompts, so the reader can verify how the subset was formed.
  4. [References] The reference 'Kiesler, Smith, Leinonen, Fox, MacNeil, & Ihantola (2025)' contains a typo in the title ('collaboraition') and appears to lack full publication details; please correct.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the 75% alignment and adherence results are measured from logged exchanges; the sequence claim is an interpretive aggregation, not a construction tautology.

full rationale

The paper's central empirical claims are measured from independently logged interaction data (1,409 prompts, 1,409 responses, 241 sessions, Section 4) and do not reduce to their own inputs. The 75% figure in Section 4.2 is a descriptive comparison between coded requests and coded responses; it is not a fitted parameter renamed as a prediction, and no metric is defined in terms of the target result. The RQ3 adherence results are similarly checked against stated system-prompt constraints on new data. The most relevant self-citation is the design lineage of SCRIPT's closed prompts, 'predefined based on recent research on AI-generated feedback (Lohr et al., 2025)' (Section 3.1), co-authored by the second author. This is not load-bearing circularity because those prompts are re-tested here on a different model with real student behavior, and most prompts (1202 of 1409) were free-form open input. Two inference-validity concerns are not circularity, and the paper itself flags them: (a) the claimed sequence KTC, KC, KH, KM, KP, KR, KCR is an aggregate narrative; Section 4.1 admits that 'the number of outgoing requests from KH to other feedback types was less than 10, which is why the KH node is isolated in the aggregated flowchart,' so the linear order is an interpretive reconstruction rather than an empirically exhibited per-session path; and (b) Section 3.3 notes all chat data were 'coded twice by the same coder (first author of this work)' with no inter-rater reliability check, and Section 5.1 concedes students 'were not explicitly asked to categorize their feedback types.' These affect measurement validity and inference strength but do not make any claim definitionally equivalent to its input. No uniqueness theorem is imported from overlapping authors, no ansatz is smuggled in via citation, and no known result is merely renamed. The core empirical findings stand on their own logged data, so overall circularity is negligible.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters or invented theoretical entities appear; the study's load-bearing assumptions are coding validity and prompt-as-intent, both acknowledged in the limitations section.

assumptions (4)
  • domain assumption The feedback typology of Keuning et al. (2018) is sufficient to characterize student requests and chatbot responses in this context.
    Invoked in Section 3.3 as the deductive coding scheme; the authors note it applied to almost all requests, with additional inductive categories.
  • domain assumption A single researcher's coding of prompts and responses accurately represents students' intended feedback types.
    Section 3.3 states all data were coded twice by the first author; no inter-rater reliability is reported.
  • domain assumption Students' textual prompts faithfully express their actual feedback needs.
    Used throughout RQ1 and RQ2; the authors acknowledge in Section 5.1 that students were not asked to categorize their own requests.
  • domain assumption ChatGPT-4o-mini's behavior is consistent enough across sessions that aggregate statistics are meaningful.
    Implicit in treating all 1,409 responses as comparable samples of the same system prompt constraints.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Students' Feedback Requests and Interactions with the SCRIPT Chatbot: Do They Get What They Ask For?." pith.science (2026). https://pith.science/paper/ADFTYREC

@misc{pith2026250717258,
  author       = {Pith},
  title        = {Pith review of: Students' Feedback Requests and Interactions with the SCRIPT Chatbot: Do They Get What They Ask For?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ADFTYREC}},
  note         = {Machine review of arXiv:2507.17258}
}
read the original abstract

Building on prior research on Generative AI (GenAI) and related tools for programming education, we developed SCRIPT, a chatbot based on ChatGPT-4o-mini, to support novice learners. SCRIPT allows for open-ended interactions and structured guidance through predefined prompts. We evaluated the tool via an experiment with 136 students from an introductory programming course at a large German university and analyzed how students interacted with SCRIPT while solving programming tasks with a focus on their feedback preferences. The results reveal that students' feedback requests seem to follow a specific sequence. Moreover, the chatbot responses aligned well with students' requested feedback types (in 75%), and it adhered to the system prompt constraints. These insights inform the design of GenAI-based learning support systems and highlight challenges in balancing guidance and flexibility in AI-assisted tools.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

50 extracted references · 28 canonical work pages

  1. [1]

    \ Grande, V

    alshaigy2024forgotten APACrefauthors Alshaigy, B. \ Grande, V. APACrefauthors \ 2024 . Forgotten Again: Addressing Accessibility Challenges of Generative AI Tools for People with Disabilities Forgotten again: Addressing accessibility challenges of generative ai tools for people with disabilities . Adjunct Proceedings of the 2024 Nordic Conference on Human...

    arXiv 2024

  2. [2]

    , Kiesler, N

    azaiz2024feedbackgeneration APACrefauthors Azaiz, I. , Kiesler, N. \ Strickroth, S. APACrefauthors \ 2024 . Feedback-Generation for Programming Exercises With GPT-4. Feedback-generation for programming exercises with gpt-4. New York, NY, USA Association for Computing Machinery . APACrefDOI doi:10.1145/3649217.3653594 APACrefDOI

    arXiv 2024

  3. [3]

    , Kiesler, N

    azaiz2025opensmallrigmarole APACrefauthors Azaiz, I. , Kiesler, N. , Strickroth, S. \ Zhang, A. APACrefauthors \ 2025 . Open, Small, Rigmarole -- Evaluating Llama 3.2 3B's Feedback for Programming Exercises. Open, small, rigmarole -- evaluating llama 3.2 3b's feedback for programming exercises. accepted to the International Journal of Engineering Pedagogy...

    arXiv doi:10.48550/arxiv.2504.01054 2025

  4. [4]

    \ Kaliff, A

    Bengtsson_Kaliff_2023 APACrefauthors Bengtsson, D. \ Kaliff, A. APACrefauthors \ 2023 . Assessment Accuracy of a Large Language Model on Programming Assignments. Assessment accuracy of a large language model on programming assignments. APACrefURL https://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-331000 APACrefURL

    work page 2023

  5. [6]

    APACrefauthors \ 2016

    boddy2016sample APACrefauthors Boddy, C R. APACrefauthors \ 2016 . Sample size for qualitative research Sample size for qualitative research . Qualitative market research: An international journal 19 4 426--432

    work page 2016

  6. [7]

    APACrefauthors \ 1986

    duboulay1986some APACrefauthors Du Boulay, B. APACrefauthors \ 1986 . Some difficulties of learning to program Some difficulties of learning to program . Journal of Educational Computing Research 2 1 57--73 . APACrefDOI doi:10.2190/3LFX-9RRF-67T8-UVK9 APACrefDOI

    doi:10.2190/3lfx-9rrf-67t8-uvk9 1986

  7. [8]

    \ Ring, M

    Ebert2016 APACrefauthors Ebert, M. \ Ring, M. APACrefauthors \ 2016 . A presentation framework for programming in programing lectures A presentation framework for programming in programing lectures . Proc. EDUCON Proc. educon \ ( \ 369--374)

    work page 2016

  8. [10]

    , Zhang, Y

    geng2023chatgpt APACrefauthors Geng, C. , Zhang, Y. , Pientka, B. \ Si, X. APACrefauthors \ 2023 . Can ChatGPT Pass An Introductory Level Functional Language Programming Course? Can ChatGPT Pass An Introductory Level Functional Language Programming Course?

    work page 2023

Show all 50 references
  1. [11]

    gill2024transformative APACrefauthors Gill, S S. , Xu, M. , Patros, P. , Wu, H. , Kaur, R. , Kaur, K. Buyya, R. APACrefauthors \ 2024 . Transformative effects of ChatGPT on modern education: Emerging Era of AI Chatbots Transformative effects of chatgpt on modern education: Eme...

    2024 doi

  2. [12]

    \ Jaschke, S

    jacobs2024evaluating APACrefauthors Jacobs, S. \ Jaschke, S. APACrefauthors \ 2024 05 . Evaluating the Application of Large Language Models to Generate Feedback in Programming Education Evaluating the Application of Large Language Models to Generate Feedback in Programming Edu...

    2024

  3. [13]

    , Kempf, M

    jacobs2025thatsnotthefeedback APACrefauthors Jacobs, S. , Kempf, M. \ Kiesler, N. APACrefauthors \ 2025 . That's Not the Feedback I Need! -- Student Engagement with GenAI Feedback in the Tutor Kai That's not the feedback i need! -- student engagement with genai feedback in the...

    2025 arXiv

  4. [15]

    kazemitabaar2024codeaid APACrefauthors Kazemitabaar, M. , Ye, R. , Wang, X. , Henley, A Z. , Denny, P. , Craig, M. \ Grossman, T. APACrefauthors \ 2024 . CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needs C...

    2024

  5. [16]

    APACrefauthors \ 1968

    keller1968good APACrefauthors Keller, F S. APACrefauthors \ 1968 . Good-bye, teacher... Good-bye, teacher... Journal of applied behavior analysis 1 1 79

    1968

  6. [17]

    , Jeuring, J

    keuning2018 APACrefauthors Keuning, H. , Jeuring, J. \ Heeren, B. APACrefauthors \ 2018 9 . A Systematic Literature Review of Automated Feedback Generation for Programming Exercises A systematic literature review of automated feedback generation for programming exercises . ACM...

    2018 doi

  7. [18]

    APACrefauthors \ 2022

    kiesler_diss_2022 APACrefauthors Kiesler, N. APACrefauthors \ 2022 . \ Kompetenzförderung in der Programmierausbildung durch Modellierung von Kompetenzen und informativem Feedback Kompetenzförderung in der programmierausbildung durch modellierung von kompetenzen und informativ...

    2022

  8. [19]

    APACrefauthors \ 2024

    kiesler2024modeling APACrefauthors Kiesler, N. APACrefauthors \ 2024 . Modeling Programming Competency: A Qualitative Analysis Modeling programming competency: A qualitative analysis . Cham Springer International Publishing . APACrefDOI doi:10.1007/978-3-031-47148-3 APACrefDOI

    2024 doi

  9. [20]

    , Lohr, D

    kiesler2023exploring APACrefauthors Kiesler, N. , Lohr, D. \ Keuning, H. APACrefauthors \ 2024 . Exploring the Potential of Large Language Models to Generate Formative Programming Feedback Exploring the potential of large language models to generate formative programming feedb...

    2024

  10. [21]

    \ Schiffner, D

    kiesler2023large APACrefauthors Kiesler, N. \ Schiffner, D. APACrefauthors \ 2023 1 . Large Language Models in Introductory Programming Education: ChatGPT's Performance and Implications for Assessments. Large language models in introductory programming education: Chatgpt's per...

    2023 doi arXiv

  11. [22]

    \ Schiffner, D

    kiesler2023why APACrefauthors Kiesler, N. \ Schiffner, D. APACrefauthors \ 2023 2 . Why We Need Open Data in Computer Science Education Research Why We Need Open Data in Computer Science Education Research . Proceedings of the 2023 Conference on Innovation and Technology in Co...

    2023

  12. [23]

    , Smith, J

    kiesler2025role APACrefauthors Kiesler, N. , Smith, J. , Leinonen, J. , Fox, A. , MacNeil, S. \ Ihantola, P. APACrefauthors \ 2025 . The Role of Generative AI in Software Student CollaborAItion. The role of generative ai in software student collaboraition. New York, NY, USA As...

    2025

  13. [25]

    , Sheese, B E

    liffiton2023codehelp APACrefauthors Liffiton, M. , Sheese, B E. , Savelka, J. \ Denny, P. APACrefauthors \ 2024 . CodeHelp: Using Large Language Models with Guardrails for Scalable Support in Programming Classes Codehelp: Using large language models with guardrails for scalabl...

    2024

  14. [26]

    , Zenke, C

    liu2024teaching APACrefauthors Liu, R. , Zenke, C. , Liu, C. , Holmes, A. , Thornton, P. \ Malan, D J. APACrefauthors \ 2024 . Teaching CS50 with AI: Leveraging Generative Artificial Intelligence in Computer Science Education Teaching cs50 with ai: Leveraging generative artifi...

    2024

  15. [28]

    , Keuning, H

    lohr2025youre APACrefauthors Lohr, D. , Keuning, H. \ Kiesler, N. APACrefauthors \ 2025 . You're ( Not ) My Type -- Can LLMs Generate Feedback of Specific Types for Introductory Programming Tasks ? You're ( Not ) My Type -- Can LLMs Generate Feedback of Specific Types for Intr...

    2025 doi

  16. [29]

    APACrefauthors \ 2016

    luxton-reilly2016 APACrefauthors Luxton-Reilly, A. APACrefauthors \ 2016 . Learning to Program is Easy Learning to Program is Easy . Proc. ITiCSE Proc. ITiCSE \ ( \ 284--289). APACrefDOI doi:10.1145/2899415.2899432 APACrefDOI

    2016

  17. [30]

    , Simon , Albluwi, I

    Luxton-Reilly2018 APACrefauthors Luxton-Reilly, A. , Simon , Albluwi, I. , Becker, B A. , Giannakos, M. , Kumar, A N. Szabo, C. APACrefauthors \ 2018 . Introductory Programming: A Systematic Literature Review Introductory Programming: A Systematic Literature Review . Proc. ITi...

    2018

  18. [31]

    , Wang, Y

    lyu2024evaluating APACrefauthors Lyu, W. , Wang, Y. , Chung, T R. , Sun, Y. \ Zhang, Y. APACrefauthors \ 2024 . Evaluating the Effectiveness of LLMs in Introductory Computer Science Education: A Semester-Long Field Study Evaluating the effectiveness of llms in introductory com...

    2024 arXiv

  19. [32]

    , Tran, A

    macneil2022experiences APACrefauthors MacNeil, S. , Tran, A. , Hellas, A. , Kim, J. , Sarsa, S. , Denny, P. Leinonen, J. APACrefauthors \ 2023 . Experiences from Using Code Explanations Generated by Large Language Models in a Web Software Development E-Book Experiences from Us...

    2023

  20. [33]

    APACrefauthors \ 2006

    narciss2006 APACrefauthors Narciss, S. APACrefauthors \ 2006 . Informatives Tutorielles Feedback: Entwicklungs- und Evaluationsprinzipien auf der Basis instruktionspsychologischer Erkenntnisse Informatives tutorielles feedback: Entwicklungs- und evaluationsprinzipien auf der b...

    2006

  21. [34]

    APACrefauthors \ 2008

    narciss2008feedback APACrefauthors Narciss, S. APACrefauthors \ 2008 . Feedback strategies for interactive learning tasks Feedback strategies for interactive learning tasks . Handbook of research on educational communications and technology 3 125--144

    2008

  22. [36]

    , Craig, M

    petersen2016revisiting APACrefauthors Petersen, A. , Craig, M. , Campbell, J. \ Tafliovich, A. APACrefauthors \ 2016 . Revisiting why students drop CS1 Revisiting why students drop cs1 . Proc. Koli Calling Proc. koli calling \ ( \ 71--80). APACrefDOI doi:10.1145/2999541.299955...

    2016

  23. [37]

    , Cambronero, J

    phung2023generating APACrefauthors Phung, T. , Cambronero, J. , Gulwani, S. , Kohn, T. , Majumdar, R. , Singla, A. \ Soares, G. APACrefauthors \ 2023 . Generating High-Precision Feedback for Programming Syntax Errors using Large Language Models . Generating High-Precision Feed...

    2023

  24. [38]

    , Denny, P

    prather2023wgfullreport APACrefauthors Prather, J. , Denny, P. , Leinonen, J. , Becker, B A. , Albluwi, I. , Craig, M. Savelka, J. APACrefauthors \ 2023 . The Robots Are Here: Navigating the Generative AI Revolution in Computing Education The robots are here: Navigating the ge...

    2023

  25. [39]

    , Leinonen, J

    prather2025beyond APACrefauthors Prather, J. , Leinonen, J. , Kiesler, N. , Gorson Benario, J. , Lau, S. , MacNeil, S. Zingaro, D. APACrefauthors \ 2025 . Beyond the Hype: A Comprehensive Review of Current Trends in Generative AI Research, Teaching Practices, and Tools Beyond ...

    2025

  26. [41]

    \ Rooshenas, P

    riazi2025llm APACrefauthors Riazi, S. \ Rooshenas, P. APACrefauthors \ 2025 . LLM-Driven Feedback for Enhancing Conceptual Design Learning in Database Systems Courses Llm-driven feedback for enhancing conceptual design learning in database systems courses . Proceedings of the ...

    2025

  27. [42]

    , Keuning, H

    roest2023nextstep APACrefauthors Roest, L. , Keuning, H. \ Jeuring, J. APACrefauthors \ 2023 . Next- Step Hint Generation for Introductory Programming Using Large Language Models Next- Step Hint Generation for Introductory Programming Using Large Language Models . Proceedings ...

    2023

  28. [43]

    \ Dickson, W J

    roethlisberger1939management APACrefauthors Roethlisberger, F J. \ Dickson, W J. APACrefauthors \ 1939 . Management and the Worker Management and the Worker . Cambridge Harvard University Press

    1939

  29. [44]

    , Denny, P

    Sarsa2022 APACrefauthors Sarsa, S. , Denny, P. , Hellas, A. \ Leinonen, J. APACrefauthors \ 2022 08 . Automatic Generation of Programming Exercises and Code Explanations Using Large Language Models Automatic generation of programming exercises and code explanations using large...

  30. [45]

    , Agarwal, A

    savelka2023large APACrefauthors Savelka, J. , Agarwal, A. , Bogart, C. \ Sakr, M. APACrefauthors \ 2023 . Large Language Models (GPT) Struggle to Answer Multiple-Choice Questions about Code. Large language models (gpt) struggle to answer multiple-choice questions about code

    2023

  31. [46]

    \ Kiesler, N

    scholl2024hownovice APACrefauthors Scholl, A. \ Kiesler, N. APACrefauthors \ 2024 . How Novice Programmers Use and Experience ChatGPT when Solving Programming Exercises in an Introductory Course How novice programmers use and experience chatgpt when solving programming exercis...

    2024

  32. [47]

    \ Kiesler, N

    Scholl_Kiesler_2025_datastudents APACrefauthors Scholl, A. \ Kiesler, N. APACrefauthors \ 2025 1 Jul . Data: Students’ Feedback Requests and Interactions with the SCRIPT Chatbot - Do They Get What They Ask For? Data: Students’ feedback requests and interactions with the script...

    2025 doi

  33. [48]

    \ Kiesler, N

    scholl2025script APACrefauthors Scholl, A. \ Kiesler, N. APACrefauthors \ 2025 2 . SCRIPT - Supportive Chatbot for Resolving Introductory Programming Tasks Script - supportive chatbot for resolving introductory programming tasks . Proceedings of the 30th ACM Conference on Inno...

    2025

  34. [49]

    , Schiffner, D

    scholl2024analyzing APACrefauthors Scholl, A. , Schiffner, D. \ Kiesler, N. APACrefauthors \ 2024 . Analyzing Chat Protocols of Novice Programmers Solving Introductory Programming Tasks with ChatGPT Analyzing chat protocols of novice programmers solving introductory programmin...

    2024 doi

  35. [50]

    \ Soloway, E

    spohrer1986novice APACrefauthors Spohrer, J C. \ Soloway, E. APACrefauthors \ 1986 . Novice mistakes: Are the folk wisdoms correct? Novice mistakes: Are the folk wisdoms correct? Communications of the ACM 29 7 624--632 . APACrefDOI doi:10.1145/6138.6145 APACrefDOI

    1986

  36. [52]

    , Vassar, A

    taylor2024dcchelperrorexplanations APACrefauthors Taylor, A. , Vassar, A. , Renzella, J. \ Pearce, H. APACrefauthors \ 2024 . dcc --help: Transforming the Role of the Compiler by Generating Context-Aware Error Explanations with Large Language Models dcc --help: Transforming th...

    2024

  37. [53]

    APACrefauthors \ 2023

    wermelinger2023using APACrefauthors Wermelinger, M. APACrefauthors \ 2023 . Using GitHub Copilot to Solve Simple Programming Problems Using github copilot to solve simple programming problems . Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 ...

    2023

  38. [54]

    , Clear, T

    whalley2007many APACrefauthors Whalley, J. , Clear, T. \ Lister, R. APACrefauthors \ 2007 . The many ways of the Bracelet project The many ways of the Bracelet project . BACIT

    2007

  39. [55]

    , Hou, X

    xiao2024exploring APACrefauthors Xiao, R. , Hou, X. \ Stamper, J. APACrefauthors \ 2024 . Exploring How Multiple Levels of GPT-Generated Programming Hints Support or Disappoint Novices Exploring how multiple levels of gpt-generated programming hints support or disappoint novic...

    2024

  40. [56]

    , Tran, K

    yeh2025bridging APACrefauthors Yeh, T Y. , Tran, K. , Gao, G. , Yu, T. , Fong, W O. \ Chen, T Y. APACrefauthors \ 2025 . Bridging Novice Programmers and LLMs with Interactivity Bridging novice programmers and llms with interactivity . Proceedings of the 56th ACM Technical Symp...

    2025

  41. [57]

    APACrefauthors \ 2022

    zhai2022chatgpt APACrefauthors Zhai, X. APACrefauthors \ 2022 . ChatGPT User Experience: Implications for Education Chatgpt user experience: Implications for education . APACrefDOI doi:http://dx.doi.org/10.2139/ssrn.4312418 APACrefDOI

    2022 doi

  42. [58]

    , Cambronero, J

    zhang2022repairing APACrefauthors Zhang, J. , Cambronero, J. , Gulwani, S. , Le, V. , Piskac, R. , Soares, G. \ Verbruggen, G. APACrefauthors \ 2022 . Repairing Bugs in Python Assignments Using Large Language Models Repairing bugs in python assignments using large language mod...

    2022 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.