REVIEW 3 cited by
Large Language Models in Introductory Programming Education: ChatGPT's Performance and Implications for Assessments
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Large Language Models in Introductory Programming Education: ChatGPT's Performance and Implications for Assessments
read the original abstract
This paper investigates the performance of the Large Language Models (LLMs) ChatGPT-3.5 and GPT-4 in solving introductory programming tasks. Based on the performance, implications for didactic scenarios and assessment formats utilizing LLMs are derived. For the analysis, 72 Python tasks for novice programmers were selected from the free site CodingBat. Full task descriptions were used as input to the LLMs, while the generated replies were evaluated using CodingBat's unit tests. In addition, the general availability of textual explanations and program code was analyzed. The results show high scores of 94.4 to 95.8% correct responses and reliable availability of textual explanations and program code, which opens new ways to incorporate LLMs into programming education and assessment.
Forward citations
Cited by 3 Pith papers
-
The Prompt Report: A Systematic Survey of Prompt Engineering Techniques
This systematic survey organizes prompt engineering into a taxonomy of 58 LLM techniques and 40 others, supplies a shared vocabulary, and offers guidelines for state-of-the-art models.
-
When LLM Tutoring Responses Work: Evidence from Student Programming Conversations
LLM response styles show small but significant associations with productive student continuation in programming dialogues, with larger differences under high cognitive load and debugging.
-
Investigating Conversational Agents to Support Secondary School Students Learning CSP
A classroom evaluation with 45 high school students finds that conversational agents can aid CSP learning by delivering context-appropriate information, comparing general and custom agent approaches for effectiveness ...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.