pith. sign in

arxiv: 2604.16213 · v1 · submitted 2026-04-17 · 💻 cs.HC · cs.SE

Investigating Conversational Agents to Support Secondary School Students Learning CSP

Matthew Frazier , Kostadin Damevski , Lori Pollock This is my paper

Pith reviewed 2026-05-10 07:35 UTC · model grok-4.3

classification 💻 cs.HC cs.SE
keywords conversational agentsAP Computer Science Principlessecondary educationexploratory searchChatGPTstudent engagementlearning supportCSP
0
0 comments X

The pith

Conversational agents like ChatGPT and custom CSP versions help secondary students overcome obstacles in finding appropriate learning resources for AP Computer Science Principles.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper investigates conversational agents as an alternative to web searches for secondary school students in AP Computer Science Principles, where the core difficulty is locating information suited to specific tasks and individual backgrounds. It deploys both general-purpose generative agents and custom fixed-response agents in real classrooms with 45 students across six sections in grades 9-11. Results focus on measuring effectiveness and engagement during exploratory search. A sympathetic reader would care because the approach directly targets a practical barrier in self-directed computer science learning that affects many high schoolers relying on external materials.

Core claim

Conversational agents, including general ones such as ChatGPT and custom agents built for CSP, serve as viable tools that address the primary obstacle of finding information appropriate for the learning task and student's background, with classroom results from 45 high school students demonstrating their potential to support knowledge acquisition through improved effectiveness and engagement in exploratory search.

What carries the argument

The comparison and classroom deployment of general-purpose generative conversational agents versus custom fixed-response agents tailored to CSP curriculum, functioning as alternatives to traditional web resources for information retrieval.

If this is right

  • Students can more readily access CSP information matched to their current tasks and prior knowledge.
  • Both general and custom agent designs can be evaluated for distinct contributions to engagement during exploratory search.
  • Classroom self-reports provide a practical basis for assessing agent effectiveness in secondary education settings.
  • Conversational agents represent a direct alternative that reduces the time spent on unsuitable web resources.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • School districts could integrate agent access into CSP courses to standardize support for resource discovery.
  • Longer-term tracking might reveal whether repeated agent use changes how students approach independent learning.
  • The same agent designs could be tested in other high school subjects that depend on external tutorials and Q&A sites.

Load-bearing premise

That the self-reported experiences of these 45 students in six specific CSP sections accurately reflect actual learning gains and will generalize to broader secondary school populations.

What would settle it

A controlled comparison in which students using either type of conversational agent show no improvement in locating suitable CSP resources or in subsequent concept understanding compared to students limited to standard web search.

Figures

Figures reproduced from arXiv: 2604.16213 by Kostadin Damevski, Lori Pollock, Matthew Frazier.

Figure 1
Figure 1. Figure 1: Conversational user interfaces for with Aida’s DialogFlow environment and ChatGPT’s HuggingFace environment. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Pedagogical DialogFlow Follow-Up Intent Design. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Pedagogical Conversation Example of DialogFlow Follow-Up Intents. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: AP College Board Computer Science Principles Curriculum Framework. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Distributions of Student Answer Completeness (left), Accuracy (middle), and Extraneousness (right). The results are aggregated [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Student Answer Example for Binary/Linear Search Task. [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Example of Student Answer containing inaccurate information for the Binary/Linear Search Task. [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Distribution of the Number of Presented Examples (left) and Computer Science Principles Exclusion Statement Occurrences [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Example student sessions: Aida conversation, ChatGPT conversation, and Google search history. [PITH_FULL_IMAGE:figures/full_fig_p019_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Results regarding measures of user engagement including the Number of Student Exploratory Actions (left) and Task Duration [PITH_FULL_IMAGE:figures/full_fig_p021_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Students’ Perception of Conversational Agent Interactions. All participants utilized Aida for one task, and either ChatGPT or [PITH_FULL_IMAGE:figures/full_fig_p022_11.png] view at source ↗
read the original abstract

Secondary school students enrolled in the AP Computer Science Principles (CSP) course commonly utilize web resources (e.g., tutorials, Q\&A sites) to better understand key concepts in the curriculum. The primary obstacle to using these resources is finding information appropriate for the learning task and student's background. In addition to web search, conversational agents are increasingly a viable alternative for CSP students. In this paper, we study the potential of conversational agents to aid secondary school students as they acquire knowledge on CSP concepts. We explore general purpose, generative conversational agents (e.g., ChatGPT) and custom, fixed-response conversational agents built specifically to aid CSP students. We present results from classroom use by 45 high school students in grades 9-11 (ages 14-17) across six CSP sections. Our main contributions are in better understanding how conversational agents can help CSP students and an evaluation of the effectiveness and engagement of different approaches for CSP exploratory search.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper investigates the use of conversational agents to support secondary school students in AP Computer Science Principles (CSP). It compares general-purpose generative agents (e.g., ChatGPT) with custom fixed-response agents built for CSP topics. The work reports on a classroom deployment involving 45 students in grades 9-11 across six CSP sections, collecting interaction data and self-reported perceptions of helpfulness, engagement, and agent preferences. The stated contributions are improved understanding of how such agents aid CSP students and an evaluation of their effectiveness and engagement for exploratory search tasks.

Significance. If the empirical observations hold under more rigorous validation, the work could provide useful HCI insights into AI-supported exploratory learning in K-12 CS curricula, particularly through real-classroom deployment. The comparison between general and custom agents is a reasonable framing. However, the significance for claims about helping students acquire knowledge is constrained by the absence of objective learning measures, limiting generalizability beyond the specific 45-student sample and self-reported perceptions.

major comments (2)
  1. [Abstract] Abstract and study description: The evaluation of effectiveness and engagement for CSP exploratory search rests entirely on post-use surveys of self-reported helpfulness, engagement, and preference from the 45 students. No pre/post knowledge assessments, task performance metrics, control conditions (e.g., web search only), or statistical analysis details are described. This is load-bearing for the central claim, as perceived utility cannot be distinguished from actual concept acquisition or retention.
  2. [Contributions] Contributions paragraph: The claim of 'better understanding how conversational agents can help CSP students' and 'evaluation of the effectiveness' is undermined by reliance on subjective data alone; the weakest assumption that self-reported measures accurately reflect learning gains is not addressed with any objective validation, weakening the contribution relative to the stated goals.
minor comments (1)
  1. [Abstract] The abstract could explicitly note the limitations of self-reported data and the narrow scope (six specific CSP sections) to avoid overgeneralization.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback highlighting the reliance on self-reported measures. We agree this constrains claims about knowledge acquisition and will revise the manuscript to clarify the scope of our contributions, explicitly describe the measures used, and add a dedicated limitations discussion. The study remains valuable as an authentic classroom exploration of student perceptions and interactions.

read point-by-point responses

  1. Referee: [Abstract] Abstract and study description: The evaluation of effectiveness and engagement for CSP exploratory search rests entirely on post-use surveys of self-reported helpfulness, engagement, and preference from the 45 students. No pre/post knowledge assessments, task performance metrics, control conditions (e.g., web search only), or statistical analysis details are described. This is load-bearing for the central claim, as perceived utility cannot be distinguished from actual concept acquisition or retention.

    Authors: We acknowledge that the evaluation relies solely on post-use self-reported surveys without pre/post knowledge assessments, task performance metrics, or control conditions. This design choice reflects the study's focus as an exploratory classroom deployment examining how students interact with and perceive conversational agents during real CSP exploratory search tasks, rather than a controlled experiment isolating learning gains. Self-reported measures are appropriate for assessing engagement and preference in an authentic setting where objective controls would be disruptive. We will revise the abstract to state explicitly that effectiveness and engagement are evaluated via self-reports, provide additional details on the survey analysis, and include a limitations section addressing the absence of objective validation and the distinction between perceived utility and measured learning outcomes. revision: yes

  2. Referee: [Contributions] Contributions paragraph: The claim of 'better understanding how conversational agents can help CSP students' and 'evaluation of the effectiveness' is undermined by reliance on subjective data alone; the weakest assumption that self-reported measures accurately reflect learning gains is not addressed with any objective validation, weakening the contribution relative to the stated goals.

    Authors: The stated contributions center on understanding support mechanisms through observed interactions and self-reported perceptions, plus evaluating effectiveness and engagement for exploratory search based on those data—not on demonstrating objective knowledge acquisition. We will revise the contributions paragraph to qualify these claims, noting that they derive from self-reported helpfulness, engagement, and preferences collected in a real classroom with 45 students. This framing is consistent with the paper's exploratory HCI focus on student experiences in K-12 CS, which provides actionable insights even without objective measures. A new limitations subsection will explicitly discuss the lack of validation against learning gains and suggest directions for future rigorous studies. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical user study with no derivations or self-referential fits

full rationale

The paper reports an empirical classroom study of 45 students using conversational agents for CSP learning, presenting observed interaction patterns, self-reported helpfulness/engagement scores, and preference comparisons between ChatGPT and custom agents. No equations, parameter fittings, predictions derived from fitted inputs, uniqueness theorems, or ansatzes appear in the provided text or abstract. Claims rest directly on collected survey and usage data rather than reducing to self-definitions or self-citations. The central contribution (understanding agent effectiveness for exploratory search) is therefore self-contained against external benchmarks of student responses and does not exhibit any of the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on standard assumptions from educational technology studies rather than new axioms or invented entities.

axioms (1)
  • domain assumption Student self-reports of effectiveness and engagement reflect meaningful differences in learning support provided by the agents
    Invoked implicitly when interpreting classroom use results as evidence of agent helpfulness.

pith-pipeline@v0.9.0 · 5458 in / 1131 out tokens · 36123 ms · 2026-05-10T07:35:06.433620+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

127 extracted references · 127 canonical work pages · 2 internal anchors

  1. [1]

    [n. d.]. AP Computer Science Principles COURSE AND EXAM DESCRIPTION Effective Fall 2020 INCLUDES Course framework Instructional section Sample exam questions Create performance task guidelines. https://apcentral.collegeboard.org/media/pdf/ap-computer-science-principles- course-and-exam-description.pdf

    work page 2020

  2. [2]

    [n. d.]. Code.org Curriculum Values. https://code.org/educate/curriculum/values

    work page

  3. [3]

    [n. d.]. National Science Foundation - Where Discoveries Begin. https://nsf.gov/events/event_summ.jsp?cntn_id=130646&org=NSF

    work page

  4. [4]

    [n. d.]. Professional Learning Values. https://code.org/educate/professional-learning/values

    work page

  5. [5]

    AP®Computer Science Principles Curricular Requirements CR1

    2023. AP®Computer Science Principles Curricular Requirements CR1. https://apcentral.collegeboard.org/media/pdf/ap-computer-science- principles-sample-syllabus-2.pdf

    work page 2023

  6. [6]

    Google . 2023. Bard. https://bard.google.com/ 28 Investigating Conversational Agents to Support Secondary School Students Learning CSP Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

    work page 2023

  7. [7]

    OpenAPI . 2023. OpenAI Platform. https://platform.openai.com/docs/guides/gpt/chat-completions-api

    work page 2023

  8. [8]

    Rabe Abdalkareem, Emad Shihab, and Juergen Rilling. 2017. What do developers use the crowd for? A study using stack overflow.IEEE Software 34, 2 (2017), 53–60

    work page 2017

  9. [9]

    Shruti Agrawal and Satheesh Abimannan. 2023. Conversational virtual educator using Google’s dialogflow.AIP Con- ference Proceedings2755, 1 (09 2023), 020017. https://doi.org/10.1063/5.0150047 arXiv:https://pubs.aip.org/aip/acp/article- pdf/doi/10.1063/5.0150047/18119897/020017_1_5.0150047.pdf

    work page doi:10.1063/5.0150047 2023

  10. [10]

    Erfan Al-Hossami, Razvan Bunescu, Ryan Teehan, Laurel Powell, Khyati Mahajan, and Mohsen Dorodchi. 2023. Socratic Questioning of Novice Debuggers: A Benchmark Dataset and Preliminary Evaluations. InProceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), Ekaterina Kochmar, Jill Burstein, Andrea Horbach, R...

    work page doi:10.18653/v1/2023.bea- 2023

  11. [11]

    Alfayez and Judy Lambert

    Abdulaziz A. Alfayez and Judy Lambert. 2019. Exploring Saudi Computer Science Teachers’ Conceptual Mastery Level of Com- putational Thinking Skills.Computers in the Schools36, 3 (2019), 143–166. https://doi.org/10.1080/07380569.2019.1639593 arXiv:https://doi.org/10.1080/07380569.2019.1639593

    work page doi:10.1080/07380569.2019.1639593 2019

  12. [12]

    Barbara Arfé, Tullio Vardanega, and Lucia Ronconi. 2020. The effects of coding on children’s planning and inhibition skills.Computers & Education 148 (2020), 103807. https://doi.org/10.1016/j.compedu.2020.103807

    work page doi:10.1016/j.compedu.2020.103807 2020

  13. [14]

    Kumaripaba Athukorala, Dorota Głowacka, Giulio Jacucci, Antti Oulasvirta, and Jilles Vreeken. 2016. Is exploratory search different? A comparison of information search behavior for exploratory and lookup tasks.Journal of the Association for Information Science and Technology67, 11 (2016), 2635–2651. https://doi.org/10.1002/asi.23617 arXiv:https://asistdl....

    work page doi:10.1002/asi.23617 2016

  14. [15]

    Anne Aula, Rehan Khan, and Zhiwei Guan. 2010. How does search behavior change as search becomes more difficult?. InProceedings of the ACM Conference on Human Factors in Computing Systems - CHI 2010. http://doi.acm.org/10.1145/1753326.1753333

    work page doi:10.1145/1753326.1753333 2010

  15. [16]

    Hannah McLean Babe, Sydney Nguyen, Yangtian Zi, Molly Q Feldman, Carolyn Jane Anderson, and Arjun Guha. 2024. StudentEval: A Benchmark of Student-Written Prompts for Large Language Models of Code. https://openreview.net/forum?id=o6XxdC4QsX

    work page 2024

  16. [17]

    David Baidoo-Anu and Leticia Owusu Ansah. 2023. Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning.A vailable at SSRN 4337484(2023)

    work page 2023

  17. [18]

    Rishabh Balse, Bharath Valaboju, Shreya Singhal, Jayakrishnan Madathil Warriem, and Prajish Prasad. 2023. Investigating the Potential of GPT-3 in Providing Feedback for Programming Assessments. InProceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1(Turku, Finland)(ITiCSE 2023). Association for Computing Machin...

    work page doi:10.1145/3587102.3588852 2023

  18. [19]

    Nikki Barrington, Nithin Gupta, Basel Musmar, David Doyle, Nicholas Panico, Nikhil Godbole, Taylor Reardon, and Randy D’amico. 2023. A Bibliometric Analysis of the Rise of ChatGPT in Medical Research.Medical Sciences11 (09 2023). https://doi.org/10.3390/medsci11030061

    work page doi:10.3390/medsci11030061 2023

  19. [20]

    Luciana Benotti, María Cecilia Martínez, and Fernando Schapachnik. 2014. Engaging High School Students Using Chatbots. InProceedings of the 2014 Conference on Innovation & Technology in Computer Science Education(Uppsala, Sweden)(ITiCSE ’14). Association for Computing Machinery, New York, NY, USA, 63–68. https://doi.org/10.1145/2591708.2591728

    work page doi:10.1145/2591708.2591728 2014

  20. [21]

    Alex Bernat. 2023. We Need to Fix Secondary School Computer Science.ACM Trans. Comput. Educ.23, 4, Article 38 (nov 2023), 6 pages. https://doi.org/10.1145/3617599

    work page doi:10.1145/3617599 2023

  21. [22]

    Virginia Braun and Victoria Clarke. 2021. Thematic Analysis. https://us.sagepub.com/en-us/nam/thematic-analysis/book248481

    work page 2021

  22. [23]

    2023.Exploring the Implications of OpenAI Codex on Education for Industry 4.0

    Robert Brennan and Jonathan Lesage. 2023.Exploring the Implications of OpenAI Codex on Education for Industry 4.0. 254–266. https://doi.org/10. 1007/978-3-031-24291-5_20

    work page 2023

  23. [24]

    Peter Brusilovsky. 2023. The Future of Computing Education Materials. https://csed.acm.org/wp-content/uploads/2023/03/Educational-Materials- First-Draft-1.pdf

    work page 2023

  24. [25]

    Christopher Bull and Ahmed Kharrufa. 2023. Generative AI Assistants in Software Development Education: A vision for integrating Generative AI into educational practice, not instinctively defending against it.IEEE Software(2023), 1–9. https://doi.org/10.1109/ms.2023.3300574

    work page doi:10.1109/ms.2023.3300574 2023

  25. [26]

    Liang Cai, Haoye Wang, Bowen Xu, Qiao Huang, Xin Xia, David Lo, and Zhenchang Xing. 2019. AnswerBot: An Answer Summary Generation Tool Based on Stack Overflow. InProceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering(Tallinn, Estonia)(ESEC/FSE 2019). Association f...

    work page doi:10.1145/3338906.3341186 2019

  26. [27]

    Listen to Music, Listen to Yourself

    Wanling Cai, Yucheng Jin, Xianglin Zhao, and Li Chen. 2023. “Listen to Music, Listen to Yourself”: Design of a Conversational Agent to Support Self-Awareness While Listening to Music. InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems(Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 1...

    work page doi:10.1145/3544548.3581427 2023

  27. [28]

    Nishi, Kostadin Damevski, Vinay Augustine, Lori Pollock, and Nicholas A

    Preetha Chatterjee, Manziba A. Nishi, Kostadin Damevski, Vinay Augustine, Lori Pollock, and Nicholas A. Kraft. 2017. What information about code snippets is available in different software-related documents? An exploratory study. In2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER). 382–386. https://doi.org/1...

    work page doi:10.1109/saner.2017.7884638 2017

  28. [29]

    Michelene T. H. Chi and Ruth Wylie. 2014. The ICAP Framework: Linking Cognitive Engagement to Active Learning Outcomes.Educational Psychologist49, 4 (2014), 219–243. https://doi.org/10.1080/00461520.2014.965823 arXiv:https://doi.org/10.1080/00461520.2014.965823

    work page doi:10.1080/00461520.2014.965823 2014

  29. [31]

    Bruno Pereira Cipriano and Pedro Alves. 2023. GPT-3 vs Object Oriented Programming Assignments: An Experience Report. InProceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1(Turku, Finland)(ITiCSE 2023). Association for Computing Machinery, New York, NY, USA, 61–67. https://doi.org/10.1145/3587102.3588814

    work page doi:10.1145/3587102.3588814 2023

  30. [32]

    Google Cloud. 2022. https://cloud.google.com/dialogflow

    work page 2022

  31. [33]

    Crook, Alex Marin, Vipul Agarwal, Samantha Anderson, Ohyoung Jang, Aliasgar Lanewala, Karthik Tangirala, and Imed Zitouni

    Paul A. Crook, Alex Marin, Vipul Agarwal, Samantha Anderson, Ohyoung Jang, Aliasgar Lanewala, Karthik Tangirala, and Imed Zitouni. 2018. Conversational Semantic Search: Looking Beyond Web Search, Q&A and Dialog Systems. InProceedings of the Eleventh ACM International Conference on Web Search and Data Mining(Marina Del Rey, CA, USA)(WSDM ’18). Association ...

    work page doi:10.1145/3159652.3160590 2018

  32. [34]

    Marian Daun and Jennifer Brings. 2023. How ChatGPT Will Change Software Engineering Education. InProceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1(Turku, Finland)(ITiCSE 2023). Association for Computing Machinery, New York, NY, USA, 110–116. https://doi.org/10.1145/3587102.3588815

    work page doi:10.1145/3587102.3588815 2023

  33. [35]

    Paul Denny, Viraj Kumar, and Nasser Giacaman. 2023. Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language. InProceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1(Toronto, ON, Canada)(SIGCSE 2023). Association for Computing Machinery, New York, NY, USA, 1136–1142. https://doi.org...

    work page doi:10.1145/3545945.3569823 2023

  34. [36]

    Becker, James Finnie-Ansley, Arto Hellas, Juho Leinonen, Andrew Luxton-Reilly, Brent N

    Paul Denny, James Prather, Brett A. Becker, James Finnie-Ansley, Arto Hellas, Juho Leinonen, Andrew Luxton-Reilly, Brent N. Reeves, Eddie Antonio Santos, and Sami Sarsa. 2024. Computing Education in the Era of Generative AI.Commun. ACM67, 2 (jan 2024), 56–67. https://doi.org/10.1145/ 3624720

    work page 2024

  35. [37]

    Richard Devine. 2021. Reflexive Thematic Analysis – The What, Why and How. https://richarddevinesocialwork.com/2021/11/12/reflexive- thematic-analysis-the-what-why-and-how/

    work page 2021

  36. [38]

    Dhotre, Niraja Jain, Chhaya Mhaske, Nitin Choubey, and Dinesh D

    Dhanraj R. Dhotre, Niraja Jain, Chhaya Mhaske, Nitin Choubey, and Dinesh D. Patil. 2023. ADiTi App: Leveraging Deep Learning and Generative AI for a Chatbot Application with Deep Belief Networks.International Journal of Intelligent Systems and Applications in Engineering12, 1s (Sep. 2023), 569–576. https://www.ijisae.org/index.php/IJISAE/article/view/3492

    work page 2023

  37. [39]

    Dr. J. T. Dillon. 1981. A Norm against Student Questions.The Clearing House: A Journal of Educational Strategies, Issues and Ideas55, 3 (1981), 136–139. https://doi.org/10.1080/00098655.1981.10113676 arXiv:https://doi.org/10.1080/00098655.1981.10113676

    work page doi:10.1080/00098655.1981.10113676 1981

  38. [40]

    Felix Dobslaw and Peter Bergh. 2023. Experiences with Remote Examination Formats in Light of GPT-4. InProceedings of the 5th European Conference on Software Engineering Education(Seeon/Bavaria, Germany)(ECSEE ’23). Association for Computing Machinery, New York, NY, USA, 220–225. https://doi.org/10.1145/3593663.3593695

    work page doi:10.1145/3593663.3593695 2023

  39. [41]

    Hugging Face. 2023. Hugging Face – On a mission to solve NLP, one commit at a time. https://huggingface.co/

    work page 2023

  40. [42]

    Victor Farinango and Santiago Castro Arias. 2023. Pedagogical Support Through the Implementation of a Virtual Assistant to Personalize Education. InProceedings of the 18th Latin American Conference on Learning Technologies (LACLO 2023), Santiago Berrezueta (Ed.). Springer Nature Singapore, Singapore, 343–356

    work page 2023

  41. [43]

    Donghui Feng, Erin Shaw, Jihie Kim, and Eduard Hovy. 2006. An Intelligent Discussion-Bot for Answering Student Queries in Threaded Discussions. InProceedings of the 11th International Conference on Intelligent User Interfaces(Sydney, Australia)(IUI ’06). Association for Computing Machinery, New York, NY, USA, 171–177. https://doi.org/10.1145/1111449.1111488

    work page doi:10.1145/1111449.1111488 2006

  42. [44]

    James Finnie-Ansley, Paul Denny, Andrew Luxton-Reilly, Eddie Antonio Santos, James Prather, and Brett A. Becker. 2023. My AI Wants to Know If This Will Be on the Exam: Testing OpenAI’s Codex on CS2 Programming Exercises. InProceedings of the 25th Australasian Computing Education Conference(Melbourne, Australia)(ACE ’23). Association for Computing Machiner...

    work page doi:10.1145/3576123 2023

  43. [45]

    Maya Flax. 2023. Strategies for enhancing teaching excellence: Providing real-world examples and building rapport.New Vistas9, 1 (2023)

    work page 2023

  44. [46]

    Fiona French, David Levi, Csaba Maczo, Aiste Simonaityte, Stefanos Triantafyllidis, and Gergo Varda. 2023. Creative Use of OpenAI in Education: Case Studies from Game Development.Multimodal Technologies and Interaction7, 8 (2023). https://doi.org/10.3390/mti7080081

    work page doi:10.3390/mti7080081 2023

  45. [47]

    Yasemin Gülbahar and Filiz Kalelioğlu. 2017. Competencies of High School Teachers and Training Needs for Computer Science Education. In Proceedings of the 6th Computer Science Education Research Conference(Helsinki, Finland)(CSERC ’17). Association for Computing Machinery, New York, NY, USA, 26–31. https://doi.org/10.1145/3162087.3162092

    work page doi:10.1145/3162087.3162092 2017

  46. [48]

    Jackson, and Anil Lalwani

    Karla Hamlen, Nigamanth Sridhar, Lisa Bievenue, Debbie K. Jackson, and Anil Lalwani. 2018. Effects of Teacher Training in a Computer Science Principles Curriculum on Teacher and Student Skills, Confidence, and Beliefs. InProceedings of the 49th ACM Technical Symposium on Computer Science Education(Baltimore, Maryland, USA)(SIGCSE ’18). Association for Com...

    work page doi:10.1145/3159450.3159496 2018

  47. [49]

    Jackson, Lisa Bievenue, Adam Voight, and Nigamanth Sridhar

    Karla Hamlen Mansour, Debbie K. Jackson, Lisa Bievenue, Adam Voight, and Nigamanth Sridhar. 2023. Understanding the Impact of Peer Instruction in CS Principles Teacher Professional Development.ACM Trans. Comput. Educ.23, 2, Article 24 (apr 2023), 21 pages. https://doi.org/10.1145/3585077 30 Investigating Conversational Agents to Support Secondary School S...

    work page doi:10.1145/3585077 2023

  48. [50]

    Qiang Hao, Brad Barnes, Robert Branch, and Ewan Wright. 2017. Predicting Computer Science Students’ Online Help-Seeking Tendencies. Knowledge Management & E-Learning: An International Journal9 (03 2017), 19–32

    work page 2017

  49. [51]

    Harrington and Lisa Egede

    Christina N. Harrington and Lisa Egede. 2023. Trust, Comfort and Relatability: Understanding Black Older Adults’ Perceptions of Chatbot Design for Health Information Seeking. InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems(Hamburg, Germany)(CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 120, 18 pag...

    work page doi:10.1145/3544548.3580719 2023

  50. [52]

    Arto Hellas, Juho Leinonen, Sami Sarsa, Charles Koutcheme, Lilja Kujanpää, and Juha Sorva. 2023. Exploring the Responses of Large Language Models to Beginner Programmers’ Help Requests. InProceedings of the 2023 ACM Conference on International Computing Education Research V.1 (ICER 2023). ACM. https://doi.org/10.1145/3568813.3600139

    work page doi:10.1145/3568813.3600139 2023

  51. [53]

    Irene Hou, Sophia Mettille, Owen Man, Zhuo Li, Cynthia Zastudil, and Stephen MacNeil. 2024. The Effects of Generative AI on Computing Students’ Help-Seeking Preferences. InProceedings of the 26th Australasian Computing Education Conference(Sydney, NSW, Australia)(ACE ’24). Association for Computing Machinery, New York, NY, USA, 39–48. https://doi.org/10.1...

    work page doi:10.1145/3636243.3636248 2024

  52. [54]

    https://rasa.com/. [n. d.]. Rasa: Open source conversational AI. visited, December 1, 2020

    work page 2020

  53. [55]

    Joy Idialu and Deborah Etsenake. [n. d.]. Whodunnit: Human or AI? https://plg.uwaterloo.ca/~migod/846/current/projects/07-Norhan-Deborah- Joy-report.pdf

    work page

  54. [56]

    Cruz Izu and Claudio Mirolo. 2020. Comparing Small Programs for Equivalence: A Code Comprehension Task for Novice Programmers. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education(Trondheim, Norway)(ITiCSE ’20). Association for Computing Machinery, New York, NY, USA, 466–472. https://doi.org/10.1145/3341525.3387425

    work page doi:10.1145/3341525.3387425 2020

  55. [57]

    LaToza, Kevin Moran, and Wing Lam

    Sajed Jalil, Suzzana Rafi, Thomas D. LaToza, Kevin Moran, and Wing Lam. 2023. ChatGPT and Software Testing Education: Promises & Perils. In2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE. https://doi.org/10.1109/icstw58534.2023.00078

    work page doi:10.1109/icstw58534.2023.00078 2023

  56. [58]

    Mahmoud Kaddoura. 2013. Think pair share: A teaching learning strategy to enhance students’ critical thinking.Educational Research Quarterly36, 4 (2013), 3–24

    work page 2013

  57. [59]

    Tyson Kendon, Leanne Wu, and John Aycock. 2023. AI-Generated Code Not Considered Harmful. InProceedings of the 25th Western Canadian Conference on Computing Education(Vancouver, BC, Canada)(WCCCE ’23). Association for Computing Machinery, New York, NY, USA, Article 3, 7 pages. https://doi.org/10.1145/3593342.3593349

    work page doi:10.1145/3593342.3593349 2023

  58. [60]

    Johannes Kiesel, Arefeh Bahrami, Benno Stein, Avishek Anand, and Matthias Hagen. 2018. Toward Voice Query Clarification. InThe 41st International ACM SIGIR Conference on Research and Development in Information Retrieval(Ann Arbor, MI, USA)(SIGIR ’18). 1257–1260

    work page 2018

  59. [61]

    Natalie Kiesler and Daniel Schiffner. 2023. Large Language Models in Introductory Programming Education: ChatGPT’s Performance and Implications for Assessments.ArXivabs/2308.08572 (2023). https://api.semanticscholar.org/CorpusID:261030787

    work page arXiv 2023

  60. [62]

    Adrian Kirwan. 2023. ChatGPT and university teaching, learning and assessment: some initial reflections on teaching academic in- tegrity in the age of Large Language Models.Irish Educational Studies0, 0 (2023), 1–18. https://doi.org/10.1080/03323315.2023.2284901 arXiv:https://doi.org/10.1080/03323315.2023.2284901

    work page doi:10.1080/03323315.2023.2284901 2023

  61. [63]

    Emily Kuang, Ehsan Jahangirzadeh Soure, Mingming Fan, Jian Zhao, and Kristen Shinohara. 2023. Collaboration with Conversational AI Assistants for UX Evaluation: Questions and How to Ask Them (Voice vs. Text). InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems(Hamburg, Germany)(CHI ’23). Association for Computing Machinery, New ...

    work page doi:10.1145/3544548.3581247 2023

  62. [64]

    Els Kuiper, Monique Volman, and Jan Terwel. 2005. The Web as an information resource in K–12 education: Strategies for supporting students in searching and processing information.Review of educational research75, 3 (2005), 285–328

    work page 2005

  63. [65]

    Els Kuiper, Monique Volman, and Jan Terwel. 2009. Developing Web literacy in collaborative inquiry activities.Computers & Education52, 3 (2009), 668–680. https://doi.org/10.1016/j.compedu.2008.11.010

    work page doi:10.1016/j.compedu.2008.11.010 2009

  64. [66]

    Ban It Till We Understand It

    Sam Lau and Philip Guo. 2023. From "Ban It Till We Understand It" to "Resistance is Futile": How University Programming Instructors Plan to Adapt as More Students Use AI Code Generation and Explanation Tools Such as ChatGPT and GitHub Copilot. InProceedings of the 2023 ACM Conference on International Computing Education Research - Volume 1(Chicago, IL, US...

    work page doi:10.1145/3568813.3600138 2023

  65. [67]

    Juho Leinonen, Paul Denny, Stephen MacNeil, Sami Sarsa, Seth Bernstein, Joanne Kim, Andrew Tran, and Arto Hellas. 2023. Comparing Code Explanations Created by Students and Large Language Models. InProceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1(Turku, Finland)(ITiCSE 2023). Association for Computing Machi...

    work page doi:10.1145/3587102.3588785 2023

  66. [68]

    Burrows, and Richard Kitchen

    Jacqueline Leonard, Andrea C. Burrows, and Richard Kitchen. 2019.Recruiting, Preparing, and Retaining STEM Teachers for a Global Generation. Brill, Leiden, The Netherlands. https://doi.org/10.1163/9789004399990

    work page doi:10.1163/9789004399990 2019

  67. [69]

    Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A Diversity-Promoting Objective Function for Neural Conversation Models. InProceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Kevin Knight, Ani Nenkova, and Owen Rambow (Eds.). Associatio...

    work page doi:10.18653/v1/n16-1014 2016

  68. [70]

    Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando De Freitas, Koray Kavukcuoglu, and Oriol Vinyals

    Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d’Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Pushm...

    work page doi:10.1126/science.abq1158 2018

  69. [71]

    Chao Liu, Xin Xia, David Lo, Cuiyun Gao, Xiaohu Yang, and John Grundy. 2020. Opportunities and Challenges in Code Search Tools.arXiv preprint arXiv:2011.02297(2020)

    work page arXiv 2020

  70. [72]

    Qiaoling Liu, Eugene Agichtein, Gideon Dror, Yoelle Maarek, and Idan Szpektor. 2012. When Web Search Fails, Searchers Become Askers: Understanding the Transition. InProceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval (Portland, Oregon, USA)(SIGIR ’12). Association for Computing Machinery, New Yor...

    work page doi:10.1145/2348283.2348390 2012

  71. [73]

    Qianou Ma, Tongshuang Wu, and Kenneth Koedinger. 2023. Is AI the better programming partner? Human-Human pair programming vs. Human-AI pAIr programming.arXiv preprint arXiv:2306.05153(2023)

    work page arXiv 2023

  72. [74]

    Hanifah Putri Elisa Mahir Pradana and Syarifuddin Syarifuddin. 2023. Discussing ChatGPT in education: A literature review and bibliometric analysis. Cogent Education10, 2 (2023), 2243134. https://doi.org/10.1080/2331186X.2023.2243134 arXiv:https://doi.org/10.1080/2331186X.2023.2243134

    work page doi:10.1080/2331186x.2023.2243134 2023

  73. [75]

    Kamil Malinka, Martin Peresíni, Anton Firc, Ondrej Hujnák, and Filip Janus. 2023. On the Educational Impact of ChatGPT: Is Artificial Intelligence Ready to Obtain a University Degree?. InProceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 (ITiCSE 2023). ACM. https://doi.org/10.1145/3587102.3588827

    work page doi:10.1145/3587102.3588827 2023

  74. [76]

    Gary Marchionini. 2006. Exploratory search: from finding to understanding.Commun. ACM49, 4 (2006), 41–46

    work page 2006

  75. [77]

    Wilfred B. W. Martin. 1987. Students’ Perceptions of Causes and Consequences of Embarrassment in the School.Canadian Journal of Education / Revue canadienne de l’éducation12, 2 (1987), 277–293. http://www.jstor.org/stable/1494907

    work page arXiv 1987

  76. [78]

    McPherson and Patricia Kearney

    Mary B. McPherson and Patricia Kearney. 1992. Classroom Embarrassment: Types, Goals, and Face Saving Strategies

    work page 1992

  77. [79]

    Microsoft. 2022. https://www.luis.ai/

    work page 2022

  78. [80]

    Daye Nam, Andrew Macvean, Vincent Hellendoorn, Bogdan Vasilescu, and Brad Myers. 2023. In-IDE Generation-based Information Support with a Large Language Model. arXiv:2307.08177 [cs.SE]

    work page arXiv 2023

  79. [81]

    OpenAI. 2023. ChatGPT. https://openai.com/blog/chatgpt

    work page 2023

  80. [82]

    OpenAI. 2023. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL]

    work page internal anchor Pith review Pith/arXiv arXiv 2023

Showing first 80 references.