REVIEW 2 major objections 5 minor 40 references
Early Adoption of Generative Artificial Intelligence in Computing Education: Emergent Student Use Cases and Perspectives in 2023
T0 review · 2 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read By the end of Spring 2023, most computer science students at a U.S.
desk verdict An honest early snapshot of GenAI use in one CS department, but the abstract overstates the evidence from a 12% opt-in sample. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central machinery is a survey instrument followed by directed content analysis, a qualitative method that applies a codebook to free-text responses. The survey asked students to rate their frequency of use of three GenAI categories (LLM chatbots, code generators, image generators), rate on a 1–10 scale how beneficial GenAI will be to computer science, and answer three free-response questions about their use, the appropriate role in education, and workplace concerns. Six human coders iteratively refined codebooks until inter-rater reliability (Krippendorff's alpha) exceeded 0.6, producing a taxonomy of emergent use cases—writing, coding, and learning—and of student opinions on degree of use, rationale, and implementation methods. That taxonomy is what lets the paper move from anecdote to a structured description of how students adopted the tools.
What would settle it
A comparison of self-reported GenAI use against actual usage telemetry (e.g., IDE plugin logs or browser history from a consenting subsample) would settle whether frequency estimates are inflated; if reported 'regular' use substantially exceeds logged use, the adoption baseline is overstated. Alternatively, replicating the survey at several institutions with 40%+ response rates and finding adoption below 30% would undermine the claim that most computing students had adopted GenAI by spring 2023.
Extended reading notes
Core claim
Surveying all computer science majors at a small engineering-focused R1 university (116 undergraduates and 17 graduate students, 12% and 7.6% of their respective populations), the paper finds that most respondents had tried a large language model chatbot, that fewer had tried code generators, and that almost none had fully turned in AI-generated assignments. Instead, students described using GenAI to draft and debug code, explain concepts, find sources, outline and polish writing, and act as an informal tutor when instructors were unavailable. Asked about the role of GenAI in education, 69 of 126 respondents called for conditional use with instructor-specified boundaries, 41 wanted it encouraged, and 16 wanted it discouraged; students were nearly split on whether it helps or harms learning (44 vs. 40 coded responses), while many viewed AI skills as necessary for future employment. The authors claim these results capture a genuine pre-policy snapshot of early adoption and student attitudes that can inform curricula, policy, and tooling.
Load-bearing premise
The findings rest on students accurately reporting how often they used tools that could look like cheating, from a department where only 12% of undergraduates and 7.6% of graduate students responded.
Editorial extensions
If this is right
- If most students are already using LLMs before policy exists, then policies written in 2023–2024 were reacting to established behavior rather than preventing it.
- Because students mostly used GenAI for supported tasks (explaining, debugging, outlining, tutoring) rather than full assignment completion, instructors can target restrictions at specific use cases such as 'no drafting code' without banning the tool outright.
- The split between students who want conditional use and those who want encouragement suggests that a one-size-fits-all ban would conflict with many students' expressed preferences and career expectations.
- Students' perception that AI literacy is needed for future jobs implies curricula should include professional AI use, not just academic integrity rules.
- The finding that students who use LLMs more frequently rate GenAI as more beneficial (p < .0008) suggests familiarity goes with acceptance, so early exposure may shape later attitudes.
Reading between the lines
- An implication the paper leaves implicit: if early adoption was already this common in spring 2023, later cohorts who have never known a pre-ChatGPT classroom will likely arrive with even higher baseline familiarity, making 'teach responsible use' the more realistic institutional stance.
- The paper's taxonomy suggests a testable design principle: restrictions keyed to course level (e.g., blocking code drafting in CS1 but allowing it in capstone courses) may preserve learning outcomes better than bans on the tool type itself, a hypothesis an experimental study could check.
- Interpreting students' 'calculator for coding' framing, assessment may need to shift from judging the produced code to judging the process of verification and design, for instance through oral exams or submitted explanations of AI-assisted changes.
- Replicating this survey at a large public university or a teaching-focused college would show whether the 12% undergraduate response rate at a single engineering-focused institution yields representative adoption estimates.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. Smith et al. report a Spring 2023 survey of computer science majors at a single small R1 U.S. university on their use and perceptions of generative AI tools. The study draws on 133 responses, presents frequencies of LLM, code-generator, and image-generator use, applies directed content analysis to open-text responses about writing, coding, and learning, and reports associations between usage frequency and benefit ratings. The central claims are that most students had tried GenAI tools for a variety of writing, coding, and learning use cases, and that students tend to view GenAI as beneficial to computing.
Significance. If limited to the respondent sample, this paper offers a timely and useful qualitative baseline of early GenAI adoption in computing education, with concrete student quotes and a clear coding methodology. Notable strengths are the iterative inter-rater reliability process, the public sharing of codebooks, and the honest acknowledgment of the non-representative sample. The paper's main value is as an exploratory snapshot that can inform later policy-oriented and larger-scale studies, but the population-level claims in the abstract currently outrun the evidence presented.
major comments (2)
- [Abstract; Section 3.4] The abstract claims that 'most students have tried GenAI tools' and that 'students tend to view GenAI tools as beneficial,' but Section 3.4 reports that the sample comprises 12% of undergraduates and 7.6% of graduate students at one institution and explicitly acknowledges the sample is not representative. Because the survey topic is GenAI itself, non-response is likely correlated with the outcome: students who have used GenAI or formed opinions may be more likely to respond. Without a non-response analysis (for example, a late-responder comparison, weighting, or benchmarking against institutional data), the headline claims cannot be distinguished from 'most survey respondents.' Please either narrow the abstract and conclusion claims to 'respondents' or supply such an analysis.
- [Section 4.1; Abstract] The 'variety of writing, coding, and learning use cases' in the abstract is grounded in the 75 of 133 respondents (56.4%) who answered the optional free-response question about their use of GenAI. The remaining 43.6% of the sample did not provide this information, and the abstract does not disclose this. Furthermore, these 75 respondents are self-selected, so their use-case descriptions may overrepresent students with more extensive or more opinionated GenAI experience. Please add this qualification to the abstract and RQ1 summary, and consider reporting a sensitivity count that treats non-responders to the optional question as missing.
minor comments (5)
- [Header; ACM Reference Format] The ACM Reference Format line lists the year as '2014' instead of '2024'.
- [Section 4.1] The sentence 'only 36.1% have ever reported trying an image generator' is not supported by the disaggregated counts in the text; please report the image-generator frequency counts shown in Table 1 to make the calculation auditable.
- [Section 3.3] The codebook link uses a URL shortener (bit.ly/SIGCSE-GenAI-codebooks); please provide a persistent repository link or DOI to protect against link rot.
- [Section 3.2] The full survey instrument is not included; adding it as an appendix would clarify the exact wording of the free-response questions and the frequency-response options, which are central to interpreting the reported percentages.
- [Section 4.2; Table 1] The claim that students 'tend to view GenAI as beneficial' is based on means of 6.78 and 7.41 on a 10-point scale with standard deviations around 2.6; please describe the distribution in prose as well, since a wide or bimodal distribution would weaken the interpretation of the mean.
Circularity Check
No circularity: the paper is an empirical survey whose qualitative codes summarize the same data they describe, and it makes no fitted prediction or derivation that reduces to its inputs.
full rationale
This paper is an exploratory survey study, not a derivation or prediction pipeline. It reports frequency-of-use counts, benefit ratings, and inductively developed qualitative codebooks from free-response answers. No equation, fitted parameter, or uniqueness theorem is invoked, and no result is claimed to follow from first principles. The only sense in which the codebooks 'describe' the data is the standard relationship of qualitative content analysis to its corpus: codes are derived from responses and then applied back to those responses. That is a descriptive summarization procedure, not a circular prediction, and the paper explicitly frames the quantitative distributions as potentially non-representative (Section 3.4: 'We acknowledge the limitation that this sample is not representative. Therefore we position our results primarily as qualitative and exploratory'). The paper also discloses the main self-report limitation (Section 3.1: 'students may have withheld or misrepresented information about behaviors perceived as cheating') and the low response rates (12% of undergraduates, 7.6% of graduates). These are validity concerns about sampling and self-report, not circularity. There are no self-citations in the reference list that carry a load-bearing premise, and no uniqueness claim is imported from the authors' prior work. The central claims are empirical generalizations from survey responses, which are exactly the paper's input data; reporting them as findings is the normal function of a survey, not a circular reduction. Score 0.
Assumptions & free parameters
assumptions (3)
- domain assumption Survey respondents answered truthfully and did not systematically conceal GenAI use.
- domain assumption A non-representative sample can still support qualitative description of emergent use cases.
- domain assumption Directed content analysis with Krippendorff's alpha above 0.6 yields meaningful themes.
Cite this review
Pith. "Pith review of Early Adoption of Generative Artificial Intelligence in Computing Education: Emergent Student Use Cases and Perspectives in 2023." pith.science (2026). https://pith.science/paper/26JXFE6S
@misc{pith2026241111166,
author = {Pith},
title = {Pith review of: Early Adoption of Generative Artificial Intelligence in Computing Education: Emergent Student Use Cases and Perspectives in 2023},
year = {2026},
howpublished = {\url{https://pith.science/paper/26JXFE6S}},
note = {Machine review of arXiv:2411.11166}
}
read the original abstract
Because of the rapid development and increasing public availability of Generative Artificial Intelligence (GenAI) models and tools, educational institutions and educators must immediately reckon with the impact of students using GenAI. There is limited prior research on computing students' use and perceptions of GenAI. In anticipation of future advances and evolutions of GenAI, we capture a snapshot of student attitudes towards and uses of yet emerging GenAI, in a period of time before university policies had reacted to these technologies. We surveyed all computer science majors in a small engineering-focused R1 university in order to: (1) capture a baseline assessment of how GenAI has been immediately adopted by aspiring computer scientists; (2) describe computing students' GenAI-related needs and concerns for their education and careers; and (3) discuss GenAI influences on CS pedagogy, curriculum, culture, and policy. We present an exploratory qualitative analysis of this data and discuss the impact of our findings on the emerging conversation around GenAI and education.
Figures
Reference graph
Works this paper leans on
-
[1]
John R. Anderson, C. Franklin Boyle, and Brian J. Reiser. 1985. Intelligent Tutoring Systems. Science 228, 4698 (April 1985), 456–462. https://doi.org/10.1126/science. Student Perspectives on Generative Artificial Intelligence in Computing Education ITiCSE 2024, Milan, Italy, 228.4698.456
doi:10.1126/science 1985
-
[2]
John R. Anderson, Albert T. Corbett, Kenneth R. Koedinger, and Ray. Pelletier
-
[3]
Paul Bruno, Mariam Saffar Perez, and Colleen Lewis. 2022. Four Practical Chal- lenges for High School Computer Science | Policy Analysis for California Educa- tion. https://edpolicyinca.org/publications/4-practical-challenges-high-school- computer-science
work page 2022
-
[4]
Miguel A Cardona, Roberto J Rodríguez, and Kristina Ishmael. 2023. Artificial Intelligence and the Future of Teaching and Learning . Technical Report. U.S. Department of Education, Office of Educational Technology, Washington, DC. 71 pages. https://www2.ed.gov/documents/ai-report/ai-report.pdf
work page 2023
-
[5]
Lorena Casal-Otero, Alejandro Catala, Carmen Fernández-Morante, Maria Taboada, Beatriz Cebreiro, and Senén Barro. 2023. AI literacy in K-12: a system- atic literature review. International Journal of STEM Education 10, 1 (April 2023),
work page 2023
-
[6]
Jack Caulfield. 2023. University Policies on AI Writing Tools | Overview & List. https://www.scribbr.com/ai-tools/chatgpt-university-policies/
work page 2023
-
[7]
Ismail Celik. 2023. Towards Intelligent-TPACK: An empirical study on teachers’ professional knowledge to ethically integrate artificial intelligence (AI)-based tools into education. Computers in Human Behavior 138 (Jan. 2023), 107468. https://doi.org/10.1016/j.chb.2022.107468
arXiv 2023
-
[8]
Cecilia Ka Yuk Chan. 2023. A comprehensive AI policy education framework for university teaching and learning. International Journal of Educational Technology in Higher Education 20, 1 (July 2023), 38. https://doi.org/10.1186/s41239-023- 00408-3
Show all 40 references
-
[9]
Jie Chao, William Finzer, Carolyn Rose, Shiyan Jiang, Rebecca Ellis, Kenia Wiede- mann, Cansu Tatar, and James Fiacco. 2023. Teach Artificial Intelligence with StoryQ, A Web-Based Machine Learning and Text Mining Tool for K-12 Students. In Proceedings of the 54th ACM Technical...
2023
-
[10]
Conner and Thomas M
Thaddieus W. Conner and Thomas M. Rabovsky. 2011. Accountability, Affordabil- ity, Access: A Review of the Recent Trends in Higher Education Policy Research. Policy Studies Journal 39, s1 (2011), 93–112. https://doi.org/10.1111/j.1541-0072. 2010.00389_7.x _eprint: https://onli...
2011 arXiv
-
[11]
Grant Cooper. 2023. Examining Science Education in ChatGPT: An Exploratory Study of Generative Artificial Intelligence. Journal of Science Education and Technology 32, 3 (June 2023), 444–452. https://doi.org/10.1007/s10956-023-10039- y
2023 doi
-
[12]
Becker, James Finnie-Ansley, Arto Hellas, Juho Leinonen, Andrew Luxton-Reilly, Brent N
Paul Denny, James Prather, Brett A. Becker, James Finnie-Ansley, Arto Hellas, Juho Leinonen, Andrew Luxton-Reilly, Brent N. Reeves, Eddie Antonio Santos, and Sami Sarsa. 2023. Computing Education in the Era of Generative AI. http: //arxiv.org/abs/2306.02608 arXiv:2306.02608 [cs]
2023 arXiv
-
[13]
Donn Emmanuel Gonda and Beatrice Chu. 2019. Chatbot as a learning resource? Creating conversational bots as a supplement for teaching assistant training course. In 2019 IEEE International Conference on Engineering, Technology and Education (TALE). 1–5. https://doi.org/10.1109/...
2019
-
[14]
Graesser, Xiangen Hu, Benjamin D
Arthur C. Graesser, Xiangen Hu, Benjamin D. Nye, Kurt VanLehn, Rohit Kumar, Cristina Heffernan, Neil Heffernan, Beverly Woolf, Andrew M. Olney, Vasile Rus, Frank Andrasik, Philip Pavlik, Zhiqiang Cai, Jon Wetzel, Brent Morgan, Andrew J. Hampton, Anne M. Lippert, Lijia Wang, Qi...
2018
-
[15]
Philip J. Guo. 2013. Online python tutor: embeddable web-based program visual- ization for cs education. In Proceeding of the 44th ACM technical symposium on Computer science education (SIGCSE ’13) . Association for Computing Machinery, New York, NY, USA, 579–584. https://doi....
2013
-
[16]
Hsiu-Fang Hsieh and Sarah E. Shannon. 2005. Three approaches to qualitative content analysis. Qualitative Health Research 15, 9 (Nov. 2005), 1277–1288. https: //doi.org/10.1177/1049732305276687
2005 doi
-
[17]
Sal Khan. 2023. Harnessing GPT-4 so that all students benefit. A nonprofit approach for equal access! https://blog.khanacademy.org/harnessing-ai-so-that- all-students-benefit-a-nonprofit-approach-for-equal-access/
2023
-
[18]
Celeste Kidd and Abeba Birhane. 2023. How AI can distort human beliefs. Sci- ence 380, 6651 (June 2023), 1222–1223. https://doi.org/10.1126/science.adi0248 Publisher: American Association for the Advancement of Science
2023 doi
-
[19]
Ban It Till We Understand It
Sam Lau and Philip J Guo. 2023. From "Ban It Till We Understand It" to "Resistance is Futile": How University Programming Instructors Plan to Adapt as More Students Use AI Code Generation and Explanation Tools such as ChatGPT and GitHub Copilot. In ICER. ICER, Chicago, IL
2023
-
[20]
Weng Marc Lim, Asanka Gunasekara, Jessica Leigh Pallant, Jason Ian Pallant, and Ekaterina Pechenkina. 2023. Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. The International Journal of Management Educatio...
2023
-
[21]
Elena Lyulina, Anastasiia Birillo, Vladimir Kovalenko, and Timofey Bryksin
-
[22]
Sathiamoorthy Manoharan and Ulrich Speidel. 2020. Contract Cheating in Computer Science: A Case Study. In 2020 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE) . 91–98. https: //doi.org/10.1109/TALE48869.2020.9368454 ISSN: 2470-6698
2020
-
[23]
Merrill, Brian J
Douglas C. Merrill, Brian J. Reiser, Michael Ranney, and J. Gregory Trafton. 1992. Effective Tutoring Techniques: A Comparison of Human Tutors and Intelligent Tutoring Systems. Journal of the Learning Sciences 2, 3 (July 1992), 277–305. https://doi.org/10.1207/s15327809jls0203_2
1992 doi
-
[24]
Chinedu Wilfred Okonkwo and Abejide Ade-Ibijola. 2020. Python-Bot: A Chatbot for Teaching Python Programming.Engineering Letters 29, 1 (March 2020), 25–34
2020
- [25]
-
[26]
Jimmy Pezzone. 2023. Harvard is bringing its own brand of generative AI to the classroom. https://www.techspot.com/news/99185-harvard-bringing-own- brand-generative-ai-classroom.html
2023
-
[27]
Thomas Price, Rui Zhi, and Tiffany Barnes. 2017. Evaluation of a Data-Driven Feedback Algorithm for Open-Ended Programming. Technical Report. International Educational Data Mining Society. https://eric.ed.gov/?id=ED596568 Publication Title: International Educational Data Minin...
2017
-
[28]
José Quiroga Pérez, Thanasis Daradoumis, and Joan Manuel Marquès Puig. 2020. Rediscovering the use of chatbots in education: A sys- tematic literature review. Computer Applications in Engineering Educa- tion 28, 6 (2020), 1549–1565. https://doi.org/10.1002/cae.22326 _eprint: h...
2020 doi
-
[29]
https://doi.org/10.1186/s40594-023-00418-7
-
[31]
Kyoungwon Seo, Joice Tang, Ido Roll, Sidney Fels, and Dongwook Yoon. 2021. The impact of artificial intelligence on learner–instructor interaction in online learning. International Journal of Educational Technology in Higher Education 18, 1 (Oct. 2021), 54. https://doi.org/10....
2021 doi
-
[32]
Zak Risha, Jordan Barria-Pineda, Kamil Akhuseyinoglu, and Peter Brusilovsky
-
[33]
In Proceedings of the 21st Koli Calling International Conference on Computing Education Research (Koli Calling ’21)
Stepwise Help and Scaffolding for Java Code Tracing Problems With an Interactive Trace Table. In Proceedings of the 21st Koli Calling International Conference on Computing Education Research (Koli Calling ’21) . Association for Computing Machinery, New York, NY, USA, 1–10. htt...
-
[34]
Kamilla Tenório, Viktoriya Olari, Margarita Chikobava, and Ralf Romeike. 2023. Artificial Intelligence Literacy Research Field: A Bibliometric Analysis from 1989 to 2021. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2023) . Asso...
2023
-
[35]
Yudelson, Kenneth R
Michael V. Yudelson, Kenneth R. Koedinger, and Geoffrey J. Gordon. 2013. In- dividualized Bayesian Knowledge Tracing Models. In Artificial Intelligence in Education, David Hutchison, Takeo Kanade, Josef Kittler, Jon M. Kleinberg, Friede- mann Mattern, John C. Mitchell, Moni Na...
2013
-
[36]
Donggil Song, Eun Young Oh, and Marilyn Rice. 2017. Interacting with a conversational agent system for educational purposes in online courses. In 2017 10th International Conference on Human System Interactions (HSI) . 78–82. https://doi.org/10.1109/HSI.2017.8005002
2017
-
[37]
Vera Liao, Michael Muller, Mayank Agarwal, Stephanie Houde, Kartik Talamadupula, and Justin D
Jiao Sun, Q. Vera Liao, Michael Muller, Mayank Agarwal, Stephanie Houde, Kartik Talamadupula, and Justin D. Weisz. 2022. Investigating Explainability of Genera- tive AI for Code through Scenario-based Design. In 27th International Conference on Intelligent User Interfaces (IUI...
2022
-
[41]
Xuesong Zhai, Xiaoyan Chu, Ching Sing Chai, Morris Siu Yung Jong, Andreja Istenic, Michael Spector, Jia-Bao Liu, Jing Yuan, and Yan Li. 2021. A Review of Artificial Intelligence (AI) in Education from 2010 to 2020.Complexity 2021 (April 2021), e8812542. https://doi.org/10.1155...
2021 doi
-
[180]
https://doi.org/10.1007/978-3-642-39112-5_18 Series Title: Lecture Notes in Computer Science
-
[1995]
Journal of the Learning Sciences 4, 2 (April 1995), 167–207
Cognitive Tutors: Lessons Learned. Journal of the Learning Sciences 4, 2 (April 1995), 167–207. https://doi.org/10.1207/s15327809jls0402_2 Publisher: Routledge _eprint: https://doi.org/10.1207/s15327809jls0402_2
1995 doi
-
[2021]
In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education (SIGCSE ’21)
TaskTracker-tool: A Toolkit for Tracking of Code Snapshots and Activity Data During Solution of Programming Tasks. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education (SIGCSE ’21) . Association for Computing Machinery, New York, NY, USA, 495–501. h...
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.