REVIEW 3 major objections 6 minor 34 references
Computing students rate short AI Markdown videos highly but resist using them as core classroom instruction.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-31 14:45 UTC pith:QV36RJJ7
load-bearing objection Solid descriptive CS-ed survey: students like short avatar-free AI Markdown videos but do not want them as core instruction—useful practice signal, modest novelty, claims stay inside the data. the 3 major comments →
Student Perceptions and Preferences Regarding AI-Generated Instructional Videos in Computing Education
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
After watching three short Knowlify-generated Markdown videos, computing students rate those videos as professionally produced, accurate, and helpful for learning, with roughly half unable to clearly identify them as AI-generated; at the same time they express limited comfort with widespread classroom adoption, prefer human-recorded instruction, and restrict appropriate use to simple, supplemental, and visual scenarios while citing risks of inaccuracy, reduced instructor interaction, and diminished educational value.
What carries the argument
A descriptive post-test survey design in which students first watch three short, avatar-free AI explainer videos on Markdown (without being told they are AI-generated) and then rate both those specific videos and broader future use of AI videos, paired with open-ended thematic coding of appropriate contexts and concerns.
Load-bearing premise
That reactions to three short, simple-syntax, avatar-free Markdown videos shown to mostly upper-level CS students will generalize to how students would judge AI videos on complex programming topics, longer formats, avatar presenters, or novice populations.
What would settle it
Run a controlled comparison in which the same students rate AI versus instructor-recorded videos on a complex CS topic (for example graph algorithms or debugging) in a longer live-coding format; if comfort, trust, and preference for AI then match or exceed the Markdown results, the paper's scoped-use claim weakens.
If this is right
- CS instructors can treat short AI explainer videos as low-friction supplements for basic syntax, visual walkthroughs, and recap material.
- Replacing core lectures or interactive sessions with AI video is likely to meet student resistance even when production quality is high.
- Disclosure and human fact-checking remain necessary because students already expect hallucination and shallow coverage.
- Avatar-free, animation-style generation may reduce uncanny-valley pushback relative to talking-head AI instructors.
- Cost and time savings of AI video production are usable today for narrow topics without waiting for perfect student trust.
Where Pith is reading between the lines
- The same student skepticism that already limits trust in text LLMs appears to transfer to video, so verification workflows designed for chat tools may transfer with little change.
- If later work finds that novices in CS1/CS2 are less critical of AI video than upper-level students, instructors may need different adoption rules by course level.
- Personalization features students mentioned (pace, language, catch-up paths) are a natural next product surface once basic quality is accepted.
- Institutions pricing tuition against human contact may face reputational risk if AI video is framed as wholesale lecture replacement rather than optional support.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This descriptive post-test survey study examines computing students’ perceptions of AI-generated instructional videos. 170 students at two U.S. institutions watched three short (~3 min), avatar-free Knowlify videos on Markdown, then rated video quality and broader classroom use and answered open-ended items on appropriate contexts and concerns. Students rated the study videos highly on production quality, accuracy, and usefulness (≈88–92% agreement), with only about half clearly identifying them as AI-generated, yet expressed limited comfort with widespread adoption, preferred AI videos mainly for simple/supplemental/visual uses, and raised concerns about inaccuracy, reduced instructor interaction, and diminished educational value. The paper reports viewing logs, a short Markdown quiz, Likert distributions (Fig. 2), and dual-author inductive thematic codes for RQs 2–3, and situates findings against prior work on AI videos and GenAI in CS education.
Significance. The work addresses a timely and under-studied gap: CS education research has focused heavily on text-based GenAI, while only limited prior work (notably Arkun et al.) has examined student perceptions of AI instructional video, and not for programming-oriented content in English without avatar presenters. The study is carefully scoped as descriptive rather than causal, uses two sites, attention checks, viewing-log corroboration, and transparent limitations (§6). If the reported frequencies and themes hold, the paper offers concrete, instructor-facing guidance on when AI videos are likely to be accepted (short, simple, supplemental, visual) versus resisted (core lectures, complex/subjective topics, wholesale replacement). Strengths include clear research questions, stimulus documentation, and dual-author consensus coding of open responses.
major comments (3)
- [§3.6 Data Analysis; §4.3–4.4] §3.6 and §4.3–4.4: The inductive thematic analysis is central to RQ2 and RQ3, yet the manuscript reports only that the first and last authors reviewed the codebook and resolved disagreements by consensus. No pre-consensus agreement statistic (e.g., percent agreement or Cohen’s κ on a double-coded subset) is given. For a journal audience this is a load-bearing transparency gap for the theme frequencies (e.g., “Inaccurate information” n=94). Please report how many responses were double-coded and an agreement metric, or explicitly justify single-pass consensus as sufficient for this descriptive design.
- [§4.1 Study Context] §4.1 (“Learning outcomes associated with video use”): The claim that students “learned fairly well” rests on post-only quiz means of 4.3/5 with no pretest, no comparison condition, and no item-level difficulty or prior-knowledge controls beyond self-reported Markdown familiarity. Limitations §6 correctly notes the design is not comparative, but §4.1 still frames the scores as evidence of learning from the videos. Soften or reframe this subsection to “post-exposure knowledge scores” and avoid causal language unless a baseline or control is added.
- [§3.3 Sample; Fig. 2; §4.2–4.4] §3.3 and Fig. 2: Institution 2 contributes only N=27 versus N=143 at Institution 1. Fig. 2 helpfully splits panels, but pooled μ/σ and many theme percentages in §4.2–4.4 are dominated by Institution 1. Please state explicitly whether any Likert item or major theme differed materially by site (even descriptively), and caveat pooled open-ended percentages accordingly so readers do not over-generalize the smaller site.
minor comments (6)
- [Abstract; §4.2] Abstract and §1: “nearly half unable to determine whether the videos were AI-generated” is slightly stronger than the item wording and results (50% agreed they could clearly tell; 35% disagreed; 15% neutral). Align the abstract phrasing with the exact item and distribution in Fig. 2.
- [§3.4 AI-generated videos] §3.4: Video cost/time figures ($50–60, ~25 minutes) are useful for instructors; briefly note whether Knowlify’s one-shot pipeline required substantive script edits beyond the stated verification time, since edit burden affects the “efficiently and cost-effectively” claim.
- [§3.5 Data Collection] §3.5: The five-item Markdown knowledge test is not included (items or answer key). Providing the items in an appendix would aid replication and let readers judge difficulty relative to the “simple topics” theme.
- [Figure 2] Figure 2: The stacked bars are informative but dense; ensure the camera-ready version has legible segment labels/percentages and a consistent neutral center. The footnote on N2=26 for one item should appear in the caption.
- [§2 Related Work] §2.1–2.2: Related work is appropriate; a brief forward pointer to how avatar-free design was chosen to reduce uncanny-valley confounds (later in §3.4/§5) would tighten the link to Arkun et al.
- Minor copy edits: arXiv/line-number artifacts and a few long sentences in §5 could be tightened; check consistency of “AI videos” vs “AI-generated videos” on first use in each section.
Circularity Check
No circularity: descriptive survey findings do not reduce to fitted inputs or self-definitional claims
full rationale
This paper is a post-test descriptive survey study (N=170) reporting Likert distributions, viewing logs, a short Markdown quiz, and inductively coded open-ended themes about three author-produced Knowlify videos. There is no derivation chain, no equations, no fitted parameters presented as predictions, no uniqueness theorem, and no load-bearing self-citation that forces the central claims. Quality/accuracy ratings were collected before the detectability item; preferences and concerns are independent open responses. Author-chosen stimuli and self-disclosed scope limits (simple Markdown, avatar-free, upper-level students) are design choices, not circular reasoning. Score 0 is the correct honest finding.
Axiom & Free-Parameter Ledger
axioms (5)
- domain assumption Post-exposure self-report Likert items and open-ended thematic codes are valid indicators of students' perceptions and stated preferences regarding AI instructional videos.
- domain assumption Withholding AI origin until after quality ratings removes demand-characteristic bias sufficiently to interpret quality and detectability items.
- ad hoc to paper Three short, avatar-free Knowlify videos on Markdown are a reasonable probe of AI-generated instructional video in computing education.
- domain assumption Inductive thematic analysis with two-author consensus yields stable theme frequencies suitable for reporting prevalence (e.g., 58% accuracy concerns).
- domain assumption Upper-level CS students at two competitive U.S. institutions, incentivized with extra credit, are an informative sample for computing-education practice guidance.
read the original abstract
Students differ in how they prefer to engage with learning resources, with some favoring textual materials and others visual or video-based content. Recent advances in generative AI have led CS education research to focus on text-based AI tools for developing learning resources. However, advances in AI video models and the rapid proliferation of AI video generation tools have made it possible for instructors to create high-quality personalized educational videos efficiently and cost-effectively. Understanding students' perceptions of AI-generated videos is thus critical for helping CS instructors know when and how to use them purposefully. To address this gap, we conducted a descriptive post-test survey study in which 170 computing students at two U.S. institutions watched three 3-minute AI videos on the Markdown markup language created with Knowlify. Students then completed a survey about their perceptions of the Markdown videos and their broader views on the use of AI-generated videos in education. Students rated the Markdown videos as high-quality, accurate, and usable, with nearly half unable to determine whether the videos were AI-generated. At the same time, students expressed limited comfort with the widespread adoption of AI videos in the classroom. They preferred AI videos for simple, supplemental, and visual use cases, while expressing concerns about lower-quality or inaccurate content, reduced instructor interaction, and diminished educational value.
Figures
Reference graph
Works this paper leans on
-
[1]
AI-generated Markdown Videos used in this study
2026. AI-generated Markdown Videos used in this study. https://drive.google. com/drive/folders/1te3WVrun2bCWMnzAkUBeRcrj0qzPp00T. Google Drive folder, accessed May 7, 2026
2026
-
[2]
Matin Amoozadeh, David Daniels, Daye Nam, Aayush Kumar, Stella Chen, Michael Hilton, Sruti Srinivasa Ragavan, and Mohammad Amin Alipour. 2024. Trust in Generative AI among Students: An exploratory study. InProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1. ACM, Portland OR USA, 67–73. doi:10.1145/3626252.3630842
arXiv 2024
-
[3]
Selay Arkün-Kocadere and Şeyma Çağlar Özhan. 2024. Video Lectures With AI-Generated Instructors: Low Video Engagement, Same Performance as Human Instructors.The International Review of Research in Open and Distributed Learning 25, 3 (Aug. 2024), 350–369. doi:10.19173/irrodl.v25i3.7815
-
[4]
Dennis J. Bouvier, Bruno Pereira Cipriano, Richard Glassey, Olga Petrovska, Emma Anderson, Anastasiia Birillo, Ryan Dougherty, Raymond Pettit, Nuno Pombo, Ebrahim Rahimi, Charanya Ramakrishnan, Alexander Steinmaurer, Shubbhi Taneja, Muhammad Usman, and Annapurna Vadaparty. 2025. The Rest of the Robots: Generative AI in Post-introductory Computing Educa- t...
arXiv 2025
-
[5]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology3, 2 (2006), 77–101
2006
-
[6]
GitHub. 2026. Markdown basic writing and formatting syntax. https://docs.github.com/en/get-started/writing-on-github/getting-started- with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax. Accessed: 2026-05-07
2026
-
[7]
Philip J. Guo, Juho Kim, and Rob Rubin. 2014. How video production affects student engagement: an empirical study of MOOC videos. InProceedings of the First ACM Conference on Learning @ Scale Conference(Atlanta, Georgia, USA) (L@S ’14). Association for Computing Machinery, New York, NY, USA, 41–50. doi:10.1145/2556325.2566239
arXiv 2014
-
[8]
Steffi Heidig and Geraldine Clarebout. 2011. Do pedagogical agents make a difference to student motivation and learning?Educational Research Review6, 1 (2011), 27–54. doi:10.1016/j.edurev.2010.07.004
-
[9]
Antti Herala, Antti Knutas, Erno Vanhala, and Jussi Kasurinen. 2017. Experiences from Video Lectures in Software Engineering Education.International Journal of Modern Education and Computer Science9, 5 (2017), 17–26. doi:10.5815/ijmecs. 2017.05.03
-
[10]
Irene Hou, Owen Man, Kate Hamilton, Srishty Muthusekaran, Jeffin Johnykutty, Leili Zadeh, and Stephen MacNeil. 2025. ’All Roads Lead to ChatGPT’: How Gen- erative AI is Eroding Social Interactions and Student Learning Communities. In Proceedings of the 30th ACM Conference on Innovation and Technology in Computer Science Education V. 1(Nijmegen, Netherland...
arXiv 2025
-
[11]
Irene Hou, Sophia Mettille, Owen Man, Zhuo Li, Cynthia Zastudil, and Stephen MacNeil. 2024. The Effects of Generative AI on Computing Students’ Help- Seeking Preferences. InProceedings of the 26th Australasian Computing Education Conference. ACM, Sydney NSW Australia, 39–48. doi:10.1145/3636243.3636248
arXiv 2024
-
[12]
Hieke Keuning, Isaac Alpizar-Chacon, Ioanna Lykourentzou, Lauren Beehler, Christian Köppe, Imke De Jong, and Sergey Sosnovsky. 2024. Students’ Per- ceptions and Use of Generative AI Tools for Programming Across Different Computing Courses. InProceedings of the 24th Koli Calling International Confer- ence on Computing Education Research. ACM, Koli Finland,...
arXiv 2024
-
[13]
Knowlify. 2026. AI Explainer Video Maker. https://knowlify.com/. Accessed: 2026-05-07
2026
-
[14]
Sam Lau and Philip Guo. 2023. From "Ban It Till We Understand It" to "Re- sistance is Futile": How University Programming Instructors Plan to Adapt as More Students Use AI Code Generation and Explanation Tools such as Chat- GPT and GitHub Copilot. InProceedings of the 2023 ACM Conference on Inter- national Computing Education Research - Volume 1(Chicago, ...
arXiv 2023
-
[15]
Daniel Leiker, Ashley Ricker Gyllen, Ismail Eldesouky, and Mutlu Cukurova. 2023. Generative AI for Learning: Investigating the Potential of Learning Videos with Synthetic Virtual Instructors. InArtificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortiu...
2023
-
[16]
Xiao Liu, Xinhao Xiang, Zizhong Li, Yongheng Wang, Zhuoheng Li, Zhuosheng Liu, Weidi Zhang, Weiqi Ye, and Jiawei Zhang. 2026. A Survey of AI-Generated Video Evaluation. doi:10.48550/arXiv.2410.19884 arXiv:2410.19884 [cs]
-
[17]
Aidan McGowan, Philip Hanna, and Neil Anderson. 2016. Teaching Programming: Understanding Lecture Capture YouTube Analytics. InProceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education. ACM, Arequipa Peru, 35–40. doi:10.1145/2899415.2899421
arXiv 2016
-
[18]
Torbjørn Netland, Oliver Von Dzengelevski, Katalin Tesch, and Daniel Kwas- nitschka. 2025. Comparing human-made and AI-generated teaching videos: An experimental study on learning effects.Computers & Education224 (Jan. 2025), 105164. doi:10.1016/j.compedu.2024.105164
arXiv 2025
-
[19]
Nikolaos Pellas. 2023. The influence of sociodemographic factors on students’ atti- tudes toward AI-generated video content creation.Smart Learning Environments 10, 1 (Nov. 2023), 57. doi:10.1186/s40561-023-00276-4
-
[20]
Zhongling Pi, Lixia Deng, Xu Wang, Peirong Guo, Tao Xu, and Yun Zhou. 2022. The influences of a virtual instructor’s voice and appearance on learning from video lectures.Journal of Computer Assisted Learning38, 6 (Dec. 2022), 1703–1713. doi:10.1111/jcal.12704
-
[21]
James Prather, Paul Denny, Juho Leinonen, Brett A. Becker, Ibrahim Albluwi, Michelle Craig, Hieke Keuning, Natalie Kiesler, Tobias Kohn, Andrew Luxton- Reilly, Stephen MacNeil, Andrew Petersen, Raymond Pettit, Brent N. Reeves, and Jaromir Savelka. 2023. The Robots Are Here: Navigating the Generative AI Revolution in Computing Education. InProceedings of t...
arXiv 2023
-
[22]
It’s Weird That it Knows What I Want
James Prather, Brent N. Reeves, Paul Denny, Brett A. Becker, Juho Leinonen, Andrew Luxton-Reilly, Garrett Powell, James Finnie-Ansley, and Eddie Antonio Santos. 2023. “It’s Weird That it Knows What I Want”: Usability and Interactions with Copilot for Novice Programmers.ACM Trans. Comput.-Hum. Interact.31, 1, Article 4 (Nov. 2023), 31 pages. doi:10.1145/3617367
doi:10.1145/3617367 2023
-
[23]
Rogers, Hannah Miller Hillberg, and Christopher L
Michael P. Rogers, Hannah Miller Hillberg, and Christopher L. Groves. 2024. Attitudes Towards the Use (and Misuse) of ChatGPT: A Preliminary Study. In Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1. ACM, Portland OR USA, 1147–1153. doi:10.1145/3626252.3630784
arXiv 2024
-
[24]
Kühne, Léane Wettstein, and Florian Brühlmann
Nicolas Scharowski, Michaela Benk, Swen J. Kühne, Léane Wettstein, and Florian Brühlmann. 2023. Certification Labels for Trustworthy AI: Insights From an Em- pirical Mixed-Method Study. In2023 ACM Conference on Fairness Accountability and Transparency. ACM, Chicago IL USA, 248–260. doi:10.1145/3593013.3593994
arXiv 2023
-
[25]
Michael James Scott and Gheorghita Ghinea. 2017. On the Educational Impact of Lecture Recording Reduction: Evidence from a Randomised Trial. InProceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education. ACM, Bologna Italy, 287–292. doi:10.1145/3059009.3059037
arXiv 2017
-
[26]
Amber Settle, Lucia Dettori, and Mary Davidson. 2011. Does lecture capture make a difference for students in traditional classrooms. InProceedings of the 16th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education, ITiCSE 2011. 78–82. doi:10.1145/1999747.1999772
arXiv 2011
-
[27]
Mohamad-Hani Temsah, Rakan Nazer, Ibraheem Altamimi, Raniah Aldekhyyel, Amr Jamal, Mohammad Almansour, Fadi Aljamaan, Khalid Alhasan, Abdulka- rim A Temsah, Ayman Al-Eyadhy, Bandar N Aljafen, and Khalid H Malki. 2025. OpenAI’s Sora and Google’s Veo 2 in Action: A Narrative Review of Artificial Intelligence-driven Video Generation Models Transforming Healt...
-
[28]
Carmen Vallis, Stephanie Wilson, Daniel Gozman, and John Buchanan. 2024. Student Perceptions of AI-Generated Avatars in Teaching Business Ethics: We Might not be Impressed.Postdigital Science and Education6, 2 (June 2024), 537–555. doi:10.1007/s42438-023-00407-7
-
[29]
Ethan Waisberg, Joshua Ong, Rahul Kumar, Mouayad Masalkhi, and Andrew G Lee. 2024. OpenAI’s Sora in medical education: artificial videos in the classroom of the future.Canadian Medical Education Journal(Nov. 2024). doi:10.36834/ cmej.79065
2024
-
[30]
Oshani Weerakoon, Ville Leppänen, and Tuomas Mäkilä. 2024. Enhancing Peda- gogy with Generative AI: Video Production from Course Descriptions. InPro- ceedings of the International Conference on Computer Systems and Technologies
2024
-
[31]
font-variant:small-caps
Tao Xu, Yuan Liu, Yaru Jin, Yueyao Qu, Jie Bai, Wenlan Zhang, and Yun Zhou. 2025. From recorded to <span style="font-variant:small-caps;">AI</span> -generated instructional videos: A comparison of learning performance and experience. British Journal of Educational Technology56, 4 (July 2025), 1463–1487. doi:10. 1111/bjet.13530
2025
-
[32]
Yidan Yin, Nan Jia, and Cheryl J. Wakslak. 2024. AI can help people feel heard, but an AI label diminishes this impact.Proceedings of the National Academy of Sciences121, 14 (April 2024), e2319112121. doi:10.1073/pnas.2319112121
-
[33]
Cynthia Zastudil, Magdalena Rogalska, Christine Kapp, Jennifer Vaughn, and Stephen MacNeil. 2023. Generative AI in Computing Education: Perspectives of Students and Instructors. In2023 IEEE Frontiers in Education Conference (FIE). IEEE, College Station, TX, USA, 1–9. doi:10.1109/FIE58773.2023.10343467
arXiv 2023
- [2024]
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.