Pith. sign in

REVIEW 3 major objections 6 minor 34 references

Computing students rate short AI Markdown videos highly but resist using them as core classroom instruction.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · grok-4.5

2026-07-31 14:45 UTC pith:QV36RJJ7

load-bearing objection Solid descriptive CS-ed survey: students like short avatar-free AI Markdown videos but do not want them as core instruction—useful practice signal, modest novelty, claims stay inside the data. the 3 major comments →

arxiv 2607.28203 v1 pith:QV36RJJ7 submitted 2026-07-30 cs.HC

Student Perceptions and Preferences Regarding AI-Generated Instructional Videos in Computing Education

classification cs.HC
keywords Generative AIAI VideosComputing EducationStudent PerceptionsInstructional VideosMarkdownCS Education
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks how computing students judge AI-generated instructional videos once they have actually watched some. One hundred seventy students at two U.S. universities watched three short, avatar-free AI videos on Markdown, then rated those videos and answered open questions about when AI videos belong in CS courses. Students called the study videos high-quality, accurate, and usable, and nearly half could not tell they were AI-made. Yet the same students were lukewarm about adopting AI videos as regular instructional material, preferred instructor- or staff-recorded videos, and trusted human-presented information more. They endorsed AI videos mainly for simple topics, visual explanations, summaries, and supplemental use, while worrying about hallucinations, shallow content, lost instructor contact, and weaker educational value. The practical upshot is guidance for CS instructors: AI video can fill narrow, well-scoped roles without displacing the human presence students still want.

Core claim

After watching three short Knowlify-generated Markdown videos, computing students rate those videos as professionally produced, accurate, and helpful for learning, with roughly half unable to clearly identify them as AI-generated; at the same time they express limited comfort with widespread classroom adoption, prefer human-recorded instruction, and restrict appropriate use to simple, supplemental, and visual scenarios while citing risks of inaccuracy, reduced instructor interaction, and diminished educational value.

What carries the argument

A descriptive post-test survey design in which students first watch three short, avatar-free AI explainer videos on Markdown (without being told they are AI-generated) and then rate both those specific videos and broader future use of AI videos, paired with open-ended thematic coding of appropriate contexts and concerns.

Load-bearing premise

That reactions to three short, simple-syntax, avatar-free Markdown videos shown to mostly upper-level CS students will generalize to how students would judge AI videos on complex programming topics, longer formats, avatar presenters, or novice populations.

What would settle it

Run a controlled comparison in which the same students rate AI versus instructor-recorded videos on a complex CS topic (for example graph algorithms or debugging) in a longer live-coding format; if comfort, trust, and preference for AI then match or exceed the Markdown results, the paper's scoped-use claim weakens.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • CS instructors can treat short AI explainer videos as low-friction supplements for basic syntax, visual walkthroughs, and recap material.
  • Replacing core lectures or interactive sessions with AI video is likely to meet student resistance even when production quality is high.
  • Disclosure and human fact-checking remain necessary because students already expect hallucination and shallow coverage.
  • Avatar-free, animation-style generation may reduce uncanny-valley pushback relative to talking-head AI instructors.
  • Cost and time savings of AI video production are usable today for narrow topics without waiting for perfect student trust.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same student skepticism that already limits trust in text LLMs appears to transfer to video, so verification workflows designed for chat tools may transfer with little change.
  • If later work finds that novices in CS1/CS2 are less critical of AI video than upper-level students, instructors may need different adoption rules by course level.
  • Personalization features students mentioned (pace, language, catch-up paths) are a natural next product surface once basic quality is accepted.
  • Institutions pricing tuition against human contact may face reputational risk if AI video is framed as wholesale lecture replacement rather than optional support.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This descriptive post-test survey study examines computing students’ perceptions of AI-generated instructional videos. 170 students at two U.S. institutions watched three short (~3 min), avatar-free Knowlify videos on Markdown, then rated video quality and broader classroom use and answered open-ended items on appropriate contexts and concerns. Students rated the study videos highly on production quality, accuracy, and usefulness (≈88–92% agreement), with only about half clearly identifying them as AI-generated, yet expressed limited comfort with widespread adoption, preferred AI videos mainly for simple/supplemental/visual uses, and raised concerns about inaccuracy, reduced instructor interaction, and diminished educational value. The paper reports viewing logs, a short Markdown quiz, Likert distributions (Fig. 2), and dual-author inductive thematic codes for RQs 2–3, and situates findings against prior work on AI videos and GenAI in CS education.

Significance. The work addresses a timely and under-studied gap: CS education research has focused heavily on text-based GenAI, while only limited prior work (notably Arkun et al.) has examined student perceptions of AI instructional video, and not for programming-oriented content in English without avatar presenters. The study is carefully scoped as descriptive rather than causal, uses two sites, attention checks, viewing-log corroboration, and transparent limitations (§6). If the reported frequencies and themes hold, the paper offers concrete, instructor-facing guidance on when AI videos are likely to be accepted (short, simple, supplemental, visual) versus resisted (core lectures, complex/subjective topics, wholesale replacement). Strengths include clear research questions, stimulus documentation, and dual-author consensus coding of open responses.

major comments (3)
  1. [§3.6 Data Analysis; §4.3–4.4] §3.6 and §4.3–4.4: The inductive thematic analysis is central to RQ2 and RQ3, yet the manuscript reports only that the first and last authors reviewed the codebook and resolved disagreements by consensus. No pre-consensus agreement statistic (e.g., percent agreement or Cohen’s κ on a double-coded subset) is given. For a journal audience this is a load-bearing transparency gap for the theme frequencies (e.g., “Inaccurate information” n=94). Please report how many responses were double-coded and an agreement metric, or explicitly justify single-pass consensus as sufficient for this descriptive design.
  2. [§4.1 Study Context] §4.1 (“Learning outcomes associated with video use”): The claim that students “learned fairly well” rests on post-only quiz means of 4.3/5 with no pretest, no comparison condition, and no item-level difficulty or prior-knowledge controls beyond self-reported Markdown familiarity. Limitations §6 correctly notes the design is not comparative, but §4.1 still frames the scores as evidence of learning from the videos. Soften or reframe this subsection to “post-exposure knowledge scores” and avoid causal language unless a baseline or control is added.
  3. [§3.3 Sample; Fig. 2; §4.2–4.4] §3.3 and Fig. 2: Institution 2 contributes only N=27 versus N=143 at Institution 1. Fig. 2 helpfully splits panels, but pooled μ/σ and many theme percentages in §4.2–4.4 are dominated by Institution 1. Please state explicitly whether any Likert item or major theme differed materially by site (even descriptively), and caveat pooled open-ended percentages accordingly so readers do not over-generalize the smaller site.
minor comments (6)
  1. [Abstract; §4.2] Abstract and §1: “nearly half unable to determine whether the videos were AI-generated” is slightly stronger than the item wording and results (50% agreed they could clearly tell; 35% disagreed; 15% neutral). Align the abstract phrasing with the exact item and distribution in Fig. 2.
  2. [§3.4 AI-generated videos] §3.4: Video cost/time figures ($50–60, ~25 minutes) are useful for instructors; briefly note whether Knowlify’s one-shot pipeline required substantive script edits beyond the stated verification time, since edit burden affects the “efficiently and cost-effectively” claim.
  3. [§3.5 Data Collection] §3.5: The five-item Markdown knowledge test is not included (items or answer key). Providing the items in an appendix would aid replication and let readers judge difficulty relative to the “simple topics” theme.
  4. [Figure 2] Figure 2: The stacked bars are informative but dense; ensure the camera-ready version has legible segment labels/percentages and a consistent neutral center. The footnote on N2=26 for one item should appear in the caption.
  5. [§2 Related Work] §2.1–2.2: Related work is appropriate; a brief forward pointer to how avatar-free design was chosen to reduce uncanny-valley confounds (later in §3.4/§5) would tighten the link to Arkun et al.
  6. Minor copy edits: arXiv/line-number artifacts and a few long sentences in §5 could be tightened; check consistency of “AI videos” vs “AI-generated videos” on first use in each section.

Circularity Check

0 steps flagged

No circularity: descriptive survey findings do not reduce to fitted inputs or self-definitional claims

full rationale

This paper is a post-test descriptive survey study (N=170) reporting Likert distributions, viewing logs, a short Markdown quiz, and inductively coded open-ended themes about three author-produced Knowlify videos. There is no derivation chain, no equations, no fitted parameters presented as predictions, no uniqueness theorem, and no load-bearing self-citation that forces the central claims. Quality/accuracy ratings were collected before the detectability item; preferences and concerns are independent open responses. Author-chosen stimuli and self-disclosed scope limits (simple Markdown, avatar-free, upper-level students) are design choices, not circular reasoning. Score 0 is the correct honest finding.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 0 invented entities

Descriptive survey paper; load-bearing commitments are methodological and scope assumptions rather than free physical parameters or invented entities. Central claims rest on standard survey/thematic-analysis practice plus the assumption that these Knowlify Markdown clips are informative about AI instructional video more broadly.

axioms (5)
  • domain assumption Post-exposure self-report Likert items and open-ended thematic codes are valid indicators of students' perceptions and stated preferences regarding AI instructional videos.
    Entire RQ1–RQ3 analysis (§3.5–3.6, §4) treats survey responses as the evidence base without behavioral or longitudinal validation.
  • domain assumption Withholding AI origin until after quality ratings removes demand-characteristic bias sufficiently to interpret quality and detectability items.
    Stated design choice in §3.5 citing prior labeling effects; underpins the 'half could not tell' claim.
  • ad hoc to paper Three short, avatar-free Knowlify videos on Markdown are a reasonable probe of AI-generated instructional video in computing education.
    Stimulus construction §3.4; limitations §6 note content simplicity, length, and tool specificity constrain generalization.
  • domain assumption Inductive thematic analysis with two-author consensus yields stable theme frequencies suitable for reporting prevalence (e.g., 58% accuracy concerns).
    §3.6 methods; no IRR statistic reported, but dual review is described.
  • domain assumption Upper-level CS students at two competitive U.S. institutions, incentivized with extra credit, are an informative sample for computing-education practice guidance.
    Recruitment §3.2–3.3; limits claims about novices and other institutional contexts (§6).

pith-pipeline@v1.2.0-daily-grok45 · 17804 in / 3008 out tokens · 61361 ms · 2026-07-31T14:45:49.360428+00:00 · methodology

0 comments
read the original abstract

Students differ in how they prefer to engage with learning resources, with some favoring textual materials and others visual or video-based content. Recent advances in generative AI have led CS education research to focus on text-based AI tools for developing learning resources. However, advances in AI video models and the rapid proliferation of AI video generation tools have made it possible for instructors to create high-quality personalized educational videos efficiently and cost-effectively. Understanding students' perceptions of AI-generated videos is thus critical for helping CS instructors know when and how to use them purposefully. To address this gap, we conducted a descriptive post-test survey study in which 170 computing students at two U.S. institutions watched three 3-minute AI videos on the Markdown markup language created with Knowlify. Students then completed a survey about their perceptions of the Markdown videos and their broader views on the use of AI-generated videos in education. Students rated the Markdown videos as high-quality, accurate, and usable, with nearly half unable to determine whether the videos were AI-generated. At the same time, students expressed limited comfort with the widespread adoption of AI videos in the classroom. They preferred AI videos for simple, supplemental, and visual use cases, while expressing concerns about lower-quality or inaccurate content, reduced instructor interaction, and diminished educational value.

Figures

Figures reproduced from arXiv: 2607.28203 by Amanpreet Kapoor, Esse Ciego, Shubbhi Taneja, Wilson Wong.

Figure 1
Figure 1. Figure 1: Video Frame from AI-generated Video 1: Intro to [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Distribution of student perceptions (N=170) for AI videos used in the study and future inclusion of AI Videos in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 7 canonical work pages

  1. [1]

    AI-generated Markdown Videos used in this study

    2026. AI-generated Markdown Videos used in this study. https://drive.google. com/drive/folders/1te3WVrun2bCWMnzAkUBeRcrj0qzPp00T. Google Drive folder, accessed May 7, 2026

  2. [2]

    Matin Amoozadeh, David Daniels, Daye Nam, Aayush Kumar, Stella Chen, Michael Hilton, Sruti Srinivasa Ragavan, and Mohammad Amin Alipour. 2024. Trust in Generative AI among Students: An exploratory study. InProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1. ACM, Portland OR USA, 67–73. doi:10.1145/3626252.3630842

  3. [3]

    Selay Arkün-Kocadere and Şeyma Çağlar Özhan. 2024. Video Lectures With AI-Generated Instructors: Low Video Engagement, Same Performance as Human Instructors.The International Review of Research in Open and Distributed Learning 25, 3 (Aug. 2024), 350–369. doi:10.19173/irrodl.v25i3.7815

  4. [4]

    Dennis J. Bouvier, Bruno Pereira Cipriano, Richard Glassey, Olga Petrovska, Emma Anderson, Anastasiia Birillo, Ryan Dougherty, Raymond Pettit, Nuno Pombo, Ebrahim Rahimi, Charanya Ramakrishnan, Alexander Steinmaurer, Shubbhi Taneja, Muhammad Usman, and Annapurna Vadaparty. 2025. The Rest of the Robots: Generative AI in Post-introductory Computing Educa- t...

  5. [5]

    Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology3, 2 (2006), 77–101

  6. [6]

    GitHub. 2026. Markdown basic writing and formatting syntax. https://docs.github.com/en/get-started/writing-on-github/getting-started- with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax. Accessed: 2026-05-07

  7. [7]

    Guo, Juho Kim, and Rob Rubin

    Philip J. Guo, Juho Kim, and Rob Rubin. 2014. How video production affects student engagement: an empirical study of MOOC videos. InProceedings of the First ACM Conference on Learning @ Scale Conference(Atlanta, Georgia, USA) (L@S ’14). Association for Computing Machinery, New York, NY, USA, 41–50. doi:10.1145/2556325.2566239

  8. [8]

    Steffi Heidig and Geraldine Clarebout. 2011. Do pedagogical agents make a difference to student motivation and learning?Educational Research Review6, 1 (2011), 27–54. doi:10.1016/j.edurev.2010.07.004

  9. [9]

    Antti Herala, Antti Knutas, Erno Vanhala, and Jussi Kasurinen. 2017. Experiences from Video Lectures in Software Engineering Education.International Journal of Modern Education and Computer Science9, 5 (2017), 17–26. doi:10.5815/ijmecs. 2017.05.03

  10. [10]

    Irene Hou, Owen Man, Kate Hamilton, Srishty Muthusekaran, Jeffin Johnykutty, Leili Zadeh, and Stephen MacNeil. 2025. ’All Roads Lead to ChatGPT’: How Gen- erative AI is Eroding Social Interactions and Student Learning Communities. In Proceedings of the 30th ACM Conference on Innovation and Technology in Computer Science Education V. 1(Nijmegen, Netherland...

  11. [11]

    Irene Hou, Sophia Mettille, Owen Man, Zhuo Li, Cynthia Zastudil, and Stephen MacNeil. 2024. The Effects of Generative AI on Computing Students’ Help- Seeking Preferences. InProceedings of the 26th Australasian Computing Education Conference. ACM, Sydney NSW Australia, 39–48. doi:10.1145/3636243.3636248

  12. [12]

    Hieke Keuning, Isaac Alpizar-Chacon, Ioanna Lykourentzou, Lauren Beehler, Christian Köppe, Imke De Jong, and Sergey Sosnovsky. 2024. Students’ Per- ceptions and Use of Generative AI Tools for Programming Across Different Computing Courses. InProceedings of the 24th Koli Calling International Confer- ence on Computing Education Research. ACM, Koli Finland,...

  13. [13]

    Knowlify. 2026. AI Explainer Video Maker. https://knowlify.com/. Accessed: 2026-05-07

  14. [14]

    Ban It Till We Understand It

    Sam Lau and Philip Guo. 2023. From "Ban It Till We Understand It" to "Re- sistance is Futile": How University Programming Instructors Plan to Adapt as More Students Use AI Code Generation and Explanation Tools such as Chat- GPT and GitHub Copilot. InProceedings of the 2023 ACM Conference on Inter- national Computing Education Research - Volume 1(Chicago, ...

  15. [15]

    Daniel Leiker, Ashley Ricker Gyllen, Ismail Eldesouky, and Mutlu Cukurova. 2023. Generative AI for Learning: Investigating the Potential of Learning Videos with Synthetic Virtual Instructors. InArtificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortiu...

  16. [16]

    Xiao Liu, Xinhao Xiang, Zizhong Li, Yongheng Wang, Zhuoheng Li, Zhuosheng Liu, Weidi Zhang, Weiqi Ye, and Jiawei Zhang. 2026. A Survey of AI-Generated Video Evaluation. doi:10.48550/arXiv.2410.19884 arXiv:2410.19884 [cs]

  17. [17]

    Aidan McGowan, Philip Hanna, and Neil Anderson. 2016. Teaching Programming: Understanding Lecture Capture YouTube Analytics. InProceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education. ACM, Arequipa Peru, 35–40. doi:10.1145/2899415.2899421

  18. [18]

    Torbjørn Netland, Oliver Von Dzengelevski, Katalin Tesch, and Daniel Kwas- nitschka. 2025. Comparing human-made and AI-generated teaching videos: An experimental study on learning effects.Computers & Education224 (Jan. 2025), 105164. doi:10.1016/j.compedu.2024.105164

  19. [19]

    Nikolaos Pellas. 2023. The influence of sociodemographic factors on students’ atti- tudes toward AI-generated video content creation.Smart Learning Environments 10, 1 (Nov. 2023), 57. doi:10.1186/s40561-023-00276-4

  20. [20]

    Zhongling Pi, Lixia Deng, Xu Wang, Peirong Guo, Tao Xu, and Yun Zhou. 2022. The influences of a virtual instructor’s voice and appearance on learning from video lectures.Journal of Computer Assisted Learning38, 6 (Dec. 2022), 1703–1713. doi:10.1111/jcal.12704

  21. [21]

    Becker, Ibrahim Albluwi, Michelle Craig, Hieke Keuning, Natalie Kiesler, Tobias Kohn, Andrew Luxton- Reilly, Stephen MacNeil, Andrew Petersen, Raymond Pettit, Brent N

    James Prather, Paul Denny, Juho Leinonen, Brett A. Becker, Ibrahim Albluwi, Michelle Craig, Hieke Keuning, Natalie Kiesler, Tobias Kohn, Andrew Luxton- Reilly, Stephen MacNeil, Andrew Petersen, Raymond Pettit, Brent N. Reeves, and Jaromir Savelka. 2023. The Robots Are Here: Navigating the Generative AI Revolution in Computing Education. InProceedings of t...

  22. [22]

    It’s Weird That it Knows What I Want

    James Prather, Brent N. Reeves, Paul Denny, Brett A. Becker, Juho Leinonen, Andrew Luxton-Reilly, Garrett Powell, James Finnie-Ansley, and Eddie Antonio Santos. 2023. “It’s Weird That it Knows What I Want”: Usability and Interactions with Copilot for Novice Programmers.ACM Trans. Comput.-Hum. Interact.31, 1, Article 4 (Nov. 2023), 31 pages. doi:10.1145/3617367

  23. [23]

    Rogers, Hannah Miller Hillberg, and Christopher L

    Michael P. Rogers, Hannah Miller Hillberg, and Christopher L. Groves. 2024. Attitudes Towards the Use (and Misuse) of ChatGPT: A Preliminary Study. In Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1. ACM, Portland OR USA, 1147–1153. doi:10.1145/3626252.3630784

  24. [24]

    Kühne, Léane Wettstein, and Florian Brühlmann

    Nicolas Scharowski, Michaela Benk, Swen J. Kühne, Léane Wettstein, and Florian Brühlmann. 2023. Certification Labels for Trustworthy AI: Insights From an Em- pirical Mixed-Method Study. In2023 ACM Conference on Fairness Accountability and Transparency. ACM, Chicago IL USA, 248–260. doi:10.1145/3593013.3593994

  25. [25]

    Michael James Scott and Gheorghita Ghinea. 2017. On the Educational Impact of Lecture Recording Reduction: Evidence from a Randomised Trial. InProceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education. ACM, Bologna Italy, 287–292. doi:10.1145/3059009.3059037

  26. [26]

    Amber Settle, Lucia Dettori, and Mary Davidson. 2011. Does lecture capture make a difference for students in traditional classrooms. InProceedings of the 16th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education, ITiCSE 2011. 78–82. doi:10.1145/1999747.1999772

  27. [27]

    Mohamad-Hani Temsah, Rakan Nazer, Ibraheem Altamimi, Raniah Aldekhyyel, Amr Jamal, Mohammad Almansour, Fadi Aljamaan, Khalid Alhasan, Abdulka- rim A Temsah, Ayman Al-Eyadhy, Bandar N Aljafen, and Khalid H Malki. 2025. OpenAI’s Sora and Google’s Veo 2 in Action: A Narrative Review of Artificial Intelligence-driven Video Generation Models Transforming Healt...

  28. [28]

    Carmen Vallis, Stephanie Wilson, Daniel Gozman, and John Buchanan. 2024. Student Perceptions of AI-Generated Avatars in Teaching Business Ethics: We Might not be Impressed.Postdigital Science and Education6, 2 (June 2024), 537–555. doi:10.1007/s42438-023-00407-7

  29. [29]

    Ethan Waisberg, Joshua Ong, Rahul Kumar, Mouayad Masalkhi, and Andrew G Lee. 2024. OpenAI’s Sora in medical education: artificial videos in the classroom of the future.Canadian Medical Education Journal(Nov. 2024). doi:10.36834/ cmej.79065

  30. [30]

    Oshani Weerakoon, Ville Leppänen, and Tuomas Mäkilä. 2024. Enhancing Peda- gogy with Generative AI: Video Production from Course Descriptions. InPro- ceedings of the International Conference on Computer Systems and Technologies

  31. [31]

    font-variant:small-caps

    Tao Xu, Yuan Liu, Yaru Jin, Yueyao Qu, Jie Bai, Wenlan Zhang, and Yun Zhou. 2025. From recorded to <span style="font-variant:small-caps;">AI</span> -generated instructional videos: A comparison of learning performance and experience. British Journal of Educational Technology56, 4 (July 2025), 1463–1487. doi:10. 1111/bjet.13530

  32. [32]

    Yidan Yin, Nan Jia, and Cheryl J. Wakslak. 2024. AI can help people feel heard, but an AI label diminishes this impact.Proceedings of the National Academy of Sciences121, 14 (April 2024), e2319112121. doi:10.1073/pnas.2319112121

  33. [33]

    Cynthia Zastudil, Magdalena Rogalska, Christine Kapp, Jennifer Vaughn, and Stephen MacNeil. 2023. Generative AI in Computing Education: Perspectives of Students and Instructors. In2023 IEEE Frontiers in Education Conference (FIE). IEEE, College Station, TX, USA, 1–9. doi:10.1109/FIE58773.2023.10343467

  34. [2024]

    doi:10.1145/3674912.3674922

    ACM, Ruse Bulgaria, 249–255. doi:10.1145/3674912.3674922