REVIEW 3 major objections 6 minor 10 references
Teacher training in the age of AI: Impact on AI Literacy and Teachers' Attitudes
T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper argues that a four-week structured online teacher training course raises in-service teachers' AI literacy and shifts their attitudes, reported usage, and preparedness to integrate AI into teaching.
desk verdict Useful pre-post evaluation of a real teacher AI course, but the abstract's causal claims outrun the design and the attitude findings are over-read. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is the training program itself: a four-week online course combining a kickoff webinar, a core self-paced module, optional units on machine learning, ethics, inclusion, and data protection, live webinars, and a project phase in which participants build an AI-supported lesson plan. The outcome machinery is a validated 29-item AI literacy test based on Long and Magerko's 17-competency framework, scored as the sum of correct answers. The course is designed to combine knowledge transfer, hands-on tool use, and peer exchange in a low-stakes environment so that both competence and confidence grow together.
What would settle it
A randomized controlled trial in which teachers are assigned either to the course or to a waitlist or no-training control group, with both groups tested at identical time points, would settle the central claim; if the control group shows similar pre-post gains, the training-specific effect would be unsupported.
Extended reading notes
Core claim
The paper's central claim is that a structured, mixed-synchronous online teacher training program produces measurable growth in AI literacy and positive attitude shifts among in-service teachers. Mean AI literacy scores rose from 13.15 to 15.13 out of 30 items, with ten of fourteen competency areas improving significantly, although with small effect sizes. All five practical-use and integration attitude items improved significantly, with the largest change on 'I feel well-prepared to integrate AI technologies into my teaching' (d = 1.65). Belief items showed a more mixed pattern: teachers became slightly more likely to see AI as a threat and as a possible replacement for teachers, while also becoming more convinced that AI offers an opportunity to support students.
Load-bearing premise
The study assumes the changes it measured were caused by the training course rather than by practice on the test, by growing familiarity through repeated questionnaires, or by the kind of teachers who volunteer for an AI course in the first place.
Editorial extensions
If this is right
- Teachers who complete a structured four-week online AI course can be expected to score higher on AI literacy tests, with gains spread across most competency areas rather than limited to technical skills.
- Teachers report greater daily AI use, more use of AI for lesson materials, and stronger intention to integrate AI into teaching immediately after the course.
- Feeling prepared to integrate AI into teaching shifts more strongly than any other measured attitude, suggesting the course addresses a key psychological barrier.
- Training appears to foster a two-sided view of AI: slightly stronger perception of risks alongside a stronger belief that AI supports students.
- Because participants volunteered, the reported effects likely represent an upper-bound estimate for the broader teacher population, a limitation the paper itself acknowledges.
Reading between the lines
- The paper does not establish causation; a plausible alternative reading is that parts of the gains come from retest practice, from repeated questionnaire exposure, or from the enthusiasm of self-selected volunteers. A waitlist-control design would separate the training's effect from these time-related influences.
- The simultaneous increase in perceived threat and perceived opportunity suggests training may cultivate a more differentiated, critical view of AI rather than uniformly positive attitudes, a possibility the authors mention but do not fully explore.
- The large rise in perceived preparedness alongside only a small rise in measured knowledge hints that confidence can outpace competence, which matters for policy: teachers may feel ready to use AI before their knowledge gaps are closed.
- An education-specific AI literacy test, aligned with classroom scenarios rather than general items like 'Predictive Policing,' might show larger training effects than the general test used here, making such an instrument a natural next step.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript evaluates a four-week online teacher training course ("Artificial Intelligence in Everyday School and Classroom Practice") offered via the fobizz platform to in-service teachers in Germany. Using a one-group pre-post design, the authors report that 291 teachers completing the externally validated AI literacy test improved from 13.15 to 15.13 correct answers (p < 0.001, d = 0.37), with significant gains in 10 of 14 competency domains. Among 436 teachers responding to a self-developed eight-item attitude questionnaire, all eight items changed significantly, with effect sizes ranging from d = 0.25 to d = 1.65; the largest gains were in perceived preparedness (I8) and use of AI for lesson creation (I2). The authors conclude that structured teacher training programs effectively enhance AI literacy and foster positive attitudes toward AI in education.
Significance. The paper addresses a timely and practically important question: whether a large-scale, commercially deployed teacher training program can improve AI literacy and attitudes. Its strengths include the use of an externally validated AI literacy test, a substantial sample, a concrete description of the intervention, and analysis of both knowledge and attitude outcomes. If the causal claims could be supported, the study would inform professional development policy. However, the one-group pre-post design and unvalidated attitude measure mean the evidence is suggestive at best; the current contribution lies in documenting pre-post changes and providing a model for future, better-controlled evaluations.
major comments (3)
- [§IV.B, §V.A, §V.B] The study uses a one-group pre-post design with no control or comparison condition. The observed improvements, especially the self-reported attitude changes (d up to 1.65 in Table 3), are indistinguishable from test-retest practice on the literacy test, demand characteristics, and regression to the mean, particularly given voluntary self-selection. The abstract's claim that "Structured teacher training programs effectively enhance AI literacy and foster positive attitudes" is a causal assertion not supported by the design. Please reframe all conclusions as pre-post changes and explicitly acknowledge the absence of a counterfactual in the Limitations section, which currently mentions self-selection but not the missing control condition.
- [§IV.A.d, §V.B] The attitude measure is a self-developed eight-item questionnaire with no psychometric validation reported: there is no evidence of reliability, factor structure, or item analysis. The analyses are conducted at the item level with 22 separate Wilcoxon signed-rank tests and no correction for multiple comparisons; at α = 0.05, at least one spurious significant result is likely. The authors should provide reliability or validity information, adjust for multiple comparisons, or explicitly label the attitude outcomes as exploratory. They should also report effect sizes for non-significant results in Table 2 rather than omitting them.
- [§V.B, §VI (RQ3)] Items I4 ("AI in schools is more of a threat than an opportunity") and I7 ("AI will replace teachers in the future") increased significantly after the training, which are negative-attitude changes, contradicting the abstract's statement that "all attitude items... demonstrated significant positive changes." The interpretation that these increases reflect an "enhanced capacity to accurately assess the risks and hazards of AI" is post hoc and unsupported by any data on participants' reasoning. This undermines the conclusion that the training uniformly fostered positive attitudes. The paper should present these results as mixed or nuanced and discuss alternative explanations such as increased awareness of risks or response shifts.
minor comments (6)
- [§IV.A.b] The statement "No personal data was collected" is inconsistent with the reporting of gender, age, and school type; please reword to indicate that no identifying personal data was collected.
- [§IV.A.d] Footnote 2 says "Item number 7 was eliminated during the validation process by Hornberger et al." but it is unclear whether the 29-item test already excludes this item; please clarify.
- [Table 2] Effect sizes are reported only for significant results; however, the definition of Cohen's d and its variance are not given, and reporting the effect size for non-significant comparisons would aid interpretation.
- [Figures 3 and 4] Please specify what the error bars represent (e.g., standard deviation, standard error) and whether individual item distributions are shown, as this is essential for interpreting the large effect sizes.
- [§IV.B] The use of Cohen's d with non-parametric Wilcoxon tests could be complemented by a non-parametric effect size (e.g., matched rank-biserial correlation) to ensure robustness of the reported effect magnitudes.
- [§III] Research questions RQ1–RQ3 are phrased causally ("How does participation... affect..."), but the design is observational and pre-post; consider rephrasing to "Are there pre-post differences..." or "Is participation associated with...".
Circularity Check
No significant circularity: the study is an empirical pre-post intervention evaluation whose outcome measures and training content are independent of the conclusions.
full rationale
This paper is an empirical intervention study, not a derivation, so the circularity patterns for claimed predictions or first-principles results do not apply. The AI literacy outcome was measured with the externally validated Hornberger et al. (2023) test, and the training program is a concrete, described course with specified formats and assignments. The attitude items were developed by the authors, but they are transparently listed as eight separate Likert items measuring usage, integration intentions, and beliefs; they are not defined in terms of the study's conclusions, and the conclusion that attitudes improved follows from the observed pre-post differences rather than from the construction of the items. The only self-citation, Lademann et al. (2025), concerns AI chatbots in physics learning and is used as background support, not as the basis for the present claim. The absence of a control group and the possibility of test-retest or demand effects are validity limitations, not circularity: they weaken causal attribution but do not make any measured quantity equal to the conclusion by definition. Because the central empirical result is not derived from its own inputs, the appropriate circularity score is 0.
Assumptions & free parameters
assumptions (3)
- domain assumption The Hornberger et al. AI literacy test is valid and reliable for in-service teachers.
- domain assumption The eight self-developed attitude items are valid and reliable measures of the intended constructs.
- domain assumption Teachers answered truthfully and the pre-post changes reflect the intervention, not testing effects.
Cite this review
Pith. "Pith review of Teacher training in the age of AI: Impact on AI Literacy and Teachers' Attitudes." pith.science (2026). https://pith.science/paper/AZZRX7Z5
@misc{pith2026250703011,
author = {Pith},
title = {Pith review of: Teacher training in the age of AI: Impact on AI Literacy and Teachers' Attitudes},
year = {2026},
howpublished = {\url{https://pith.science/paper/AZZRX7Z5}},
note = {Machine review of arXiv:2507.03011}
}
read the original abstract
The rapid integration of artificial intelligence (AI) in education requires teachers to develop AI competencies while preparing students for a society influenced by AI. This study evaluates the impact of an online teacher training program on German in-service teachers' AI literacy, usage behaviors, and attitudes toward AI. A pre-post design study was conducted with teachers (N1 = 291 for AI literacy, N2 = 436 for attitude assessment) participating in the course. The program combined synchronous and asynchronous learning formats, including webinars, self-paced modules, and practical projects. The participants exhibited notable improvements across all domains: AI literacy scores increased significantly, and all attitude items regarding AI usage and integration demonstrated significant positive changes. Teachers reported increased confidence in AI integration. Structured teacher training programs effectively enhance AI literacy and foster positive attitudes toward AI in education.
Reference graph
Works this paper leans on
-
[1]
Adiguzel, T., Kaya, M. H., & Cansu, F. K. (2023). Revolutionizing education with AI: Exploring the transformative potential of ChatGPT. Contemporary Educational Technology, 15(3), ep429. https://doi.org/10.30935/cedtech/13152 Akgun, S., & Greenhow, C. (2022). Artificial intelligence in education: Addressing ethical challenges in K -12 settings. AI and Eth...
-
[4]
https://doi.org/10.3390/educsci13040410 Long, D., & Magerko, B. (2020). What is AI Literacy? Competencies and Design Considerations. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems , 1 –16. https://doi.org/10.1145/3313831.3376727 Moura, A., & Carvalho, A. A. A. (2024). Teachers’ perceptions of the use of artificial intelligenc...
arXiv 2020
-
[7]
https://doi.org/10.53761/1.20.7.01 Dolezal, D., Motschnig, R., & Ambros, R. (2025). Pre-Service Teachers’ Digital Competence: A Call for Action. Education Sciences, 15(2),
-
[9]
https://doi.org/10.3389/frai.2021.654924 Wu, R., & Yu, Z. (2024). Do AI chatbots improve students learning outcomes? Evidence from a meta -analysis. British Journal of Educational Technology, 55(1), 10–33. https://doi.org/10.1111/bjet.13334 Yang, T.-C., & Chen, J.-H. (2023). Pre-service teachers’ perceptions and intentions regarding the use of chatbots th...
arXiv 2024
-
[14]
https://doi.org/10.3390/su14148636 Lademann, J., Henze, J., & Becker -Genschow, S. (2025). Augme nting learning environments using AI custom chatbots: Effects on learning performance, cognitive load, and affective variables. Physical Review Physics Education Research, 21(1), 010147. https://doi.org/10.1103/PhysRevPhysEducRes.21.010147 Laupichler, M., Aste...
arXiv 2025
-
[22]
https://doi.org/10.1186/s41039-017-0062-8 R Core Team. (2024). R: A Language and Environment for Statistical Computing (Version 4.4.0) [Computer software]. https://www.R-project.org Revelle, W. (2025). psych: Procedures for Psychological, Psychometric, and Personality Re search (p. 2.5.3) [Dataset]. https://doi.org/10.32614/CRAN.package.psych Rienties, B....
-
[45]
Crawford, J., Vallis, C., Yang, J., Fitzgerald, R., O’Dea, C., & Cowling, M. (2023). Editorial: Artificial Intelligence is Awesome, but Good Teaching Should Alway s Come First. Journal of University Teaching and Learning Practice, 20(7), Article
work page 2023
-
[46]
https://doi.org/10.1145/3589649 Zheng, Z., Sun, Y., Song, X., Zhu, H., & Xiong, H. (2023). Generative Learning Plan Recommendation for Employees: A Performance-aware Reinforcement Learning Approach. Proceedings of the 17th ACM Conference on Recommender Systems, 443–454. https://doi.org/10.1145/3604915.3608795
arXiv 2023
Show all 10 references
-
[160]
Can we just Please slow it all Down?
https://doi.org/10.3390/educsci15020160 Dunnigan, J., Henriksen, D., Mishra, P., & Lake, R. (2023). “Can we just Please slow it all Down?” School Leaders Take on ChatGPT. TechTrends, 67(6), 878–884. https://doi.org/10.1007/s11528-023-00914-1 Fauzi, F., Tuhuteru, L., Sampe, F.,...
2023
-
[2023]
(Vol. 17, pp. 140 –150). Atlantis Press International BV. https://doi.org/10.2991/978-94- 6463-380-1_13 Nazaretsky, T., Ariely, M., Cukurova, M., & Alexandron, G. (2022). Teachers’ trust in AI-powered educational technology and a professional development program to improve it....
2022
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.