pith. sign in

arxiv: 2512.10785 · v2 · submitted 2025-12-11 · ⚛️ physics.ed-ph · cs.AI· cs.HC

Developing and Evaluating a Large Language Model-Based Automated Feedback System Grounded in Evidence-Centered Design for Supporting Physics Problem Solving

Holger Maus , Paul Tschisgale , Fabian Kieser , Stefan Petersen , Peter Wulff This is my paper

Pith reviewed 2026-05-16 23:07 UTC · model grok-4.3

classification ⚛️ physics.ed-ph cs.AIcs.HC
keywords LLM feedbackphysics problem solvingevidence-centered designautomated feedbackAI in educationPhysics Olympiadstudent perceptionsfeedback errors
0
0 comments X

The pith

An evidence-centered LLM feedback system for physics problems is rated useful and accurate by students even though it contains errors in 20 percent of cases that often go unnoticed.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper designs an automated feedback system that uses large language models to assist students with physics problem solving. The system is structured according to evidence-centered design principles so that feedback targets specific aspects of student reasoning. Evaluation in the German Physics Olympiad shows that participants generally view the feedback as useful and highly accurate. An in-depth review, however, finds factual or conceptual errors in 20 percent of the responses, and these errors frequently escape student notice. The work therefore examines both the immediate practicality and the longer-term risks of deploying such systems in expert domains.

Core claim

The central claim is that an LLM-based feedback system grounded in evidence-centered design can generate feedback for advanced physics problem solving that students perceive as useful and highly accurate, although a subsequent analysis reveals errors in 20 percent of cases that students typically fail to detect. The study discusses the risks of uncritical reliance on this feedback and sketches directions for creating more adaptive and reliable future versions.

What carries the argument

Evidence-centered design (ECD) framework that structures the LLM prompts and feedback generation around observable evidence of student understanding in physics problem solving.

If this is right

  • Students may accept incorrect physics explanations without realizing it, leading to persistent misconceptions.
  • Uncritical reliance on LLM feedback carries measurable risks in domains that require advanced expertise.
  • Grounding the system in evidence-centered design improves alignment between feedback and learning goals but does not eliminate errors.
  • Future LLM feedback systems will need explicit mechanisms for error detection and greater adaptivity to student responses.
  • Scaling such systems to olympiad-level physics problems is feasible with current models but requires ongoing quality checks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Adding independent expert review or human-AI hybrid loops could reduce the rate of undetected errors before deployment.
  • The same ECD-grounded approach might transfer to other STEM subjects that rely on multi-step problem solving.
  • Measuring actual learning gains over time, rather than immediate perception ratings, would give a clearer test of the system's educational value.
  • Broader use could change how olympiad training and classroom problem sets are supported, provided reliability thresholds are first established.

Load-bearing premise

Student self-reports of usefulness and accuracy, together with the authors' error audit, provide a sufficient measure of feedback quality without independent expert verification or comparison to human tutor performance.

What would settle it

A side-by-side experiment in which the same set of physics problems is solved by matched student groups receiving either the LLM feedback or human-tutor feedback, followed by measurement of differences in subsequent problem-solving accuracy and conceptual understanding.

Figures

Figures reproduced from arXiv: 2512.10785 by Fabian Kieser, Holger Maus, Paul Tschisgale, Peter Wulff, Stefan Petersen.

Figure 1
Figure 1. Figure 1: Idealized phases of the problem-solving process of [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Simplified representation of evidence-centered design [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Screenshot of the feedback system’s interface displaying one of the integrated physics problems. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Perceived usefulness (M = 3.6, SD = 1.7) and accuracy (M = 4.4, SD = 1.0) of the LLM-generated feedback rated on 5-point Likert scales from N = 64 student ratings. In contrast, n = 12 students criticized that the feedback was not sufficiently adaptive with regard to their individual solutions. Specifically, a 3/5 rating came with the comment, ”My solution would have worked on the first try, but a more comp… view at source ↗
read the original abstract

Generative AI offers new opportunities for individualized and adaptive learning, e.g., through large language model (LLM)-based feedback systems. While LLMs can produce effective feedback for relatively straightforward conceptual tasks, delivering high-quality feedback for tasks that require advanced domain expertise, such as physics problem solving, remains a substantial challenge. This study presents the design of an LLM-based feedback system for physics problem solving grounded in evidence-centered design (ECD) and evaluates its performance within the German Physics Olympiad. Participants assessed the usefulness and accuracy of the generated feedback, which was generally perceived as useful and highly accurate. However, an in-depth analysis revealed that the feedback contained errors in 20% of cases; errors that often went unnoticed by the students. We discuss the risks associated with uncritical reliance on LLM-based feedback and outline potential directions for generating more adaptive and reliable LLM-based feedback in the future.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript presents the design of an LLM-based automated feedback system for physics problem solving grounded in evidence-centered design (ECD) and its evaluation with participants from the German Physics Olympiad. Students rated the generated feedback as generally useful and highly accurate, but an in-depth analysis by the authors identified errors in 20% of cases that students often failed to notice; the paper discusses associated risks and outlines directions for more reliable future systems.

Significance. If the empirical findings hold after improved validation, the work offers timely evidence on both the promise and the pitfalls of LLM feedback for advanced physics tasks, contributing to physics education research by demonstrating an ECD-grounded implementation and highlighting unnoticed errors as a practical concern for adaptive learning tools.

major comments (2)
  1. [Results / In-depth analysis subsection] The central claim of a 20% error rate (with errors often unnoticed) rests solely on the authors' in-depth coding plus student self-reports; no independent expert re-coding, inter-rater reliability statistics, or external validation of error presence/severity is reported. This directly affects the reliability of the risk discussion and the cautionary conclusion.
  2. [Evaluation methods] No parallel human-tutor feedback was collected on the identical Olympiad problems, so the absolute 20% error rate and any relative advantage of the LLM system remain unanchored against a domain-expert baseline. This is load-bearing for claims about usefulness and accuracy in a field where subtle misconceptions are common.
minor comments (2)
  1. [Abstract] Clarify the exact sample size, number of problems, and participant demographics in the abstract and methods to allow readers to assess generalizability.
  2. [System design section] Provide more detail on how ECD components (e.g., evidence models, task models) were translated into specific LLM prompts or system architecture.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed comments. We address each major comment below, indicating planned revisions where appropriate to strengthen the manuscript's claims and transparency.

read point-by-point responses

  1. Referee: [Results / In-depth analysis subsection] The central claim of a 20% error rate (with errors often unnoticed) rests solely on the authors' in-depth coding plus student self-reports; no independent expert re-coding, inter-rater reliability statistics, or external validation of error presence/severity is reported. This directly affects the reliability of the risk discussion and the cautionary conclusion.

    Authors: We agree that reporting inter-rater reliability would increase confidence in the error-rate finding. In the revised manuscript we will add a second independent domain expert who will re-code a random subset (approximately 30%) of the feedback instances. We will report agreement statistics (Cohen's kappa) along with a more detailed description of the coding protocol used to identify errors and their severity. This addresses the concern directly without altering the original 20% figure. revision: yes

  2. Referee: [Evaluation methods] No parallel human-tutor feedback was collected on the identical Olympiad problems, so the absolute 20% error rate and any relative advantage of the LLM system remain unanchored against a domain-expert baseline. This is load-bearing for claims about usefulness and accuracy in a field where subtle misconceptions are common.

    Authors: The manuscript's core claims concern the LLM system's standalone performance: student ratings of usefulness and accuracy, plus the authors' expert identification of errors that students frequently overlooked. These are absolute measures grounded in the ECD framework and the specific Olympiad problems. A human-tutor baseline would provide useful context for future work but is not required to support the reported student perceptions or the cautionary discussion of unnoticed errors. We will expand the limitations and future-work sections to explicitly acknowledge the absence of a human baseline and to recommend such comparisons in subsequent studies. revision: partial

Circularity Check

0 steps flagged

No circularity: empirical evaluation of ECD-grounded LLM feedback

full rationale

The paper describes the design of an LLM feedback system grounded in the established Evidence-Centered Design framework and reports an empirical evaluation based on participant ratings of usefulness/accuracy plus the authors' own coding of feedback instances for errors (yielding the 20% figure). No mathematical derivations, equations, fitted parameters presented as predictions, or self-referential definitions appear. Claims rest on direct data collection and coding rather than any chain that reduces to inputs by construction. Self-citations, if present, are not load-bearing for any derivation. This is a standard empirical study whose central results are independently falsifiable via external re-coding or human-tutor baselines.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

This is an applied empirical design-and-evaluation study. No mathematical derivations, fitted parameters, or new theoretical entities are introduced; the work relies on standard assumptions of educational measurement and LLM capability.

pith-pipeline@v0.9.0 · 5474 in / 1138 out tokens · 28203 ms · 2026-05-16T23:07:20.342488+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

52 extracted references · 52 canonical work pages · 1 internal anchor

  1. [1]

    Using a Chatbot to Provide Formative Feedback: A Longitudinal Study of Intrinsic Motivation, Cognitive Load, and Learning Performance,

    J. Yin, T.-T. Goh, and Y . Hu, “Using a Chatbot to Provide Formative Feedback: A Longitudinal Study of Intrinsic Motivation, Cognitive Load, and Learning Performance,”IEEE Transactions on Learning Technologies, vol. 17, pp. 1378–1389, 2024

    work page 2024

  2. [2]

    Grading Explanations of Problem-Solving Process and Generating Feedback Using Large Language Models at Human- Level Accuracy,

    Z. Chen and T. Wan, “Grading Explanations of Problem-Solving Process and Generating Feedback Using Large Language Models at Human- Level Accuracy,”Physical Review Physics Education Research, vol. 21, no. 1, p. 010126, Mar. 2025

    work page 2025

  3. [3]

    Science Education in the Age of Artificial Intelligence: Opportunities, Challenges, and Research,

    M. H. M. Cheng and Z. H. Wan, “Science Education in the Age of Artificial Intelligence: Opportunities, Challenges, and Research,”IEEE Transactions on Learning Technologies, vol. 18, pp. 635–638, 2025

    work page 2025

  4. [4]

    J. D. Bransford, A. L. Brown, and R. R. Cocking,How People Learn. Washington, DC: National Academy Press, 2000, vol. 11

    work page 2000

  5. [5]

    Teaching Discipline- Based Problem Solving,

    R. F. Frey, C. J. Brame, A. Fink, and P. P. Lemons, “Teaching Discipline- Based Problem Solving,”CBE—Life Sciences Education, vol. 21, no. 2, Jun. 2022

    work page 2022

  6. [6]

    Scientific study of expert levels of performance: General implications for optimal learning and creativity,

    K. A. Ericsson, “Scientific study of expert levels of performance: General implications for optimal learning and creativity,”High Ability Studies, no. 9, pp. 75–110, 1998

    work page 1998

  7. [7]

    Exploring the Develop- ment of Conceptual Understanding through Structured Problem-solving in Physics,

    E. Gaigher, J. M. Rogan, and M. W. H. Braun, “Exploring the Develop- ment of Conceptual Understanding through Structured Problem-solving in Physics,”International Journal of Science Education, vol. 29, no. 9, pp. 1089–1110, Jul. 2007

    work page 2007

  8. [8]

    A Brief Introduction to Evidence-Centered Design,

    R. J. Mislevy, R. G. Almond, and J. F. Lukas, “A Brief Introduction to Evidence-Centered Design,”ETS Research Report Series, vol. 2003, no. 1, Jun. 2003

    work page 2003

  9. [9]

    Summerfield,These strange new minds: How AI learned to talk and what it means

    C. Summerfield,These strange new minds: How AI learned to talk and what it means. London: Penguin Viking, 2025

    work page 2025

  10. [10]

    Toward a Unified Theory of Problem Solving: A View from Biology,

    M. U. Smith, “Toward a Unified Theory of Problem Solving: A View from Biology,” inAnnual Meeting of the American Educational Research Association, New Orleans, LA, Apr. 1988

    work page 1988

  11. [11]

    Field research on complex decision-making processes—the phase theorem,

    E. Witte, “Field research on complex decision-making processes—the phase theorem,”International Studies of Management & Organization, vol. 2, no. 2, pp. 156–182, 1972

    work page 1972

  12. [12]

    Types and Qualities of Knowledge and their Relations to Problem Solving in Physics,

    G. Friege and G. Lind, “Types and Qualities of Knowledge and their Relations to Problem Solving in Physics,”International Journal of Science and Mathematics Education, vol. 4, no. 3, pp. 437–465, Nov. 2006

    work page 2006

  13. [13]

    Exploring the sequential structure of students’ physics problem-solving approaches using process mining and sequence analysis,

    P. Tschisgale, M. Kubsch, P. Wulff, S. Petersen, and K. Neumann, “Exploring the sequential structure of students’ physics problem-solving approaches using process mining and sequence analysis,”Physical Review Physics Education Research, vol. 21, no. 1, p. 010111, Jan. 2025

    work page 2025

  14. [14]

    Using Qualitative Problem-solving Strategies to Highlight the Role of Conceptual Knowl- edge in Solving Problems,

    W. J. Leonard, R. J. Dufresne, and J. P. Mestre, “Using Qualitative Problem-solving Strategies to Highlight the Role of Conceptual Knowl- edge in Solving Problems,”American Journal of Physics, vol. 64, no. 12, pp. 1495–1503, Dec. 1996, publisher: American Association of Physics Teachers

    work page 1996

  15. [15]

    The mathematical universe,

    M. Tegmark, “The mathematical universe,”Foundations of Physics, vol. 38, no. 2, pp. 101–150, 2008

    work page 2008

  16. [16]

    Students do not overcome conceptual difficulties after solving 1000 traditional problems,

    E. Kim and S.-J. Pak, “Students do not overcome conceptual difficulties after solving 1000 traditional problems,”American Journal of Physics, vol. 70, no. 7, pp. 759–765, 2002

    work page 2002

  17. [17]

    The Power of Feedback,

    J. Hattie and H. Timperley, “The Power of Feedback,”Review of Educational Research, vol. 77, no. 1, pp. 81–112, Mar. 2007

    work page 2007

  18. [18]

    Assessing Student Written Problem Solutions: A Problem-Solving Rubric with Application to Introductory Physics,

    J. L. Docktor, J. Dornfeld, E. Frodermann, K. Heller, L. Hsu, K. A. Jackson, A. Mason, Q. X. Ryan, and J. Yang, “Assessing Student Written Problem Solutions: A Problem-Solving Rubric with Application to Introductory Physics,”Physical Review Physics Education Research, vol. 12, no. 1, May 2016

    work page 2016

  19. [19]

    Comparison of Machine Learning Performance Using Analytic and Holistic Coding Approaches across Constructed Response Assessments Aligned to a Science Learn- ing Progression,

    L. N. Jescovitch, E. E. Scott, J. A. Cerchiara, J. Merrill, M. Urban- Lurain, J. H. Doherty, and K. C. Haudek, “Comparison of Machine Learning Performance Using Analytic and Holistic Coding Approaches across Constructed Response Assessments Aligned to a Science Learn- ing Progression,”Journal of Science Education and Technology, vol. 30, no. 2, pp. 150–16...

    work page 2021

  20. [20]

    Assessing Student Teachers’ Reflective Writing through Quantitative Content Analysis,

    E. Poldner, M. Van Der Schaaf, P. R.-J. Simons, J. Van Tartwijk, and G. Wijngaards, “Assessing Student Teachers’ Reflective Writing through Quantitative Content Analysis,”European Journal of Teacher Education, vol. 37, no. 3, pp. 348–373, Jul. 2014

    work page 2014

  21. [21]

    Prompt engineering as a new 21st century skill,

    D. Federiakin, D. Molerov, O. Zlatkin-Troitschanskaia, and A. Maur, “Prompt engineering as a new 21st century skill,”Frontiers in Education, vol. 9, Nov. 2024

    work page 2024

  22. [22]

    Using ChatGPT for Teaching Physics,

    K. E. Avila, S. Steinert, S. Ruzika, J. Kuhn, and S. K ¨uchemann, “Using ChatGPT for Teaching Physics,”The Physics Teacher, vol. 62, no. 6, pp. 536–537, Sep. 2024

    work page 2024

  23. [23]

    Empowering Instructors with AI: Evalu- ating the Impact of an AI-driven Feedback Tool in Learning Analytics,

    C. Xavier, L. Rodrigues, N. Costa, R. Neto, G. Alves, T. P. Falc ˜ao, D. Gaˇsevi´c, and R. F. Mello, “Empowering Instructors with AI: Evalu- ating the Impact of an AI-driven Feedback Tool in Learning Analytics,” IEEE Transactions on Learning Technologies, vol. 18, pp. 498–512, 2025

    work page 2025

  24. [24]

    NotebookLM: An LLM with RAG for active learning and collaborative tutoring,

    E. Tufino, “NotebookLM: An LLM with RAG for active learning and collaborative tutoring,” 2025

    work page 2025

  25. [25]

    Exploring generative AI assisted feedback writing for students’ written responses to a physics conceptual question with prompt engineering and few-shot learning,

    T. Wan and Z. Chen, “Exploring generative AI assisted feedback writing for students’ written responses to a physics conceptual question with prompt engineering and few-shot learning,”Physical Review Physics Education Research, vol. 20, no. 1, p. 010152, Jun. 2024

    work page 2024

  26. [26]

    The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: Insights from a meta-analysis,

    J. Wang and W. Fan, “The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: Insights from a meta-analysis,”Humanities and Social Sciences Communications, vol. 12, no. 1, p. 621, May 2025

    work page 2025

  27. [27]

    Examining the effect of artificial intelligence in relation to students’ academic achievement: A meta- analysis,

    L. Dong, X. Tang, and X. Wang, “Examining the effect of artificial intelligence in relation to students’ academic achievement: A meta- analysis,”Computers and Education: Artificial Intelligence, vol. 8, p. 100400, Jun. 2025

    work page 2025

  28. [28]

    AI tutoring outperforms in-class active learning: An RCT introducing a novel research-based design in an authentic educational setting,

    G. Kestin, K. Miller, A. Klales, T. Milbourne, and G. Ponti, “AI tutoring outperforms in-class active learning: An RCT introducing a novel research-based design in an authentic educational setting,”Scientific Reports, vol. 15, no. 1, p. 17458, Jun. 2025

    work page 2025

  29. [29]

    Beware of Metacognitive Laziness: Effects of Generative Artificial Intelligence on Learning Motivation, Processes, and Perfor- mance,

    Y . Fan, L. Tang, H. Le, K. Shen, S. Tan, Y . Zhao, Y . Shen, X. Li, and D. Ga ˇsevi´c, “Beware of Metacognitive Laziness: Effects of Generative Artificial Intelligence on Learning Motivation, Processes, and Perfor- mance,”British Journal of Educational Technology, vol. 56, no. 2, pp. 489–530, Mar. 2025

    work page 2025

  30. [30]

    Your Brain on ChatGPT: Accumulation of Cognitive Debt When Using an AI Assistant for Essay Writing Task,

    N. Kosmyna, E. Hauptmann, Y . T. Yuan, J. Situ, X.-H. Liao, A. V . Beresnitzky, I. Braunstein, and P. Maes, “Your Brain on ChatGPT: Accumulation of Cognitive Debt When Using an AI Assistant for Essay Writing Task,” 2025

    work page 2025

  31. [31]

    Generative ai without guardrails can harm learning: Evidence from high school mathematics,

    H. Bastani, O. Bastani, A. Sungu, H. Ge, ¨O. Kabakcı, and R. Mariman, “Generative ai without guardrails can harm learning: Evidence from high school mathematics,”Proceedings of the National Academy of Sciences of the United States of America, vol. 122, no. 26, p. e2422633122, 2025

    work page 2025

  32. [32]

    Syco- phantic ai decreases prosocial intentions and promotes dependence,

    M. Cheng, C. Lee, P. Khadpe, S. Yu, D. Han, and D. Jurafsky, “Syco- phantic ai decreases prosocial intentions and promotes dependence,” arXiv, 2025. IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES - PREPRINT 8

    work page 2025

  33. [33]

    Retrieval augmentation reduces hallucination in conversation,

    K. Shuster, S. Poff, M. Chen, D. Kiela, and J. Weston, “Retrieval augmentation reduces hallucination in conversation,”arXiv preprint, 2021, arXiv:2104.07567. [Online]. Available: https://arxiv.org/abs/2104. 07567

    work page arXiv 2021

  34. [34]

    Unreflected Acceptance – Investigating the Negative Consequences of ChatGPT-assisted Problem Solving in Physics Education,

    L. Krupp, S. Steinert, M. Kiefer-Emmanouilidis, K. E. Avila, P. Lukow- icz, J. Kuhn, S. K ¨uchemann, and J. Karolus, “Unreflected Acceptance – Investigating the Negative Consequences of ChatGPT-assisted Problem Solving in Physics Education,” inFrontiers in Artificial Intelligence and Applications, F. Lorig, J. Tucker, A. Dahlgren Lindstr ¨om, F. Dignum, P...

    work page 2024

  35. [35]

    When the Robotic Maths Tutor is Wrong - Can Children Identify Mistakes Generated by ChatGPT?

    M. Helal, P. Holthaus, L. Wood, V . Velmurugan, G. Lakatos, S. Moros, and F. Amirabdollahian, “When the Robotic Maths Tutor is Wrong - Can Children Identify Mistakes Generated by ChatGPT?” in2024 5th International Conference on Artificial Intelligence, Robotics and Control (AIRC). Cairo, Egypt: IEEE, Apr. 2024, pp. 83–90

    work page 2024

  36. [36]

    What large language models know and what people think they know,

    M. Steyvers, H. Tejeda, A. Kumar, C. Belem, S. Karny, X. Hu, L. W. Mayer, and P. Smyth, “What large language models know and what people think they know,”Nature Machine Intelligence, vol. 7, pp. 221– 231, 2025

    work page 2025

  37. [37]

    GPT-4 Technical Report

    OpenAI, “GPT-4 technical report,” mar 2023. [Online]. Available: https://arxiv.org/abs/2303.08774

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  38. [38]

    The boiling-frog problem of physics education,

    G. Kortemeyer, “The boiling-frog problem of physics education,” aug 2025, arXiv:2508.08842v1

    work page arXiv 2025

  39. [39]

    Evaluating GPT- and Reasoning-Based Large Language Models on Physics Olympiad Problems: Surpassing Human Performance and Impli- cations for Educational Assessment,

    P. Tschisgale, H. Maus, F. Kieser, B. Kroehs, S. Petersen, and P. Wulff, “Evaluating GPT- and Reasoning-Based Large Language Models on Physics Olympiad Problems: Surpassing Human Performance and Impli- cations for Educational Assessment,”Physical Review Physics Education Research, vol. 21, no. 2, p. 020115, Aug. 2025

    work page 2025

  40. [40]

    HiPhO: How far are (M)LLMs from humans in the latest high school physics Olympiad benchmark?

    F. Yu, H. Wan, Q. Cheng, Y . Zhang, J. Chen, F. Han, Y . Wu, J. Yao, R. Hu, N. Ding, Y . Cheng, T. Chen, L. Bai, D. Zhou, Y . Luo, G. Cui, and P. Ye, “HiPhO: How far are (M)LLMs from humans in the latest high school physics Olympiad benchmark?” 2025

    work page 2025

  41. [41]

    Using AI large language models for grading in education: A hands-on test for physics,

    R. Mok, F. Akhtar, L. Clare, C. Li, J. Ida, L. Ross, and M. Campanelli, “Using AI large language models for grading in education: A hands-on test for physics,” 2024

    work page 2024

  42. [42]

    Grading assistance for a handwritten thermodynamics exam using artificial intelligence: An ex- ploratory study,

    G. Kortemeyer, J. N ¨ohl, and D. Onishchuk, “Grading assistance for a handwritten thermodynamics exam using artificial intelligence: An ex- ploratory study,”Physical Review Physics Education Research, vol. 20, no. 2, p. 020144, Nov. 2024

    work page 2024

  43. [43]

    Using large language models to probe cognitive constructs, augment data, and design instructional materials,

    F. Kieser and P. Wulff, “Using large language models to probe cognitive constructs, augment data, and design instructional materials,” inMachine Learning in Educational Sciences, M. S. Khine, Ed. Singapore: Springer Nature Singapore, 2024, pp. 293–313

    work page 2024

  44. [44]

    Could an Artificial-Intelligence agent pass an intro- ductory physics course?

    G. Kortemeyer, “Could an Artificial-Intelligence agent pass an intro- ductory physics course?”Physical Review Physics Education Research, vol. 19, no. 1, May 2023

    work page 2023

  45. [45]

    Feedback that clicks: Introductory physics students’ valued features in AI feedback generated from self- crafted and engineered prompts,

    A. Sirnoorkar and N. S. Rebello, “Feedback that clicks: Introductory physics students’ valued features in AI feedback generated from self- crafted and engineered prompts,” 2025

    work page 2025

  46. [46]

    Toward Learning Progression Analytics — Developing Learning Environments for the Automated Analysis of Learning Using Evidence Centered Design,

    M. Kubsch, B. Czinczel, J. Lossjew, T. Wyrwich, D. Bednorz, S. Bern- holt, D. Fiedler, S. Strauß, U. Cress, H. Drachsler, K. Neumann, and N. Rummel, “Toward Learning Progression Analytics — Developing Learning Environments for the Automated Analysis of Learning Using Evidence Centered Design,”Frontiers in Education, vol. 7, p. 981910, Aug. 2022

    work page 2022

  47. [47]

    Multilingual Performance of a Multimodal Artificial Intelli- gence System on Multisubject Physics Concept Inventories,

    G. Kortemeyer, M. Babayeva, G. Polverini, R. Widenhorn, and B. Gre- gorcic, “Multilingual Performance of a Multimodal Artificial Intelli- gence System on Multisubject Physics Concept Inventories,”Physical Review Physics Education Research, vol. 21, no. 2, Jul. 2025

    work page 2025

  48. [48]

    The technology acceptance model (tam): A meta-analytic structural equation modeling approach to explaining teachers’ adoption of digital technology in education,

    R. Scherer, F. Siddiq, and J. Tondeur, “The technology acceptance model (tam): A meta-analytic structural equation modeling approach to explaining teachers’ adoption of digital technology in education,” Computers & Education, vol. 128, pp. 13–35, 2019

    work page 2019

  49. [49]

    Mann-Whitney U test,

    P. E. McKnight and J. Najab, “Mann-Whitney U test,” inThe Corsini Encyclopedia of Psychology, 1st ed., I. B. Weiner and W. E. Craighead, Eds. Wiley, Jan. 2010, pp. 1–1

    work page 2010

  50. [50]

    Beyond final answers: Evaluating large language models for math tutoring,

    A. Gupta, J. Reddig, T. Cal `o, D. Weitekamp, and C. J. MacLellan, “Beyond final answers: Evaluating large language models for math tutoring,” feb 2025

    work page 2025

  51. [51]

    Student modeling approaches: A lit- erature review for the last decade,

    K. Chrysafiadi and M. Virvou, “Student modeling approaches: A lit- erature review for the last decade,”Expert Systems with Applications, vol. 40, no. 11, pp. 4715–4729, Sep. 2013. VII. BIOGRAPHYSECTION Holger Mausreceived his Master of Education in physics and mathematics from Kiel University, Germany, in 2015. He is currently teaching at a German seconda...

    work page 2013

  52. [52]

    He is currently a postdoctoral researcher at the Leibniz Institute for Science and Mathematics Education in Kiel, Germany. His research focuses on nurturing high-ability students and on using AI to improve physics learning, with an emphasis on the assessment and development of physics problem- solving abilities. He also explores the use of AI as a researc...

    work page 2019