pith. sign in

arxiv: 2512.02824 · v1 · submitted 2025-12-02 · ⚛️ physics.ed-ph

Learning Science and the Illusion of Understanding: Exploring the Effects of Integrating Learning Tasks after Explainer Videos

Madeleine H\"ornlein , Christoph Kulgemeyer This is my paper

Pith reviewed 2026-05-17 02:24 UTC · model grok-4.3

classification ⚛️ physics.ed-ph
keywords illusion of understandingexplainer videosscience learninglearning tasksphysics educationmetacognitionoverconfidenceprior knowledge
0
0 comments X

The pith

High-level learning tasks after explainer videos reduce the illusion of understanding compared to watching the video alone.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Explainer videos in science often leave learners thinking they understand more than they actually do. This paper tests whether adding learning tasks right after the video can correct that overconfidence. Two experiments with hundreds of learners compared watching a physics video by itself to watching it plus either high-level or low-level tasks. High-level tasks produced a clear drop in the illusion right away, while both kinds of tasks showed similar benefits months later. The effect was strongest for students who started with less background knowledge.

Core claim

The central claim is that high-level learning tasks significantly reduce the illusion of understanding immediately after the intervention compared to watching the video alone, with a large effect size. Over the long term both high-level and low-level tasks prove similarly effective at lowering overconfidence. Learners with lower prior content knowledge are more susceptible to the illusion, so explainer videos should not be used in isolation but paired with cognitively demanding follow-up activities.

What carries the argument

The central mechanism is the addition of high-level learning tasks after the explainer video, which requires active cognitive engagement and self-assessment to counteract the passive viewing that produces overconfidence.

If this is right

  • Explainer videos should be combined with demanding follow-up tasks rather than used alone in classrooms.
  • Long-term reduction of overconfidence occurs with either high- or low-level tasks.
  • Students who have less prior knowledge need extra support to avoid misjudging their understanding.
  • Metacognitive accuracy improves when passive video watching is followed by active processing.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same task structure might reduce overconfidence in explainer videos on topics outside physics.
  • Classroom schedules could be adjusted to always include a short high-level task immediately after video segments.
  • Longer-term studies could check whether the reduced illusion leads to better problem-solving performance weeks later.

Load-bearing premise

The chosen distinction between high-level and low-level tasks is valid and the measures of illusion of understanding are not simply reflecting how hard the tasks felt.

What would settle it

A new study using the same explainer video and measures that finds no difference in illusion scores between the video-only group and the high-level task group would falsify the immediate-effect claim.

read the original abstract

Explainer videos are increasingly used to support science learning. While prior research has demonstrated their potential, studies have also identified limitations - particularly their tendency to foster an illusion of understanding, where learners overestimate their grasp of a topic despite gaps in their actual knowledge. Pairing explainer videos with cognitively engaging elements may help mitigate this effect. This paper reports two experimental studies examining the immediate and long-term effects of learning tasks following a physics explainer video on learners' illusion of understanding. Study 1 (N = 244 learners) compared high-level learning tasks with watching the video alone. Study 2 (N = 175) compared high-level and low-level tasks. Results show that high-level learning tasks significantly reduce the illusion of understanding immediately after the intervention compared to watching the video alone (t(88) = 6.50, p < .001, d = 0.69). Over the long term, both high-and low-level tasks are similarly effective. Learners with lower prior content knowledge are more susceptible to an illusion of understanding. We conclude that explainer videos should not be used in isolation in science classrooms. To prevent misjudged understanding - particularly among students with limited prior knowledge - they should be combined with cognitively demanding follow-up activities.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript reports two experiments (N=244 and N=175) testing whether high-level learning tasks following a physics explainer video reduce learners' illusion of understanding (overestimation of knowledge) compared to video alone or low-level tasks. Study 1 finds a significant immediate reduction after high-level tasks (t(88)=6.50, p<.001, d=0.69); both task types show similar long-term effects. Learners with lower prior knowledge are more prone to the illusion. The authors conclude that explainer videos should be paired with cognitively demanding follow-up activities rather than used in isolation.

Significance. If the central claim holds after addressing potential confounds, the work contributes to science education research by providing experimental evidence on mitigating the illusion of understanding from passive video viewing. It aligns with calls for active learning and offers practical implications for classroom use of explainer videos, particularly for students with limited prior knowledge.

major comments (2)
  1. [Results (Study 1)] Results section (Study 1 comparison): the headline t(88)=6.50 result on reduced illusion of understanding does not report whether the effect survives after covarying actual post-test scores or after equating conditions on actual knowledge. High-level tasks are expected to produce larger knowledge gains; if actual performance increases while predicted performance remains similar, the smaller |predicted – actual| difference could be an artifact of differential learning rather than improved metacognitive calibration.
  2. [Methods and Results (Study 2)] Methods and Results (Study 2): the high-level vs. low-level task comparison is vulnerable to the same confound if low-level tasks produce weaker actual knowledge gains. The manuscript should report actual test scores by condition and test whether the illusion reduction remains after controlling for them.
minor comments (2)
  1. [Methods] Clarify the exact operational definition of 'illusion of understanding' (e.g., whether it is absolute difference, signed difference, or a calibrated measure) and how predicted performance was elicited.
  2. [Results] Provide effect sizes and confidence intervals for all key comparisons, including long-term outcomes.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments on our manuscript. We have addressed the concerns about potential confounds by conducting additional analyses and reporting actual knowledge scores. Below we respond point by point to the major comments.

read point-by-point responses

  1. Referee: [Results (Study 1)] Results section (Study 1 comparison): the headline t(88)=6.50 result on reduced illusion of understanding does not report whether the effect survives after covarying actual post-test scores or after equating conditions on actual knowledge. High-level tasks are expected to produce larger knowledge gains; if actual performance increases while predicted performance remains similar, the smaller |predicted – actual| difference could be an artifact of differential learning rather than improved metacognitive calibration.

    Authors: We agree that distinguishing improved actual learning from better metacognitive calibration is important. In the revised manuscript we now report mean actual post-test scores by condition for Study 1. We also conducted an ANCOVA on the illusion-of-understanding difference score with actual post-test performance as covariate. The condition effect remains significant after controlling for actual knowledge, indicating the reduction is not solely an artifact of differential learning gains. These results have been added to the Results section. revision: yes

  2. Referee: [Methods and Results (Study 2)] Methods and Results (Study 2): the high-level vs. low-level task comparison is vulnerable to the same confound if low-level tasks produce weaker actual knowledge gains. The manuscript should report actual test scores by condition and test whether the illusion reduction remains after controlling for them.

    Authors: We concur that actual knowledge gains must be reported to rule out the confound. The revised manuscript now includes actual post-test scores by condition in Study 2. We additionally tested the high-level versus low-level comparison on the illusion measure after covarying actual performance; the difference remains reliable. These analyses and descriptive statistics have been incorporated into the Methods and Results sections. revision: yes

Circularity Check

0 steps flagged

No circularity: claims rest on independent experimental data

full rationale

The paper reports two new experiments (N=244 and N=175) that collect fresh participant data on illusion of understanding after explainer videos with or without follow-up tasks. Illusion of understanding is operationalized as the standard discrepancy between learners' predicted and actual post-test performance; high- versus low-level tasks are defined by task design features rather than by the outcome measure itself. All reported statistics (t-tests, effect sizes, p-values) are direct comparisons of these independently measured variables. No equations, fitted parameters, or self-citations are invoked to derive the headline result; the central finding therefore does not reduce to its own inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on empirical experimental data and standard statistical assumptions rather than additional free parameters, invented entities, or non-standard axioms.

axioms (1)
  • standard math Standard assumptions for independent-samples t-tests and Cohen's d hold (normality, independence).
    The reported t(88) = 6.50 and d = 0.69 rely on these conventional statistical conditions.

pith-pipeline@v0.9.0 · 5528 in / 1138 out tokens · 31923 ms · 2026-05-17T02:24:09.718251+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

19 extracted references · 19 canonical work pages

  1. [1]

    Research Questions and Hypotheses

    compared high-level and low-level tasks. Results show that high-level learning tasks significantly reduce the illusion of understanding immediately after the intervention compared to watching the video alone (t(88) = 6.50, p < .001, d = 0.69). Over the long term, both high- and low-level tasks are similarly effective. Learners with lower prior content kno...

    work page 2021

  2. [2]

    Illusion of ILLUSION OF UNDERSTANDING AND LEARNING TASKS 7 understanding

    and multimedia learning (Mayer, 2001). She recommends, among other things, that elements that promote cognitive activation can positively influence learning with videos (Brame, 2016). A number of factors have been found to enhance the quality of explanations including clear structure, adaptation to the learner's needs, tools for adaptation, concise explan...

    work page 2001

  3. [3]

    monitoring and self-regulation of one’s own cognitive abilities

    The learner’s assessment of their own understanding can initially be described neutrally as the belief of understanding. An illusion of understanding arises only when this belief diverges from one’s actual understanding. For example, previous research has shown that presenting common misconceptions as scientifically correct explanations can lead to a high...

    work page 2023

  4. [4]

    In a nutshell, an illusion of understanding is not merely a judgment of understanding but is closely tied to how the explanation is perceived

    and adds an important perspective to research on metacognition. In a nutshell, an illusion of understanding is not merely a judgment of understanding but is closely tied to how the explanation is perceived. Instead of representing a general self-evaluation of knowledge, it combines the perception of the instructional situation with the perception of its c...

    work page 2018

  5. [5]

    illusion of explanatory depth

    describes the tendency for learners to overestimate their abilities (see also Bol and Hacker (2012) for similar findings in mathematics). Those who perform poorly are especially challenged by the metacognitive task of evaluating their performance (e.g., Dunning, 2011). The overestimation of one’s own understanding may also be associated with a phenomenon ...

    work page 2012

  6. [6]

    offers a potential explanation. In accordance with this hypothesis, learners – at least K-12 learners – often consume social media and online videos during their leisure time with the intention of achieving immediate gratification (Salmerón, Sampietro & ILLUSION OF UNDERSTANDING AND LEARNING TASKS 10 Delgado, 2020). This may contribute to a superficial pr...

    work page 2020

  7. [7]

    Materials: Explainer video and learning tasks For the use in both studies, we created an explainer video to introduce the concept of energy in a qualitative manner1

    and prior education in physics (advanced level: 41). Materials: Explainer video and learning tasks For the use in both studies, we created an explainer video to introduce the concept of energy in a qualitative manner1. All participants across all groups watched the video during the intervention; therefore, we do not expect the video's characteristics to h...

    work page 2012

  8. [8]

    This conceptualization is consistent with curricula in Germany (e.g., KMK, 2020)

    The total amount of energy in a system remains constant; energy is neither created nor destroyed (conservation of energy). This conceptualization is consistent with curricula in Germany (e.g., KMK, 2020). The video has a duration of 7 minutes and 30 seconds, making it comparable to typical YouTube explainer videos (Wolf, 2015). It follows a validated fram...

    work page 2020

  9. [9]

    #$%=0.78; 𝛼&'()=0.77; 𝛼*%++%,-./=0.81) and study 2 (𝛼!

    Overview of the learning tasks. The left panel shows the low-level task, the right panel depicts the high-level task. The tasks were originally developed in German language. To investigate the effect of learning tasks on the illusion of understanding when using explainer videos, two distinct learning tasks were developed (figure 2). One task was designed ...

    work page 2023

  10. [10]

    (2017) (cited from: Neumann et al., 2012), in our study only complexity information A was presented

    Sample item for concept knowledge from Viering et al. (2017) (cited from: Neumann et al., 2012), in our study only complexity information A was presented. Overview of the studies, procedure, and analysis Both studies followed a similar experimental design (not quasi-experimental), in which learners were randomly assigned to groups, with one key difference...

    work page 2017

  11. [11]

    Descriptive statistics of study variables (study 1). High-level Group M (SD) (N) Control Group M (SD) (N) Overall M (SD) (N) Range Academic self-concept physics 𝛼=0.91 2.24 (0.83) (68) 2.33 (0.80) (73) 2.18 (0.79) (190) 1-5 Belief of Understanding (post I) 𝛼=0.78 3.60 (0.38) (88) 3.59 (0.40) (96) 3.60 (0.39) (184) 1-5 Belief of Understanding (post II) 𝛼=0...

    work page 2003

  12. [12]

    Illustrating the ANCOVA: Comparison of the belief of understanding scale for the low-level task group (left) and high-level task group (right) between after the video (before the task) and after the task. However, no significant correlation was found between belief of understanding and conceptual knowledge after the task in either the low-level (𝑟=0.17, 𝑝...

    work page doi:10.1007/s10212-010-0049-y 2008

  13. [13]

    Bloom, B. S. (1956). The cognitive domain. David McKay. Bol, L., & Hacker, D. J. (2012). Calibration research: Where do we go from here? Frontiers in Psychology, 3, 1–6. Brame, C. J. (2016). Effective educational videos: Principles and guidelines for maximizing student learning from video content. CBE-Life Sciences Education, 15(4), 1–6. https://doi.org/1...

    work page doi:10.1187/cbe.16-03-0125 1956

  14. [14]

    1080/ 09500 69030 5016 Dunning, D. (2011). The Dunning-Kruger effect: On being ingnorant of one’s own ignorance. Advances in experimental social psychology (Vol. 44, pp. 247-296). Academic Press Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why people fail to recognize their own competence. Current Directions in Psychological Science, 12, ...

    work page doi:10.1037/13274-019 2011

  15. [15]

    https://doi.org/10.1103/PhysRevPhysEducRes.20.010107 ILLUSION OF UNDERSTANDING AND LEARNING TASKS 38 Lingel, K., Lenhart, J., & Schneider, W. (2019). Metacognition in mathematics: Do different metacognitive monitoring measures make a difference? ZDM, 51(4), 587–600. https://doi.org/10.1007/s11858-019-01062-8 Lloyd, S. A., & Robertson, C. L. (2012). Screen...

    work page doi:10.1103/physrevphyseducres.20.010107 2019

  16. [16]

    1177/ 00986 28311 430640 Lowe (2003) Lowe, R. K. (2003). Animation and learning: Selective processing of information in dynamic graphics. Learning and Instruction, 13(2), 157–176 Mayer, R.E. (2001). Multimedia Learning, Cambridge University Press. Medienpädagogischer Forschungsverbund Südwest [MPFS] (2021). JIM Studie

    work page 2003

  17. [17]

    easy” and print is “tough

    Jugend, Information, Medien. Basisuntersuchung zum Medienumgang 12 bis 19-Jähriger [YIM Study 2021: Youth, information, media. Basic study on the media consumption of 12 to 19-year-olds.]. MPFS. Mihalca, L., Mengelkamp, C., Schnotz, W., & Paas, F. (2015). Completion problems can reduce the illusion of understanding in a computer-based learning environment...

    work page doi:10.1016/j.cedpsych.2015.01.001 2021

  18. [18]

    003 Viering, T.; Neumann, K.; Fischer, H. (2017). Energy Concept Assessment [Testinstrument: Version 1.0]. Initial application

    work page 2017

  19. [19]

    http://dx.doi.org/10.7477/113:138:21 Webb, N

    Forschungsdatenzentrum Bildung am DIPF. http://dx.doi.org/10.7477/113:138:21 Webb, N. M., Ing, M., Kersting, N., & Nemer, K. M. (2006). Help seeking in cooperative learning groups. In S. A. Karabenick & R. S. Newman (Eds.), Help seeking in academic settings: Goals, groups, and contexts (pp. 45–88). Lawrence Erlbaum Associates. ILLUSION OF UNDERSTANDING AN...

    work page doi:10.7477/113:138:21 2006