REVIEW 4 major objections 5 minor 10 references
Cosmology and general relativity (GR) in upper secondary school through new targeted teaching materials: a study on student learning and motivation
T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read A semester-long cosmology and general relativity course for non-specialist high-school students produced very large conceptual learning gains and modest motivation gains in a pre-post study of 70 students.
desk verdict A transparent implementation study with a plausible feasibility claim, but the d=2.78 learning gain is not yet credible given the graded post-test, single teacher, and author-built instrument. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the course-as-designed: a structured teaching sequence, originally built around an eight-chapter book with accompanying exercises, that deliberately anchors modern cosmology and general relativity in the classical physics students already know (Newtonian mechanics, electromagnetism, waves) and treats each advanced result through quantitative derivations that stay within upper secondary mathematics. A recurring pattern is the simplified derivation that yields the correct physical dependence and differs from the full relativistic result only by a numerical factor, for example the light-deflection angle, the Schwarzschild radius, black-hole density scaling as $\rho \propto 1/M^2$, and Hawking temperature, so the mathematics is real but the prerequisites are not. The design principle is what the paper calls 'productive forms of complexity': enough depth to preserve the physics without the formal apparatus of the theory. The evaluation mechanism is the pre-post design in which the same 68 to 70 students took a 14-item conceptual test and an affective questionnaire before and after the semester module.
What would settle it
Give the same 14-item conceptual questionnaire to a comparable group of upper secondary students who take a different physics module in the same complementary-option program, with the same graded-post-test incentive but no cosmology instruction; if that group also improves substantially, the course is not the cause of the gain. Alternatively, re-administer the test ungraded several months after a future course and check whether the gain remains in the very-large range rather than fading.
Extended reading notes
Core claim
On the paper's own terms, the discovery is a positive answer to all three research questions. RQ1: a semester-long cosmology and GR course can be developed and delivered at upper secondary level, with materials that combine conceptual explanations and accessible mathematics rather than narrative-only simplification. RQ2: students' conceptual learning improved substantially, with an overall pre-post effect of $d = 2.78$, per-item effects between 0.44 and 1.59, and shifts in common misconceptions; for example, the belief that the Big Bang was an explosion dropped from 79% to 25%, and the belief that dark energy is the energy of dark matter dropped from 41% to 3%. RQ3: affective outcomes moved positively but unevenly, with a significant small-to-medium gain in curiosity state ($d = 0.46$), a non-significant gain in self-concept ($d = 0.26$, $p = 0.09$), and no significant change in interest or perceived relevance of science. The paper also reports no or small effects of gender and class on these gains.
Load-bearing premise
The load-bearing assumption is that the large pre-to-post gain in conceptual scores reflects the course itself, rather than the fact that the post-test counted toward the semester grade, that the same teacher who helped create the materials taught every cohort, or that the pre-test's low internal consistency (Cronbach $\alpha = 0.37$) made the baseline artificially weak. With no comparison group, the design does not by itself separate the course effect from these context effects.
Editorial extensions
If this is right
- A single semester of roughly sixteen 90-minute lessons is enough for upper secondary non-specialists to reach correct conceptual answers on about three quarters of a cosmology test, up from about a third before instruction.
- The misconception shifts imply that the course's active, data-rich exercises can replace intuitive but wrong pictures, such as the Big Bang as an explosion, a static universe, or dark matter as antimatter, with the standard relativistic ones in most students.
- Because the course is built on classical physics already in the curriculum, adopting it strengthens rather than displaces the traditional syllabus; students report that the course made Newtonian mechanics and waves more meaningful.
- Curiosity about current research responds to this kind of course, while interest in further physics coursework does not automatically follow, so teachers should expect to work separately on sustaining long-term interest.
- Gender and class had no statistically significant effect on gains, so within this sample the course's benefits are not reserved for students already oriented toward physics or for any particular cohort.
Reading between the lines
- A natural extension, not tested in the paper, is to convert the 14-item instrument into a broader cosmology concept inventory for upper secondary classrooms, using the items whose pre-test misconception patterns were strongest; such an inventory would let other teachers measure their own courses against these effect sizes.
- The simplified-derivation pattern, correct physical dependence with a constant that is off by a known factor from the full theory, could transfer to other modern physics topics and offers a recipe for 'real math without prerequisites' that other curriculum designers could imitate.
- Because the post-test was graded, an ungraded delayed retest or a comparison module would clarify how much of the $d = 2.78$ gain is durable learning rather than incentive-driven response; the paper's own stated plan to involve other teachers and schools points toward such a design.
- The interview finding that students who feel they already know the basics lose interest in going deeper suggests an untested follow-up structure: optional project work on current observations might convert short-term curiosity into sustained engagement.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper reports an implementation study of a one-semester cosmology and general relativity course for non-specialist upper secondary students in a Swiss complementary option (CO). Seventy students across four cohorts (2020–2024) were taught by the same teacher. A 14-item author-written conceptual multiple-choice test and a 28-item affective questionnaire (interest, self-concept, curiosity state, perceived relevance of science) were administered as pre- and post-tests, supplemented by 15 semi-structured interviews. The paper reports a very large overall conceptual learning gain (d = 2.78), moderate positive effects on curiosity state and self-concept, and no significant change in interest or perceived relevance. It concludes that the course is feasible and can significantly improve students' knowledge of cosmology and GR.
Significance. If the reported learning gain could be credibly attributed to the instructional design, this would be an important contribution to physics education research, demonstrating that conceptually deep modern physics can be taught to non-specialists at the upper secondary level. The paper has notable strengths: the detailed description of the teaching sequence and materials, the inclusion of the full questionnaire, the item-level analysis, the interview data, and the public availability of raw data. The feasibility claim (RQ1) is reasonably supported by the rich implementation description and qualitative evidence. However, the magnitude of the conceptual learning gain (RQ2) is not yet established as an estimate of the course effect because of the one-group design, the author-instrument alignment, the grading confound, and the absence of uncertainty measures for the main effect sizes.
major comments (4)
- [§5.1.2, §5.3, Table 6] The ANCOVA results are internally contradictory. In §5.1.2 the authors report that prior physics grades significantly influenced learning outcomes (F(1, 62) = 10.90, p = .002) and that the class effect was marginally non-significant (F(3, 62) = 2.68, p = .054). In §5.3 and Table 6, however, the same conceptual-learning outcome shows no significant predictor at all, with very different F values (e.g., prior physics grade F = 0.09, p = .77). The authors must clarify which dependent variable was used in each analysis, report the full model specification, and reconcile the discrepancy in the text.
- [§4.1, §7] The study uses a one-group pre-post design with no comparison condition, and the conceptual post-test contributed to students' semester grades while the pre-test did not. This asymmetry can inflate post-test performance through differential effort and test-wiseness, so the observed d = 2.78 cannot be attributed to the course alone. The conclusion in §7 that the study 'demonstrates' an effective course is stronger than the design supports; the authors should either provide a robustness analysis (e.g., ungraded items or an internal control) or substantially temper the causal wording.
- [§4.3.1, Tables 2 and 3] The conceptual instrument was authored by the research team from their own course content, and its pre-test psychometric properties are weak: Cronbach's alpha is 0.37 and several discrimination indices are negative in the pre-test (items 10, 11, 12, 13, 14). Low pre-test reliability and floor effects can mechanically inflate gain scores and make the very large post-test gain partly a measurement artifact. The authors should report confidence intervals for the mean gain and for Cohen's d, and discuss the construct validity of the instrument beyond item alignment with the taught content.
- [§4.1, §7.1] All four cohorts were taught by the same teacher, who is also a co-author of the course materials and the study. This confounds teacher quality and implementation fidelity with the effect of the course content itself. The future-work paragraph in §7.1 acknowledges that classes from different teachers and schools are planned, but the current generalization from this sample to 'upper secondary students' in general is not yet supported. The authors should explicitly state that the results are limited to this teacher-implementer or provide evidence from independent implementations.
minor comments (5)
- [§5.1.1, §6.1.2] The text states that 'for 10 items out of 15' the wrong responses were unevenly distributed, but the questionnaire has 14 items; the number should be corrected throughout.
- [Table 3] The column headers in Table 3 are difficult to parse because the pre/post statistics are interleaved with 'P(SD) Cohen d' without clear grouping; please restructure the table so that pre-test and post-test columns are explicitly labeled.
- [Table 6] The table caption contains the typo 'predictros'; it should read 'predictors'.
- [Appendix 1] The appendix contains several unresolved 'Error! Reference source not found.' placeholders and garbled equation fragments; these must be fixed before publication.
- [References [40] and [41]] References [40] and [41] cite the same work (Levrini and Fantini, 2013) with identical details; the duplication should be removed.
Circularity Check
No circularity: the learning gain is a measured pre-post difference, and the test-design/grading issues are validity limitations rather than constructional equivalences.
full rationale
The paper is an empirical implementation study, not a derivation from first principles, so most enumerated circularity patterns (self-definitional equations, imported uniqueness, ansatz smuggled via citation, renaming a known result) do not apply. The central quantitative claim (overall Cohen's d = 2.78) is computed directly from pre/post scores on the authors' 14-item conceptual questionnaire. That instrument is admittedly authored by the research team from the course content (Section 4.3.1), and the post-test contributed to semester grades (Section 4.1) while all cohorts were taught by the same teacher. These are genuine threats to internal and external validity, not circularity: the paper does not fit a model and then rename a fitted parameter as a prediction, and no equation in the paper is equivalent by construction to the outcome. The gain is a measured difference, not a derived consequence of the measurement instrument itself. The low pre-test alpha (0.37, Table 2), negative pre-test discrimination indices (Table 3), and the grading asymmetry are confounds and measurement-quality limitations, which the authors partially acknowledge in Section 7.1 by planning classes from different teachers and schools. Self-citations to course materials ([55]-[57]) and to the affective instrument ([60]) describe the intervention and its measures; they are not used as an external proof that the intervention worked, since the pre-post design and interview data carry that evidential weight. Therefore no circular step is exhibited, and the circularity score is 0.
Assumptions & free parameters
assumptions (3)
- domain assumption Pre-post differences reflect the course rather than test familiarity, grading incentives, or the same teacher's enthusiasm.
- domain assumption The author-developed 14-item conceptual questionnaire is a valid measure of cosmology understanding despite low pre-test reliability.
- domain assumption The adapted affective scales measure the intended constructs in this population, including the unvalidated 'relevance of science' scale.
Cite this review
Pith. "Pith review of Cosmology and general relativity (GR) in upper secondary school through new targeted teaching materials: a study on student learning and motivation." pith.science (2026). https://pith.science/paper/CJHZ2A6P
@misc{pith2026241201551,
author = {Pith},
title = {Pith review of: Cosmology and general relativity (GR) in upper secondary school through new targeted teaching materials: a study on student learning and motivation},
year = {2026},
howpublished = {\url{https://pith.science/paper/CJHZ2A6P}},
note = {Machine review of arXiv:2412.01551}
}
read the original abstract
Cosmology and GR remain largely inaccessible to high-school teaching due to the advanced prerequisites to master these topics. Integrating them into upper secondary teaching is a significant challenge that remains unresolved. This contribution reports on an implementation study of a GR and cosmology course for upper secondary school students as part of an educational project launched during the centenary of GR and tested ever since for several years. The course aimed to expand students' knowledge to include current physics topics while highlighting their foundations in areas of classical physics such as Newtonian mechanics, electromagnetism, and waves. Targeted teaching and learning materials are focused on conceptual and qualitative understanding, while systematically combined with a mathematical treatment accessible at the upper secondary level, avoiding oversimplification. A key element is an active learning approach, incorporating activities and tasks such as engaging applications related to current research, reflective exercises, thought experiments, and hands-on tasks. The main research objective was to explore whether a conceptually deep and educationally effective GR and cosmology course could be successfully implemented for non-specialist upper secondary students. A pre-post study assessed both conceptual learning and affective outcomes, including interest, curiosity, self-concept, and perceived relevance of science. Results showed encouraging gains in both learning and motivation, with large to very large effect sizes for conceptual learning of core principles. Additionally, no or small effects of predictors such as gender were observed. We conclude that the integration of GR and cosmology into upper secondary physics teaching, in the form of courses and materials that are engaging, comprehensible, and impactful, is feasible.
Reference graph
Works this paper leans on
-
[1]
(B) une explosion d’un astre massif qui s’est produite il y a plus de dix milliards d’années
Le Big Bang est (A) une explosion à un instant précis dans le passé qui a donné naissance à l’univers. (B) une explosion d’un astre massif qui s’est produite il y a plus de dix milliards d’années. (C) l’instant dans le passé où le temps a commencé. (D) un état de l’univers primordial que les théories physiques actuelles ne peuvent pas décrire. (E) la phas...
-
[4]
Parmi les schémas suivants, lequel mieux décrit la trajectoire des deux rayons ?
Deux rayons de lumière parallèles passent à proximité d’un astre massif. Parmi les schémas suivants, lequel mieux décrit la trajectoire des deux rayons ?
-
[5]
(B) des particules d’antimatière
On appelle « matière noire » (A) de la matière qui n’émet pas de radiation dans le visible. (B) des particules d’antimatière. (C) de la matière qui n’interagit pas électromagnétiquement. (D) de la matière à l’intérieur des trous noirs. (E) l’espace vide entre les galaxies. 6) Un signal lumineux part d’une galaxie en direction d’une autre galaxie. Au momen...
-
[7]
(B) émis dans l’univers primordial quand les atomes neutres se sont formés
Le « fond diffus cosmologique » est le rayonnement (A) produit lors de la formation des premières galaxies dans l’univers primordial. (B) émis dans l’univers primordial quand les atomes neutres se sont formés. (C) émis lors de mécanismes qui produisent des substances radioactives dans l’univers. (D) produit par des mécanismes de fusion nucléaire à l’intér...
-
[10]
Considérez deux galaxies lointaines de forme sphérique, G1 et G2, parfaitement alignées par rapport à un observateur sur la Terre (O). Dans cette configuration (A) l’observateur (O) ne voit pas d’image de la galaxie la plus lointaine (G1). (B) l’observateur (O) peut voir une image non déformée de la galaxie la plus lointaine (G1), déplacée par rapport à s...
-
[11]
Veuillez mettre une croix sur la colonne qui cor-respond le mieux à votre opinion
Comment avez-vous trouvé le dernier module du cours OC de Physique ? Avec ce questionnaire vous pouvez donner votre avis, concernant le cours de physique à option complémentaire du module du semestre passé. Veuillez mettre une croix sur la colonne qui cor-respond le mieux à votre opinion. Avec cette affirmation, je suis… pas d’ accord du tout pas d’ accor...
-
[12]
L’observateur (A) ne pourra jamais recevoir le signal émis
Un objet qui est tombé à l’intérieur d’un trou noir émet un signal sous forme de radiation électromagnétique en direction d’un observateur juste à l’extérieur du trou noir. L’observateur (A) ne pourra jamais recevoir le signal émis. (B) peut recevoir le signal très fortement décalé vers le rouge. (C) peut recevoir le signal s’il attend assez longtemps. (D...
-
[13]
Brush, S. G., & Belloni, L. (1983). The history of modern physics: An international bibliography. New York: Garland. [14] Lewis, J. L. (1962). The teaching of modern physics in schools. Physics Bulletin, 13(1), 6. [15] Ogborn, J. (1979). Modern physics curricula in higher education. Reports on Progress in Physics, 42(4), 727. [16] Abell, S. K., & Lederman...
work page 1983
Show all 10 references
-
[37]
Astronomy from Four Perspectives,
Kersting, M., Blair, D., Sandrelli, S., Sherson, J., & Woithe, J. (2023). Making an IMPRESSion: Mapping out future directions in modern physics education. Physics Education, 59(1), 015501. [38] Karplus, R. (1971). What is modern physics? Physics Today, 24(3), 13–14. [39] Levri...
2023
-
[59]
Interstellar
Ding, L., & Beichner, R. (2009). Approaches to data analysis of multiple-choice questions. Physical Review Physics Education Research, 5(2), 020103. [60] Gasparini, A. (2021). Using mobile devices as experimental tools in physics lessons: An empirical study of the effects on l...
2009
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.