pith. sign in

arxiv: 2512.18418 · v2 · submitted 2025-12-20 · 🧮 math.HO

Development of Testing Methodology in Mathematics Education in the Context of Digitalization

I.V. Orlovskyi , O. A. Tymoshenko This is my paper

Pith reviewed 2026-05-16 21:08 UTC · model grok-4.3

classification 🧮 math.HO
keywords mathematics educationdigital assessmentMoodle platformClassical Test Theorystep-by-step verificationtesting methodologymultistage taskseducational technology
0
0 comments X

The pith

Moodle-based testing with step-by-step verification reduces errors in mathematics assessments and improves feedback quality.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper addresses declining mathematical proficiency by proposing a Moodle-based testing system that adds step-by-step solution verification and interactive exercises. This digital approach creates a more structured evaluation process than traditional methods. The system is presented as reducing student errors while delivering higher-quality feedback to support conceptual understanding. Classical Test Theory is applied to measure the reliability of the multistage tasks within this format. A sympathetic reader would see this as a practical route to more transparent and effective assessment in mathematics education.

Core claim

The central claim is that integrating step-by-step verification and interactive exercises into a Moodle platform produces a structured evaluation process for mathematics that reduces student errors and improves feedback quality, while Classical Test Theory establishes the reliability of these multistage tasks.

What carries the argument

The Moodle-based testing system incorporating step-by-step solution verification and interactive exercises, analyzed via Classical Test Theory for reliability of multistage tasks.

Load-bearing premise

That adding step-by-step verification and interactive exercises to Moodle will reduce errors and improve understanding without any controlled comparison or empirical data presented.

What would settle it

A side-by-side trial measuring error rates, feedback uptake, and conceptual understanding scores between students using the Moodle system and students using standard paper-and-pencil tests would falsify the claim if the two groups show no measurable difference.

Figures

Figures reproduced from arXiv: 2512.18418 by I.V. Orlovskyi, O. A. Tymoshenko.

Figure 1
Figure 1. Figure 1: Moodle question implementation using LaTeX, displaying determinant evaluation, [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: In this activity, students must correctly place integration limits by dragging the appropriate answer blocks into the corresponding empty slots. This process allows 9 [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 2
Figure 2. Figure 2: Moodle test interface for solving a determinant-based equation, showing stepwise evalu [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Interactive drag-and-drop exercise for determining integration bounds, reinforcing con [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Multiple-choice exercise for determining integration bounds [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Interactive drag-and-drop exercise for matching vector sums with their corresponding [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Correlation heatmap showing the relationship between task scores and overall test scores. [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Results of the correlation analysis for the quadratic equation coefficients and roots. The left panel displays a heatmap of correlations between the coefficients (E, F, G) and the roots (H, I). The right panel shows a histogram of the distribution of C values with an overlaid probability density function. steps, such as validating determinant calculations before forming the equation. Fur￾thermore, Task C s… view at source ↗
read the original abstract

Access to quality education remains a global challenge, particularly in crisis-affected regions. This study examines the decline in students' mathematical proficiency and proposes an innovative Moodle-based testing system that incorporates step-by-step solution verification and interactive exercises. Unlike traditional assessments, this approach ensures a more structured evaluation process, reducing student errors and improving feedback quality. Additionally, the study integrates Classical Test Theory to analyse test reliability, offering a novel perspective on the effectiveness of digital assessments. The proposed methodology improves assessment transparency and conceptual understanding, while Classical Test Theory is used to evaluate the reliability of multistage mathematical tasks.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes a Moodle-based digital testing system for mathematics education that incorporates step-by-step solution verification and interactive exercises. It claims this structured approach ensures reduced student errors, improved feedback quality, and greater conceptual understanding, particularly in crisis-affected regions, while integrating Classical Test Theory (CTT) to evaluate the reliability of multistage mathematical tasks.

Significance. If the proposed system were implemented and its benefits empirically validated, it could provide a practical framework for enhancing transparency and feedback in digital mathematics assessments. The integration of CTT offers a standard psychometric lens, but without any reported data or results the contribution remains prospective rather than demonstrated.

major comments (2)
  1. [Abstract and Methodology] Abstract and proposed methodology sections: the central claim that the system 'ensures a more structured evaluation process, reducing student errors and improving feedback quality' is presented as a guaranteed outcome of the design, yet the manuscript supplies no implementation details, student cohort, measured error rates, pre/post comparisons, or any CTT statistics (item difficulty, discrimination, reliability coefficients) computed on actual responses. This absence is load-bearing for the asserted benefits.
  2. [CTT Application] CTT integration section: the text states that Classical Test Theory is used to analyse test reliability, but no concrete application, formulas, or example calculations appear; the reliability evaluation of multistage tasks is described only at the level of intention.
minor comments (2)
  1. [System Description] The workflow of step-by-step verification would be clearer with an accompanying flowchart or pseudocode example.
  2. [Terminology] Ensure consistent terminology between 'multistage mathematical tasks' and 'interactive exercises' throughout.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments on our manuscript proposing a Moodle-based testing methodology for mathematics education. We acknowledge that the work is prospective and will revise to clarify the intended benefits of the design, provide concrete CTT details, and distinguish proposals from empirical results.

read point-by-point responses

  1. Referee: [Abstract and Methodology] Abstract and proposed methodology sections: the central claim that the system 'ensures a more structured evaluation process, reducing student errors and improving feedback quality' is presented as a guaranteed outcome of the design, yet the manuscript supplies no implementation details, student cohort, measured error rates, pre/post comparisons, or any CTT statistics (item difficulty, discrimination, reliability coefficients) computed on actual responses. This absence is load-bearing for the asserted benefits.

    Authors: We agree that the claims in the abstract and methodology should be framed as intended design outcomes rather than guaranteed results. The manuscript presents a methodological proposal, not an empirical study with collected data. In revision we will rephrase the abstract and relevant sections to state that the structured step-by-step verification and interactive exercises are designed to reduce errors and improve feedback, while explicitly noting that empirical validation (including student cohorts, error rates, and pre/post measures) remains future work. This will accurately reflect the prospective contribution without overstating current evidence. revision: yes

  2. Referee: [CTT Application] CTT integration section: the text states that Classical Test Theory is used to analyse test reliability, but no concrete application, formulas, or example calculations appear; the reliability evaluation of multistage tasks is described only at the level of intention.

    Authors: We accept that the CTT section requires greater specificity. We will revise it to include the standard CTT formulas (e.g., item difficulty p = correct responses / total responses, discrimination index, and reliability coefficient such as KR-20 or Cronbach's alpha) and add an illustrative example calculation using a hypothetical multistage mathematics task. This will demonstrate the intended application to multistage tasks while keeping the focus on methodological development rather than reporting new empirical results. revision: yes

Circularity Check

0 steps flagged

No derivation chain or fitted predictions; paper is a descriptive proposal of a Moodle system with CTT mention but no equations or self-referential reductions.

full rationale

The manuscript proposes a Moodle-based testing system incorporating step-by-step verification and interactive exercises, stating that it 'ensures a more structured evaluation process, reducing student errors and improving feedback quality' and that 'Classical Test Theory is used to evaluate the reliability of multistage mathematical tasks.' No equations, parameters, or predictions appear. No self-citations are invoked to justify uniqueness or load-bearing premises. The central claims are presented as design outcomes rather than derived results that reduce to inputs by construction. This is a standard non-circular proposal paper; the absence of empirical data is a separate evidence issue, not circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that step-by-step digital verification will improve learning outcomes and that Classical Test Theory is suitable for evaluating multistage math tasks; no free parameters or invented entities are introduced.

axioms (1)
  • domain assumption Classical Test Theory provides a valid framework for assessing reliability of multistage mathematical tasks
    Invoked when the authors state they integrate CTT to analyse test reliability

pith-pipeline@v0.9.0 · 5393 in / 1109 out tokens · 22689 ms · 2026-05-16T21:08:47.422978+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

25 extracted references · 25 canonical work pages

  1. [1]

    E., & Caon, F

    Balboni, P. E., & Caon, F. (2023). Using Moodle in EFL teaching in Italy: The case of Everywhere.Journal of Language and Education, 9(1), 45-60. https://www.academia.edu/104569683

    work page arXiv 2023

  2. [2]

    J., & Biel, R

    Brame, C. J., & Biel, R. (2015). Test-enhanced learning: The potential for testing to promote greater learning in undergraduate science courses.CBE Life Sciences Education, 14(2), 14:es4

    work page 2015

  3. [3]

    P., & Núnez, J

    Cerezo, R., Sánchez-Santillán, M., Paule-Ruiz, M. P., & Núnez, J. C. (2016). Students’ LMS interaction patterns and their relationship with achievement: A case study in higher education.Computers & Education, 96, 42-54. https://doi.org/10.1016/j.compedu.2016.02.006

    work page doi:10.1016/j.compedu.2016.02.006 2016

  4. [4]

    (1985).The didactic transposition: From scholarly knowledge to taught knowledge

    Chevallard, Y. (1985).The didactic transposition: From scholarly knowledge to taught knowledge. Grenoble: La Pensée Sauvage

    work page 1985

  5. [5]

    Cosgrove, J., & Cartwright, F. (2014). Changes in achievement on PISA: The case of Ireland and implications for international assessment practice.Large- scale Assessments in Education, 2(1), Article 2. https://doi.org/10.1186/2196-0739-2-2

    work page doi:10.1186/2196-0739-2-2 2014

  6. [6]

    Holt, Rinehart and Winston

    Crocker, L., &Algina, J.(1986).Introduction to classical and modern test theory. Holt, Rinehart and Winston

    work page 1986

  7. [7]

    Deiser, O., & Reiss, K. (2014). Mathematics education at the transition from school to university: Difficulties and opportunities.International Journal of STEM Education, 3(4), 87-99

    work page 2014

  8. [8]

    (2010).The effect of conflict on education in Palestine

    Di Maio, M. (2010).The effect of conflict on education in Palestine. Centre for Economic Policy Research. 18

    work page 2010

  9. [9]

    (2011).Mathematics Education in Europe: Common Challenges and National Policies

    European Commission/EACEA/Eurydice. (2011).Mathematics Education in Europe: Common Challenges and National Policies. Eurydice Report. Publi- cations Office of the European Union

    work page 2011

  10. [10]

    Kalanlar, B. (2021). Distance education of midwifery students during the COVID-19 pandemic: Challenges and recommendations.Journal of Midwifery & Women’s Health, 66(4), 579-585

    work page 2021

  11. [11]

    Machisi, E. (2018). Comparing school and university mathematics curricula: Challengesinthetransitiontohighereducation.African Journal of Mathematics Education, 5(2), 67-82

    work page 2018

  12. [12]

    McDougall, R. (2015). Psychological barriers to learning mathematics: Stress and its impact on performance.Educational Psychology Journal, 28(1), 90-105

    work page 2015

  13. [13]

    Mullen, C., Pettigrew, J., Cronin, A., Rylands, L., & Shearman, D. (2021). The rapid move to online mathematics support: Changes in pedagogy and social interaction.International Journal of Mathematical Education in Science and Technology, 53(1), 64-91

    work page 2021

  14. [14]

    Nuno, C., & Márôco, J. (2024). Should mathematicians worry with PISA and TIMSS math results?Eur. Math. Soc. Mag, 133, 42-47.https://doi.org/10. 4171/MAG/210

    work page 2024

  15. [15]

    Mathematics in a Modern Technical University

    Omelchuk, T. S., Orlovskyi, I. V., & Tymoshenko, O. A. (2018). Features of the creation and quality analysis of step-by-step tests in linear algebra and analytical geometry.Seventh International Scientific and Practical Conference "Mathematics in a Modern Technical University", December 27-28, 2018, Kyiv, 268-271

    work page 2018

  16. [16]

    (2023).PISA 2022 Results: Impact of COVID-19 on student learning outcomes

    OECD. (2023).PISA 2022 Results: Impact of COVID-19 on student learning outcomes. Organisation for Economic Co-operation and Development

    work page 2023

  17. [17]

    (2021).Moodle at the Open University: A case study in large- scale online learning management

    Open University. (2021).Moodle at the Open University: A case study in large- scale online learning management. Moodle Docs. https://docs.moodle.org/en/

    work page 2021

  18. [18]

    Proshkin, V., & Foster, C. (2024). Challenges faced by Ukrainian students learn- ing mathematics in UK schools: A comparative study.Cambridge Journal of Education, 54(1), 112-130. 19

    work page 2024

  19. [19]

    (2013).Computer Aided Assessment of Mathematics

    Sangwin, C. (2013).Computer Aided Assessment of Mathematics. Oxford Uni- versity Press

    work page 2013

  20. [20]

    A., & Tacy, M

    Seaton, K. A., & Tacy, M. (2021). The value of varying question design.Inter- national Journal of Mathematical Education in Science and Technology, 53(1), 240-250

    work page 2021

  21. [21]

    Samuels, C. A. (2024, July 31). Mathematics test scores in some countries have been dropping for years, even as the subject grows in importance.The Hechinger Report. https://hechingerreport.org/

    work page 2024

  22. [22]

    Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4-14

    work page 1986

  23. [23]

    (2023).Geneva Palais briefing note on the impact of the Ukraine war on children’s education

    UNICEF. (2023).Geneva Palais briefing note on the impact of the Ukraine war on children’s education

    work page 2023

  24. [24]

    Vollstedt, M., & Rach, S. (2014). Difficulties in mathematics at the beginning of university studies: A German perspective.International Journal of Mathe- matical Education in Science and Technology, 45(5), 721-737

    work page 2014

  25. [25]

    Weiglhofer, H., & Sabitzer, B. (2024). Enhancing mathematics education through collaborative digital material design: Lessons from a national project. European Journal of Science and Mathematics Education, 13(1), 1-15. 20

    work page 2024