pith. sign in

arxiv: 2604.16031 · v1 · submitted 2026-04-17 · 📊 stat.ME · stat.AP

A Comparison of Joint and Stepwise Dynamic Cognitive Diagnostic Models

Yawen Ma , Anastasia Ushakova , Kate Cain , Gabriel Wallin This is my paper

Pith reviewed 2026-05-10 08:13 UTC · model grok-4.3

classification 📊 stat.ME stat.AP
keywords cognitive diagnostic modelslongitudinal datajoint modelinglatent transition modelsBayesian estimationstepwise estimationMonte Carlo simulationtransition parameters
0
0 comments X

The pith

Joint Bayesian modeling recovers transition parameters more accurately than bias-corrected stepwise methods in longitudinal cognitive diagnostic models, especially with short tests or small samples.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper evaluates two ways to extend cognitive diagnostic models to track how skills change over time: a joint Bayesian approach that estimates measurement and transition parts together, and a common stepwise method that handles them separately with bias correction. Simulations compare how well each recovers the key transition parameters when tests are short, samples are limited, or covariates are present. The joint model shows clearer advantages in accuracy under those constraints. This matters for fields like education where researchers need reliable pictures of skill development to guide interventions.

Core claim

In longitudinal settings with covariates, a unified Bayesian dynamic cognitive diagnostic model that jointly estimates item parameters, latent attribute profiles, and transition parameters provides more accurate recovery of the transition parameters than a bias-corrected stepwise latent transition CDM, with the performance gap widening under limited test length and sample size.

What carries the argument

The joint Bayesian dynamic cognitive diagnostic model that simultaneously estimates measurement components (item parameters and latent attribute profiles) and transition components (transition parameters with covariates) in one step, versus stepwise estimation that separates measurement and transition modeling.

If this is right

  • Applied researchers obtain more reliable estimates of how cognitive attributes transition between time points when using joint rather than stepwise procedures.
  • The accuracy gain from joint modeling grows larger precisely when data are scarcest, such as brief assessments or small participant groups.
  • Covariate effects on transitions can be estimated with less sequential bias in a single unified framework.
  • Longitudinal diagnostic applications gain practical guidance favoring joint models for settings where test length or sample size is constrained.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Real student data might show the joint approach leading to earlier or more accurate detection of skill declines that stepwise methods miss.
  • The pattern could extend to similar longitudinal latent variable problems in psychology or health, where joint estimation avoids error buildup across stages.
  • Direct comparisons on actual school or clinic records would test whether simulation advantages translate outside controlled conditions.

Load-bearing premise

The Monte Carlo simulation conditions and data-generating processes accurately represent the performance differences that would occur with real longitudinal cognitive diagnostic data.

What would settle it

A real longitudinal dataset from educational testing where the bias-corrected stepwise model recovers transition parameters at least as well as the joint model across multiple test lengths and sample sizes.

read the original abstract

To extend cognitive diagnostic models (CDMs) to longitudinal settings, stepwise approaches that integrate a CDM model with a latent transition model and covariates are widely used due to their flexibility. Previous research has shown that stepwise estimation can yield biased results, motivating classification-error correction as a means of improving inference over uncorrected stepwise procedures. In this study, we evaluate a unified Bayesian dynamic cognitive diagnostic model that jointly estimates measurement (item parameters, latent attribute profiles) and transition components (transition parameters) in longitudinal settings with covariates. We compare this joint approach with the bias-corrected stepwise latent transition CDM through a Monte Carlo study. Results demonstrate that joint modeling provides more accurate recovery of transition parameters, particularly under limited test length and sample size, underscoring its advantages for longitudinal diagnostic analysis and offering practical guidance for applied researchers.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript introduces a unified Bayesian dynamic cognitive diagnostic model that jointly estimates item parameters, latent attribute profiles, and transition parameters (including covariates) in longitudinal settings. It compares this joint approach to a bias-corrected stepwise latent transition CDM via Monte Carlo simulation and claims that joint modeling yields more accurate recovery of transition parameters, especially under limited test length and sample size.

Significance. If the Monte Carlo recovery results hold under the reported conditions, the work provides practical guidance for applied researchers choosing between joint and stepwise procedures in longitudinal CDMs. The simulation-based comparison is a strength for reproducibility when design factors and metrics are fully specified.

major comments (2)
  1. [Monte Carlo study] Monte Carlo study section: The data-generating process (DGP) for the transition model, including the exact functional form of transition probabilities and how covariates enter the model, is not specified with equations or parameter values. This is load-bearing for the central claim because the reported advantage in transition-parameter recovery cannot be evaluated without knowing whether the simulated conditions are representative of real longitudinal CDM data.
  2. [Results] Results section: No numerical recovery metrics (bias, RMSE, or coverage rates for transition parameters) or the number of replications are reported, even in summary form. Without these, the magnitude and robustness of the claimed superiority under limited sample size and test length cannot be assessed.
minor comments (1)
  1. [Abstract] Abstract: Consider adding one sentence on the number of replications and the primary recovery metrics to strengthen the summary of evidence.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their thoughtful review and constructive suggestions. We address each major comment below and will revise the manuscript accordingly to improve transparency and completeness.

read point-by-point responses

  1. Referee: [Monte Carlo study] Monte Carlo study section: The data-generating process (DGP) for the transition model, including the exact functional form of transition probabilities and how covariates enter the model, is not specified with equations or parameter values. This is load-bearing for the central claim because the reported advantage in transition-parameter recovery cannot be evaluated without knowing whether the simulated conditions are representative of real longitudinal CDM data.

    Authors: We agree that explicit specification of the DGP is necessary for readers to assess the simulation conditions. In the revised manuscript, we will add the full set of equations for the transition probabilities (including the multinomial logistic form and the role of covariates) along with the specific parameter values used to generate the data. revision: yes

  2. Referee: [Results] Results section: No numerical recovery metrics (bias, RMSE, or coverage rates for transition parameters) or the number of replications are reported, even in summary form. Without these, the magnitude and robustness of the claimed superiority under limited sample size and test length cannot be assessed.

    Authors: We acknowledge that the results would be more informative with quantitative summaries. We will include tables reporting bias, RMSE, and coverage rates for the transition parameters across all design conditions, and we will state the number of Monte Carlo replications performed. revision: yes

Circularity Check

0 steps flagged

No significant circularity

full rationale

The manuscript is a Monte Carlo simulation study that generates data under known DGPs and compares recovery of transition parameters between joint Bayesian estimation and bias-corrected stepwise procedures. The central claim rests on direct numerical comparison of bias, RMSE, and coverage across controlled conditions (sample size, test length, etc.), not on any algebraic derivation, parameter fitting that is then relabeled as a prediction, or self-citation chain that substitutes for independent evidence. No equation or modeling step reduces to its own inputs by construction; the simulation design functions as an external benchmark.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The paper rests on standard assumptions of cognitive diagnostic models and latent transition models; simulation parameters are chosen by the authors to probe performance differences.

free parameters (1)
  • simulation design factors
    Sample sizes, test lengths, number of attributes, and transition probabilities chosen to evaluate method performance under varying conditions.
axioms (1)
  • domain assumption Standard CDM measurement model assumptions and latent transition model assumptions hold in the data-generating process.
    Invoked to justify the simulation setup and model estimation.

pith-pipeline@v0.9.0 · 5434 in / 1171 out tokens · 31606 ms · 2026-05-10T08:13:30.884714+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

53 extracted references · 53 canonical work pages

  1. [1]

    The generalized DINA model framework.Psychometrika, 76(2):179–199, April 2011

    Jimmy de la Torre. The generalized DINA model framework.Psychometrika, 76(2):179–199, April 2011

    work page 2011

  2. [2]

    Zheng, J

    X. Zheng, J. S. Yang, and J. R. Harring. Latent growth curve analysis with item response data: A method- ological investigation of model parameterization, estimation, and attrition.Structural Equation Modeling: A Multidisciplinary Journal, 29(2):182–206, 2021

    work page 2021

  3. [3]

    Yinghan Chen, Steven Andrew Culpepper, and Jeffrey A. Douglas. A Hidden Markov Model for Learning Trajec- tories in Cognitive Diagnosis With Application to Spatial Rotation Skills.Applied Psychological Measurement, 42(1):3–19, September 2017

    work page 2017

  4. [4]

    Doctoral dissertation, University of Maryland, College Park, College Park, MD, 2015

    Ming Li.Investigating Methods of Incorporating Covariates in Growth Mixture Modeling: A Simulation Study. Doctoral dissertation, University of Maryland, College Park, College Park, MD, 2015

    work page 2015

  5. [5]

    Shiyu Wang, Yan Yang, Steven Andrew Culpepper, and Jeffrey A. Douglas. Tracking Skill Acquisition With Cognitive Diagnosis Models: A Higher-Order, Hidden Markov Model With Covariates.Journal of Educational and Behavioral Statistics, 43(1):57–87, January 2018

    work page 2018

  6. [6]

    Liang, J

    Q. Liang, J. de la Torre, M. E. Larimer, and E.-Y . Mun. Mental Health Symptom Profiles Over Time: A Three-Step Latent Transition Cognitive Diagnosis Modeling Analysis with Covariates. In M. Stemmler, W. Wiedermann, and F. Huang, editors,Dependent Data in Social Sciences Research: Forms, Issues, and Methods of Analysis, pages XXX–XXX, New York, 2024. Springer

    work page 2024

  7. [7]

    Leo A. Goodman. Exploratory Latent Structure Analysis Using Both Identifiable and Unidentifiable Models. Biometrika, 61(2):215–231, 1974

    work page 1974

  8. [8]

    Collins and Stephanie T

    Linda M. Collins and Stephanie T. Lanza.Latent Class and Latent Transition Analysis: With Applications in the Behavioral, Social, and Health Sciences. John Wiley & Sons, Hoboken, NJ, 2010

    work page 2010

  9. [9]

    Latent Variable Analysis: Growth Mixture Modeling and Related Techniques for Longitudinal Data.Handbook of Quantitative Methodology for the Social Sciences, pages 345–368, 2004

    Bengt Muthén. Latent Variable Analysis: Growth Mixture Modeling and Related Techniques for Longitudinal Data.Handbook of Quantitative Methodology for the Social Sciences, pages 345–368, 2004

    work page 2004

  10. [10]

    Nylund, Tihomir Asparouhov, and Bengt Muthén

    Karen L. Nylund, Tihomir Asparouhov, and Bengt Muthén. Deciding on the Number of Classes in Latent Class Analysis and Growth Mixture Modeling: A Monte Carlo Simulation Study.Structural Equation Modeling: A Multidisciplinary Journal, 14(4):535–569, 2007

    work page 2007

  11. [11]

    Ming Li and Jeffrey R. Harring. Investigating Approaches to Estimating Covariate Effects in Growth Mixture Modeling: A Simulation Study.Educational and Psychological Measurement, 77(5):766–791, 2017

    work page 2017

  12. [12]

    Tekle, and Jeroen K

    Zsuzsa Bakk, Fetene B. Tekle, and Jeroen K. Vermunt. Estimating the Association between Latent Class Membership and External Variables Using Bias-Adjusted Three-Step Approaches.Sociological Methodology, 43(1):272–311, 2013. 8 Joint VS Stepwise Dynamic CDMsA PREPRINT

    work page 2013

  13. [13]

    Relating latent class analysis results to variables not included in the analysis

    Stephen Clark and Bengt Muthén. Relating latent class analysis results to variables not included in the analysis. Technical report, Muthén & Muthén, 2009. Retrieved from http://www.statmodel.com/download/relatinglca.pdf

    work page 2009

  14. [14]

    Marcel A. Croon. Using Predicted Latent Class Membership in Subsequent Analyses. In Jacques A. Hagenaars and Allan L. McCutcheon, editors,Applied Latent Class Analysis, pages 194–223. Cambridge University Press, Cambridge, 2002

    work page 2002

  15. [15]

    Auxiliary Variables in Latent Class Analysis: Three-Step Approaches

    Tihomir Asparouhov and Bengt Muthén. Auxiliary Variables in Latent Class Analysis: Three-Step Approaches. Structural Equation Modeling: A Multidisciplinary Journal, 21(3):329–341, 2014

    work page 2014

  16. [16]

    Two-Step Estimation of Models Between Latent Classes and External Variables

    Zsuzsa Bakk and Jouni Kuha. Two-Step Estimation of Models Between Latent Classes and External Variables. Psychometrika, 83(4):871–892, 2018

    work page 2018

  17. [17]

    Estimating Relationships Between Latent Class Member- ship and External Variables Using Stepwise Latent Class Analysis.Political Analysis, 12(1):3–20, 2004

    Annabel Bolck, Marcel Croon, and Jacques Hagenaars. Estimating Relationships Between Latent Class Member- ship and External Variables Using Stepwise Latent Class Analysis.Political Analysis, 12(1):3–20, 2004

    work page 2004

  18. [18]

    Jeroen K. Vermunt. Latent Class Modeling with Covariates: Two-Step Estimation Revisited.Political Analysis, 18(4):450–469, 2010

    work page 2010

  19. [19]

    Montanari, and Silvia Pandolfi

    Francesco Bartolucci, Giorgio E. Montanari, and Silvia Pandolfi. Latent Markov Models: A Review of a Class of Models for Longitudinal Data with Discrete Latent Variables.African Stat. J, 18:43–82, 2014

    work page 2014

  20. [20]

    Miglioretti, Scott L

    Karen Bandeen-Roche, Diana L. Miglioretti, Scott L. Zeger, and Paul J. Rathouz. Latent Variable Regression for Multiple Discrete Outcomes.Journal of the American Statistical Association, 92(440):1375–1386, December 1997

    work page 1997

  21. [21]

    Combining Multiple Outcomes to Derive Score-Based Stages of Physical Frailty: The Women’s Health and Aging Study.Epidemiology, 13(6):661–670, November 2002

    Qian-Li Xue and Karen Bandeen-Roche. Combining Multiple Outcomes to Derive Score-Based Stages of Physical Frailty: The Women’s Health and Aging Study.Epidemiology, 13(6):661–670, November 2002

    work page 2002

  22. [22]

    Clifford C. Clogg. New Developments in Latent Structure Analysis.Evaluation Review, 5(3):445–459, 1981

    work page 1981

  23. [23]

    Mitchell Dayton and George B

    C. Mitchell Dayton and George B. Macready. A Latent Class Covariate Model with Applications to Criterion- Referenced Testing.Biometrika, 75(1):173–178, 1988

    work page 1988

  24. [24]

    Lanza, Xianming Tan, and Beth C

    Stephanie T. Lanza, Xianming Tan, and Beth C. Bray. Latent Class Analysis with Antenatal Predictors: A One- Step Approach Using Full Information Maximum Likelihood.Structural Equation Modeling: A Multidisciplinary Journal, 20(4):692–711, 2013

    work page 2013

  25. [25]

    Jimmy de la Torre and Jeffrey A. Douglas. Higher-order Latent Trait Models for Cognitive Diagnosis.Psychome- trika, 69(3):333–353, 2004

    work page 2004

  26. [26]

    Ayers, S

    E. Ayers, S. Rabe-Hesketh, and R. Nugent. Incorporating Student Covariates in Cognitive Diagnosis Models. Journal of Classification, 30(2):195–224, August 2013

    work page 2013

  27. [27]

    Yoon Soo Park and Young-Sun Lee. An Extension of the DINA Model Using Covariates: Examining Factors Affecting Response Probability and Latent Classification.Applied Psychological Measurement, 38(5):556–570, October 2014

    work page 2014

  28. [28]

    Iaconangelo and J

    C. Iaconangelo and J. de la Torre. Three-step estimation of cognitive diagnosis models with covariates. In Proceedings of the 81st International Meeting of the Psychometric Society, page 32, Asheville, NC, United States, July 2016. The Psychometric Society. Paper presentation

    work page 2016

  29. [29]

    Oberski, and Jeroen K

    Roberto Di Mari, Daniel L. Oberski, and Jeroen K. Vermunt. Bias-Adjusted Three-Step Latent Markov Modeling With Covariates.Structural Equation Modeling: A Multidisciplinary Journal, 23(5):649–666, 2016

    work page 2016

  30. [30]

    A statistical framework for dynamic cognitive diagnosis in digital learning environments

    Yawen Ma, Anastasia Ushakova, Kate Cain, and Gabriel Wallin. A statistical framework for dynamic cognitive diagnosis in digital learning environments. June 2025

    work page 2025

  31. [31]

    Latent Transition Cognitive Diagnosis Model With Covariates: A Three-Step Approach.Journal of Educational and Behavioral Statistics, 48(6):690–718, 2023

    Qianru Liang, Jimmy de la Torre, and Nancy Law. Latent Transition Cognitive Diagnosis Model With Covariates: A Three-Step Approach.Journal of Educational and Behavioral Statistics, 48(6):690–718, 2023

    work page 2023

  32. [32]

    Yigit and Jeffrey A

    Hulya D. Yigit and Jeffrey A. Douglas. First-Order Learning Models With the G-DINA: Estimation With the EM Algorithm and Applications.Journal of Educational and Behavioral Statistics, 45(3):264–302, 2021

    work page 2021

  33. [33]

    The simple view of reading.Reading and Writing, 2:127–160, 01 1990

    Wesley Hoover and Philip Gough. The simple view of reading.Reading and Writing, 2:127–160, 01 1990

    work page 1990

  34. [34]

    Culpepper

    Yinyin Chen and Steven A. Culpepper. A multivariate probit model for learning trajectories: A fine-grained evaluation of an educational intervention.Applied Psychological Measurement, 44(7-8):515–530, 2020

    work page 2020

  35. [35]

    Culpepper, and Yinyin Chen

    Yang Liu, Steven A. Culpepper, and Yinyin Chen. Identifiability of hidden markov models for learning trajectories in cognitive diagnosis.Psychometrika, 88(2):361–386, 2023

    work page 2023

  36. [36]

    Using response times to assess learning progress: A joint model for responses and response times.Measurement: Interdisciplinary Research and Perspectives, 16(1):45–58, 2018

    Shiyu Wang, Susu Zhang, Jeff Douglas, and Steven Culpepper. Using response times to assess learning progress: A joint model for responses and response times.Measurement: Interdisciplinary Research and Perspectives, 16(1):45–58, 2018. 9 Joint VS Stepwise Dynamic CDMsA PREPRINT

    work page 2018

  37. [37]

    R package version 2.9.3

    Wen-Chao Ma and Jimmy de la Torre.GDINA: An R Package for Cognitive Diagnosis Modeling, 2020. R package version 2.9.3

    work page 2020

  38. [38]

    Using JAGS for Bayesian Cognitive Diagnosis Modeling: A Tutorial

    Peida Zhan, Hong Jiao, and Kaiwen Man. Using JAGS for Bayesian Cognitive Diagnosis Modeling: A Tutorial. Journal of Educational and Behavioral Statistics, 44(4):473–503, 2019

    work page 2019

  39. [39]

    Culpepper

    Steven A. Culpepper. Bayesian estimation of the dina model.Journal of Educational and Behavioral Statistics, 2015

    work page 2015

  40. [40]

    Dina model and parameter estimation: A didactic.Journal of Educational and Behavioral Statistics, 34(1):115–130, 2009

    Jimmy de la Torre. Dina model and parameter estimation: A didactic.Journal of Educational and Behavioral Statistics, 34(1):115–130, 2009

    work page 2009

  41. [41]

    Sufficient and necessary conditions for the identifiability of the q-matrix.Statistica Sinica, 31(1):449–472, 2021

    Yuqi Gu and Gongjun Xu. Sufficient and necessary conditions for the identifiability of the q-matrix.Statistica Sinica, 31(1):449–472, 2021

    work page 2021

  42. [42]

    Model Similarity, Model Selection, and Attribute Classification.Applied Psychological Measurement, 40(3):401–416, June 2016

    Wenchao Ma, Charles Iaconangelo, and Jimmy de la Torre. Model Similarity, Model Selection, and Attribute Classification.Applied Psychological Measurement, 40(3):401–416, June 2016

    work page 2016

  43. [43]

    Chia-Yi Chiu and Jeffrey A. Douglas. A nonparametric approach to cognitive diagnosis by proximity to ideal response patterns.Journal of Classification, 30(2):225–250, 2013

    work page 2013

  44. [44]

    Cognitive diagnosis modelling incorporating item response times

    Peiying Zhan, Hong Jiao, and Dandan Liao. Cognitive diagnosis modelling incorporating item response times. British Journal of Mathematical and Statistical Psychology, 71(2):262–286, 2018

    work page 2018

  45. [45]

    Andrew Gelman and Donald B. Rubin. Inference from iterative simulation using multiple sequences.Statistical Science, 7(4):457–472, 1992

    work page 1992

  46. [46]

    Elizabeth Koehler, Elizabeth Brown, and Sebastien J.-P. A. Haneuse. On the assessment of monte carlo error in simulation-based statistical analyses.The American Statistician, 63(2):155–162, 2009

    work page 2009

  47. [47]

    Templin and Robert A

    Jonathan L. Templin and Robert A. Henson. Measurement of Psychological Disorders Using Cognitive Diagnosis Models.Psychological Methods, 11(3):287–305, 2006

    work page 2006

  48. [48]

    A General Method of Empirical Q-matrix Validation.Psychometrika, 81(2):253–273, 2016

    Jimmy de la Torre and Chia-Yi Chiu. A General Method of Empirical Q-matrix Validation.Psychometrika, 81(2):253–273, 2016

    work page 2016

  49. [49]

    Jimmy de la Torre and Jeffrey A. Douglas. Higher-Order Latent Trait Models for Cognitive Diagnosis.Psychome- trika, 69(3):333–353, June 2004

    work page 2004

  50. [50]

    Culpepper

    Ying Chen and Steven A. Culpepper. Bayesian estimation of the DINA Q-matrix.British Journal of Mathematical and Statistical Psychology, 70(3):357–380, 2017

    work page 2017

  51. [51]

    A Gibbs sampling algorithm that estimates the Q-matrix for the DINA model.Journal of Mathematical Psychology, 93:102275, 2019

    Mengta Chung. A Gibbs sampling algorithm that estimates the Q-matrix for the DINA model.Journal of Mathematical Psychology, 93:102275, 2019

    work page 2019

  52. [52]

    Q-Matrix Estimation Methods for Cognitive Diagnosis Models: Based on Partial Known Q-Matrix.Multivariate Behavioral Research, 55(1):1–13, April 2020

    Daxun Wang, Yan Cai, and Dongbo Tu. Q-Matrix Estimation Methods for Cognitive Diagnosis Models: Based on Partial Known Q-Matrix.Multivariate Behavioral Research, 55(1):1–13, April 2020

    work page 2020

  53. [53]

    R Foundation for Statistical Computing, Vienna, Austria, 2023

    R Core Team.R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2023. A Code Availability The implementation of the joint estimation framework using R (version 4.5.2; [53]) is available at https://github. com/Yawen-Ma/IMPS_proceeding2025. The stepwise procedure builds on the R implementation of...

    work page 2023