Pith. sign in

REVIEW 4 major objections 5 minor 29 references

On an EM-based closed-form solution for 2 parameter IRT models

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read In 2-parameter IRT models, EM iteration plus ordinary least squares yields closed-form estimators of discrimination and difficulty, so gradient search may be unnecessary.

desk verdict A clearly written EM+OLS variant for 2PL whose convergence-to-MML claim is asserted, not proven, because it replaces the M-step score sum with a stronger term-wise zero condition. read the letter →

arxiv 2411.18351 v1 pith:E53LY2FL submitted 2024-11-27 stat.ME stat.CO

Stefano Noventa (1) , Roberto Faleh (1) , Augustin Kelava (1) ((1) Methods Center , University of Tuebingen) This is my paper · ORCID
classification stat.MEstat.CO MSC 62F1062P15
keywords itemresponsetheory2PLmodelEMalgorithmclosed-formestimationordinaryleastsquaresmarginalmaximumlikelihoodquadraturelatentvariable
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Item response theory models with two parameters (discrimination and difficulty) are usually calibrated by numerical gradient search because maximum-likelihood item parameters have no closed form. This paper argues that if the EM algorithm is used, a closed form does exist: the discrimination and difficulty estimates at each M-step are simply the ordinary least squares regression of a latent response variable on ability. The latent response variable is built from the E-step proportions of correct responses at each quadrature point, converted to log-odds. If the argument is correct, 2PL item calibration reduces to iterating an E-step and a linear regression, and the sequence of OLS estimates converges to the true parameter values. A proof-of-concept simulation shows the resulting estimates are comparable to those from a standard calibration routine.

What carries the argument

The load-bearing object is the pointwise residual $N_{jt}^{1,(n)}-N_t^{(n)}P_j(\theta_t,\Gamma_j^{(n)})$: the M-step's score equation is a weighted sum of such residuals across quadrature points, while the paper replaces that sum with the stronger requirement that every residual vanishes. That replacement yields the latent response variable $y_{jt}^{(n)}$ and the linear regression $y_{jt}^{(n)}=a_j^{(n)}\theta_t+\tau_j^{(n)}+\epsilon_{jt}^{(n)}$, whose OLS solution is the closed-form estimator. The $\phi$-terms introduced in Equation (22) are shown to converge to zero at the true parameters, which is what bridges the EM proportions to the item response function.

What would settle it

Simulate a large-sample 2PL dataset with known parameters, run the EM-OLS routine and a standard gradient-search MML estimator from the same starting values, and compare the converged values; if the two fixed points differ by more than sampling error—especially for discrimination—the claimed equivalence between the pointwise-zero condition and the likelihood equations is refuted.

Watch

Extended reading notes

Core claim

The paper's central claim is that within an EM framework for 2-parameter logistic IRT models, the item parameters can be obtained in closed form by ordinary least squares. At iteration $n$, the E-step produces $N_{jt}^{1,(n)}$, the expected number of correct responses to item $j$ at quadrature point $\theta_t$, and $N_t^{(n)}$, the expected number of respondents at that point, conditional on provisional parameters. The method forms the latent response variable $y_{jt}^{(n)}=\log\big(N_{jt}^{1,(n)}/(N_t^{(n)}-N_{jt}^{1,(n)})\big)$ and regresses it on $\theta_t$, giving OLS estimates for discrimination $a_j$ and threshold $\tau_j$ (and hence difficulty $b_j=-\tau_j/a_j$) via Equations (31) and (32). The authors argue that this sequence converges to the true parameters because, at the true solution, each term in the first-order conditions vanishes individually, so the M-step's usual weighted sum can be replaced by pointwise residuals that OLS minimizes. They call this a 'trivial' solution of the partial derivatives and note that it makes gradient search unnecessary.

Load-bearing premise

The whole construction rests on assuming that requiring the model's item response function to match the observed proportion correct at every single ability-grid point, rather than only in the likelihood's weighted sum, still converges to the same maximum-likelihood estimates.

Editorial extensions

If this is right

  • 2PL and 1PL item parameters can be estimated by iterating an EM E-step with an OLS M-step, with no Newton-Raphson or other gradient search.
  • Each EM iteration produces a simple linear regression, so the per-iteration computational cost is small; the paper's timing comparisons suggest faster runs than the reference routine, though the code is not optimized.
  • The choice of the number of quadrature points matters: in the simulations, two points work best for the 1PL and three to four for the 2PL, with larger counts adding bias and variance.
  • Because the estimators are closed-form at each step, they expose practical identifiability issues directly: with finite noisy data the sequence can converge to an alternative parameter set that generates frequencies close to the observed ones.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the fixed point really coincides with the MLE, the same E-step-plus-regression trick could generalize to multidimensional IRT models by replacing the univariate regression with a multivariate OLS on several ability dimensions.
  • The closed-form M-step opens a natural route to standard errors: one could apply OLS-based inference to the final latent response regression, a step the paper does not take.
  • A direct diagnostic would be to compare the response-pattern frequencies predicted by the OLS fixed point with those predicted by a gradient-search MML solution; the paper's practical-identifiability discussion suggests they need not be identical in finite samples.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper revisits the complete-data likelihood for 2-parameter Item Response Theory (IRT) models and proposes an EM-based procedure in which the M-step is replaced by an Ordinary Least Squares regression. Specifically, the authors define a latent response variable as the log-odds of posterior expected proportions of correct responses at each quadrature point, then estimate discrimination and difficulty parameters by regressing this variable on the quadrature points. They claim that this yields a sequence of closed-form estimators that converges to the true item parameters and that gradient search is unnecessary. The paper includes a didactic review of the complete-data likelihood, a simulation study comparing the proposed procedure with the mirt package for 1PL and 2PL models, and an appendix exploring the effect of the number of quadrature points.

Significance. If the central claim were correct, the paper would provide a practically useful alternative to gradient-based MML estimation in IRT, with potential computational benefits. The paper also offers a clear pedagogic restatement of the complete-data likelihood framework and includes a reasonably large simulation study. However, the central theoretical claim is not established: the OLS M-step does not solve the EM score equations, and no convergence proof is supplied. The paper's own simulation results, particularly the degradation with increased quadrature points in Appendix C, are more consistent with the proposed procedure solving a different optimization problem than with it approximating the MML solution. The didactic portions and the empirical comparison are useful, but the load-bearing theoretical contribution is unsupported.

major comments (4)
  1. [Section 4.2, Eq. (26)] The derivation is invalid at the step from Eq. (22) to Eq. (26). The score equation (22) is a weighted sum over quadrature points equal to zero; the MML solution requires only that this sum vanish. Eq. (26) imposes the much stronger condition that every summand N^{1,(n)}_{jt} - N^{(n)}_t P_j(\theta_t, \Gamma^{(n)}_j) vanish separately. Eq. (24) only shows that, under asymptotic frequency assumptions and in the limit \Gamma^{(n)} \to \Gamma, the ratio N^{1,(n)}_{jt}/N^{(n)}_t approaches P_j(\theta_t, \Gamma_j); it does not imply that the term-wise condition holds at the fixed point of the proposed iteration. Consequently, the OLS estimates in Eqs. (31) and (32) are not roots of the score equations and do not maximize Q(\Gamma|\Gamma^{(n)}).
  2. [Section 4.2, after Eq. (32)] The statement that 'by construction, the sequence of estimators ... converges to the true values' is not supported by any argument. The proposed algorithm is a fixed-point iteration whose M-step is an OLS fit to a dependent variable y^{(n)}_{jt} that itself depends on the current parameter estimates through the posterior in Eq. (18). No proof of convergence, contraction, or monotonicity of the Q-function or the observed likelihood is provided, and no theorem is cited that would apply to this modified EM. The convergence claim is therefore an assertion, not a result, and it is load-bearing for the paper's main thesis.
  3. [Appendix C] The numerical evidence in Appendix C is inconsistent with the paper's central claim. If Eqs. (31) and (32) were solving the M-step of an EM algorithm for the quadrature-approximated observed likelihood, increasing the number of quadrature points should improve the approximation and should not systematically degrade the estimates. The reported increase in bias and RMSE with the number of quadrature points suggests that the OLS step is solving a different problem, one that depends on the range of the quadrature points rather than on the likelihood. The authors' explanation in terms of practical identifiability does not account for this systematic pattern.
  4. [Section 4.2, Eqs. (28)-(29)] The construction is circular in a way that undermines the 'closed-form' claim. The latent response variable y^{(n)}_{jt} is defined as the log-odds of the posterior expected proportions N^{1,(n)}_{jt}/N^{(n)}_t, and those proportions are computed from the current item parameters \Gamma^{(n)}. Regressing this variable on the quadrature points to recover a and \tau therefore amounts to a self-consistency equation rather than an independent regression of an observed outcome on ability. The paper does not show that the solution of this self-consistency equation coincides with the marginal maximum likelihood estimator.
minor comments (5)
  1. [Throughout] There are several typographical errors: 'Gauss-Hermitee' appears in the reference list, 'the the 2PL model' appears in the Table 3 caption, and 'n quads' is used in Figures 4-6 and Appendix C without a definition.
  2. [Eqs. (21), (26)-(30)] The notation for the total expected counts is inconsistent: Eq. (21) defines N^{(n)}_t with a superscript, but Eqs. (26)-(30) use N_t without the superscript. This makes it harder to follow which quantities are iteration-dependent.
  3. [Section 5] The comparison with mirt does not state the quadrature settings, convergence criteria, or starting values used by mirt. Without these details, the reported accuracy and timing comparisons are difficult to interpret.
  4. [Section 5, code availability] The manuscript states that the R script is 'available upon request'; for a reproducibility-oriented journal, a public repository would be preferable.
  5. [Section 4.2, Eqs. (28)-(29)] Equations (28) and (29) both define y^{(n)}_{jt} with the same left-hand side, but only Eq. (29) introduces an error term. The relation between the two equations should be stated more explicitly.

Circularity Check

2 steps flagged · score 6.0 of 10

OLS M-step solves an imposed term-wise zero system, not the likelihood score equations; convergence is asserted by construction.

  1. self definitional [Section 4.2, Eq. (26) (derived from Eq. (24))]
    "if at every step of the EM one were to consider the IRF Pj(θt, Γ(n) j ) and require such an IRF to be given exactly by Equation (24), that is an ideal case in which Pj(θt, Γ(n) j ) = P (n) j (θt), this would yield the following system of equations N 1,(n) jt − N (n) t Pj(θt, Γ(n) j ) = 0."

    The actual M-step score condition, Eq. (23), is a weighted sum over quadrature points equal to zero. Eq. (26) instead imposes each summand separately equal to zero. The paper justifies this by observing that at the true parameters the term-wise condition holds in the population limit, but that is an additional assumption, not the MML condition. The OLS estimators in Eqs. (31)–(32) are then constructed to solve this imposed term-wise system, so the claimed closed-form MML solution is the solution of a system the paper has defined rather than derived from the likelihood equations. The result is equivalent to its input by construction.

  2. self definitional [Section 4.2, immediately after Eqs. (31)–(32)]
    "By construction, the sequence of estimators Γ̂(n) j = {â(n) j , τ̂(n) j , b̂(n) j } converges to the true values Γ j = {aj, τj, bj}."

    The convergence statement is not derived from a fixed-point or monotonicity argument; it restates the limit in Eq. (24), which assumes Γ(n) → Γ and that the observed frequencies exactly equal the model probabilities. The OLS step recovers a and τ by inverting the same IRF that generated the latent response variable y(n), and y(n) itself is computed from posterior expected counts under the current Γ(n). Hence the true values appear both as the premise of the limiting argument and as the conclusion of the estimator sequence; the convergence claim reduces to a self-consistency condition rather than an independent result.

full rationale

The paper's central derivation is not circular through self-citation: it does not invoke a uniqueness theorem from the authors' prior work, and the empirical comparison with mirt is external evidence. The circularity lies instead in the internal derivation chain of Section 4.2. The M-step of a standard EM for IRT requires the weighted sum in Eq. (23) to vanish; the paper replaces this with the stronger term-wise condition of Eq. (26), justified only by a population-limit argument at the true parameters. The OLS estimators are then defined as the solution to this imposed term-wise system, which makes the claimed equivalence to the MML solution definitional rather than derived. The subsequent statement that the estimator sequence converges to the true values 'by construction' repeats the same reduction: the limiting argument already assumes the sequence approaches the true parameters and that the data frequencies equal the model probabilities. The external simulations are useful evidence that the algorithm produces reasonable estimates in practice, but they do not establish that the sequence is a standard EM or that its fixed point satisfies the likelihood score equations. Because the central claim reduces to the imposed term-wise zero system, a score of 6 is appropriate: partial circularity in the derivation of the closed-form claim.

Assumptions & free parameters 1 free parameters · 5 assumptions · 1 invented entities

The central derivation rests on standard IRT assumptions (local independence, normal ability, quadrature approximation), on an asymptotic replacement of observed frequencies by model probabilities, and on the ad hoc step-wise phi equals zero condition that turns the M-step into OLS. The number of quadrature points is a tuned computational parameter, and the latent response variable is constructed from the model itself rather than from independent data.

free parameters (1)
  • number of Gauss-Hermite quadrature points T = 2 for 1PL, 4 for 2PL
    Tuned by the authors for stability; Appendix C shows bias and RMSE vary strongly with T, so the reported performance depends on this hand-chosen setting.
assumptions (5)
  • domain assumption Local independence of item responses conditional on ability (Eq. 2)
    Assumed throughout the likelihood derivation and used to factor the complete-data log-likelihood.
  • domain assumption Ability is standard normal and approximated by Gauss-Hermite quadrature (Section 2 and Section 5)
    The latent distribution is fixed to N(0,1) and membership probabilities are replaced by quadrature weights, not estimated.
  • domain assumption Observed pattern frequencies NX are proportional to P(X|Gamma)N under the true parameters (Eq. 24)
    Used to show the ratio N^1/N_t converges to the IRF in the limit; in finite samples this holds only approximately due to sampling noise.
  • ad hoc to paper Every phi_jt term is zero at every EM step (Eqs. 26-27)
    Stronger than the actual first-order condition (22), which only requires a weighted sum over t to vanish; not implied by EM.
  • ad hoc to paper The sequence of OLS estimators converges to the true parameters by construction (Section 4.2)
    No proof is provided; the fixed point of the modified EM may differ from the MML solution.
invented entities (1)
  • latent response variable y_jt
    purpose: Linearizes the IRF so OLS can estimate a_j and tau_j in Eq. (29)
    y_jt is defined as the logit of posterior expected proportions, which depend on the current parameter estimates; it is not an observed variable and has no independent falsifiable handle.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On an EM-based closed-form solution for 2 parameter IRT models." pith.science (2026). https://pith.science/paper/E53LY2FL

@misc{pith2026241118351,
  author       = {Pith},
  title        = {Pith review of: On an EM-based closed-form solution for 2 parameter IRT models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E53LY2FL}},
  note         = {Machine review of arXiv:2411.18351}
}
read the original abstract

It is a well-known issue that in Item Response Theory models there is no closed-form for the maximum likelihood estimators of the item parameters. Parameter estimation is therefore typically achieved by means of numerical methods like gradient search. The present work has a two-fold aim: On the one hand, we revise the fundamental notions associated to the item parameter estimation in 2 parameter Item Response Theory models from the perspective of the complete-data likelihood. On the other hand, we argue that, within an Expectation-Maximization approach, a closed-form for discrimination and difficulty parameters can actually be obtained that simply corresponds to the Ordinary Least Square solution.

Figures

Figures reproduced from arXiv: 2411.18351 by the authors.

Figure 1
Figure 1. Box plots of the estimates of the difficulty parameters for the [PITH_FULL_IMAGE:figures/full_fig_p017_1.png] view at source ↗
Figure 2
Figure 2. Box plots of the estimates of the discrimination parameters for the [PITH_FULL_IMAGE:figures/full_fig_p018_2.png] view at source ↗
Figure 3
Figure 3. Box plots of the estimates of the difficulty parameters for the [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Mean values and RMSEs of the difficulty parameters in the 1PL [PITH_FULL_IMAGE:figures/full_fig_p026_4.png]
Figure 5
Figure 5. Figure 5: Mean values and RMSEs of the discrimination parameters in the [PITH_FULL_IMAGE:figures/full_fig_p027_5.png]
Figure 6
Figure 6. Figure 6: Mean values and RMSEs of the difficulty parameters in the 2PL [PITH_FULL_IMAGE:figures/full_fig_p028_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 28 canonical work pages

  1. [1]

    Blocker, A.W. (2022). fastGHQuad: Fast 'Rcpp' Implementation of Gauss-Hermitee Quadrature. R package version 1.0.1, https://CRAN.R-project.org/package=fastGHQuad

    work page 2022

  2. [2]

    Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29-–51

    work page 1972

  3. [3]

    D., & Aitkin, M

    Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: An application of an EM algorithm. Psychometrika, 46, 443--459

    work page 1981

  4. [4]

    D., & Lieberman, M

    Bock, R. D., & Lieberman, M. (1970). Fitting a response model for n dichotomously scored items. Psychometrika, 35, 179--197

    work page 1970

  5. [5]

    Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of statistical Software, 48, 1-29

    work page 2012

  6. [6]

    Christoffersson, A. (1975). Factor analysis of dichotomized variables. Psychometrika, 40, 5--32

    work page 1975

  7. [7]

    P., Laird, N

    Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood estimation from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society B, 39, 1--38

    work page 1977

  8. [8]

    R., Baker, F

    Harwell, M. R., Baker, F. B., & Zwarts, M. (1988). Item parameter estimation via marginal maximum likelihood and an EM algorithm: A didactic. Journal of Educational Statistics, 13(3), 243--271

    work page 1988

Show all 29 references
  1. [9]

    Holland, P. W. (1990). On the sampling theory foundations of item response theory models. Psychometrika, 55(4),577--601

    1990

  2. [10]

    Lawley, D.N. (1944). The factorial analysis of multiple item tests. Proceedings of the Royal Society of Edinburgh, 62-A, 74--82

    1944

  3. [11]

    Liu, Q., & Pierce, D. A. (1994). A Note on Gauss-Hermitee Quadrature. Biometrika, 81(3), 624–629. https://doi.org/10.2307/2337136

    1994 doi

  4. [12]

    McLachlan, G. J. & Krishnan, T. (1997). The EM algorithm and extensions. New York: John Wiley and Sons

    1997

  5. [13]

    Mersmann, O. (2023). microbenchmark: Accurate Timing Functions. R package version 1.4.10, https://CRAN.R-project.org/package=microbenchmark

    2023

  6. [14]

    Microsoft Corporation & Weston, S. (2022). doParallel: Foreach Parallel Adaptor for the `parallel' Package. R package version 1.0.17, https://CRAN.R-project.org/package=doParallel

    2022

  7. [15]

    Microsoft Corporation & Weston, S. (2022). foreach: Provides Foreach Looping Construct. R package version 1.5.2, https://CRAN.R-project.org/package=foreach

    2022

  8. [16]

    Mislevy, R. J. (1986). Bayes modal estimation in item response models. Psychometrika, 51, 177--195

    1986

  9. [17]

    Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159--176

    1992

  10. [18]

    Contributions to factor analysis of dichotomous variables

    Muthén, B.(1978). Contributions to factor analysis of dichotomous variables. Psychometrika, 43, 551--560

    1978

  11. [19]

    Orchard, T., & Woodbury, M. A. (1972). A missing information principle: Theory and applications. Proceedings of the 6th Berkeley Symposium on Mathematical Statistics and Probability. 1, 697--71

    1972

  12. [20]

    R: A Language and Environment for Statistical Computing

    R Core Team (2024). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/

    2024

  13. [21]

    Schneider, L., Strobl, C., Zeileis, A., & Debelak, R. (2022). An R Toolbox for Score-Based Measurement Invariance Tests in IRT Models. Behavior Research Methods, 54(5), 2101--2113

    2022

  14. [22]

    Takane, Y., & De Leeuw, J. (1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52(3), 393-408

    1987

  15. [23]

    Thissen D. 1982. Marginal maximum likelihood estimation for the one-parameter logistic model. Psychometrika, 47, 175-–186

    1982

  16. [24]

    E., & Tsutakawa, R

    Rigdon S. E., & Tsutakawa, R. K. (1983). Parameter estimation in latent trait models. Psychometrika, 48, 567–-574

    1983

  17. [25]

    Van der Linden, W. J. (Ed.). (2016). Handbook of Item Response Theory: Volume 2: Statistical Tools. CRC press

    2016

  18. [26]

    (2024) tidyr: Tidy Messy Data

    Wickham, H., Vaughan, D., & Girlich, M. (2024) tidyr: Tidy Messy Data. R package version 1.3.1, https://CRAN.R-project.org/package=tidyr

    2024

  19. [27]

    G., Hauber, A

    Wieland, F. G., Hauber, A. L., Rosenblatt, M., Tönsing, C., & Timmer, J. (2021). On structural and practical identifiability. Current Opinion in Systems Biology, 25, 60-69

    2021

  20. [28]

    J., & Hanson, B

    Woodruff, D. J., & Hanson, B. A. (1996). Estimation of Item Response Models Using the EM Algorithm for Finite Mixtures. ACT Research Report Series, 96(6)

    1996

  21. [29]

    Wu, C. J. (1983). On the convergence properties of the EM algorithm. The Annals of statistics, 95--103

    1983

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.