{"id":"b21bf21d-fb77-4eb3-82c0-dacce7a1e1f8","arxiv_id":"2411.18351","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"A new EM variant estimates 2PL IRT discrimination and difficulty parameters by ordinary least squares on the logit of posterior expected proportions, with simulations matching mirt.","lead":"This paper proposes replacing the numerical gradient-search step in EM estimation of two-parameter Item Response Theory models with a simple least-squares fit of log-odds of expected response proportions. The authors argue this gives a closed-form, faster estimator, and show simulations comparing it with the standard mirt package.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The OLS M-step in Eqs. (31)-(32) does not solve the EM score equations (22)-(23); replacing the weighted sum by term-wise zero conditions is an unjustified extra requirement, so convergence to the MML solution is asserted rather than proven.","rationale":"The reader's weakest_assumption pinpoints the same derivation gap: the paper substitutes the actual M-step condition, a weighted sum equal to zero, with the stronger term-wise condition, then solves that over-determined system by OLS. My read agrees, and I would keep the REJECT verdict. The paper does provide a coherent revision of the complete-data likelihood and a proof-of-concept simulation, but the central methodological guarantee -- that the sequence of OLS estimators converges to the true 2PL parameters and that this is a closed-form solution of the MML first-order conditions -- is not established. The argument in Section 4.2 rests on Eq. (24), which is asymptotic and assumes Gamma^(n) already converges to Gamma, and on the unsupported statement that the estimators converge by construction. No fixed-point or monotonicity argument is supplied, and the simulation, besides using tuned quadrature counts and showing bias for extreme parameters, cannot substitute for a proof. The Appendix C observation that more quadrature points worsen the estimates is evidence against the claimed robustness of the construction. Given the high correctness risk of the central claim, rejection remains appropriate, but the idea is presented clearly and the simulation is a useful proof of concept.","tokens_in":16285,"tokens_out":6211,"duration_ms":59853,"concrete_test":"Run the EM-OLS routine on simulated 2PL data with N=100,000, a moderate item set (e.g., 10 items), and T=20 Gauss-Hermite quadrature points; compare converged estimates against mirt's MML estimates and the generating values, and record per-iteration Q(Gamma^(n+1)|Gamma^(n)) - Q(Gamma^(n)|Gamma^(n)). If the estimates deviate systematically from the MML solution or Q decreases at any iteration, the M-step replacement in Eqs. (26)-(32) is not solving the EM maximization and the convergence claim is refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on the step in Section 4.2 from Eq. (22), where the M-step score equation is a weighted sum over quadrature points equal to zero, to Eq. (26), where every term N1_jt - N_t P_j(theta_t, Gamma_j^(n)) is required to vanish. The MML solution only requires the sum to vanish; the term-wise condition is stronger, and Eq. (24) does not imply it. Eq. (24) shows that, conditional on Gamma^(n) approaching Gamma and on observed frequencies exactly matching true probabilities, the ratio N1_jt/N_t approaches P_j(theta_t, Gamma_j); it does not establish that the fixed point of the modified iteration satisfies Eq. (22). Because the term-wise system (26) is over-determined for T > 2, the OLS estimates (31)-(32) minimize a sum of squared log-odds residuals; they are not a maximizer of Q(Gamma|Gamma^(n)) and not a root of the score equations. Hence the sequence is not a standard EM algorithm: no monotonicity of Q or the observed log-likelihood is guaranteed, and the assertion that the sequence converges to the true values by construction is unsupported. The Appendix C finding that estimates degrade as quadrature points increase is consistent with this mismatch rather than a minor implementation detail. The paper's own caveat about practical identifiability further weakens the claim that the limit, if it exists, is the true Gamma rather than some other parameter vector.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper revisits the complete-data likelihood for 2-parameter Item Response Theory (IRT) models and proposes an EM-based procedure in which the M-step is replaced by an Ordinary Least Squares regression. Specifically, the authors define a latent response variable as the log-odds of posterior expected proportions of correct responses at each quadrature point, then estimate discrimination and difficulty parameters by regressing this variable on the quadrature points. They claim that this yields a sequence of closed-form estimators that converges to the true item parameters and that gradient search is unnecessary. The paper includes a didactic review of the complete-data likelihood, a simulation study comparing the proposed procedure with the mirt package for 1PL and 2PL models, and an appendix exploring the effect of the number of quadrature points.","tokens_in":16636,"tokens_out":4978,"duration_ms":43814,"significance":"If the central claim were correct, the paper would provide a practically useful alternative to gradient-based MML estimation in IRT, with potential computational benefits. The paper also offers a clear pedagogic restatement of the complete-data likelihood framework and includes a reasonably large simulation study. However, the central theoretical claim is not established: the OLS M-step does not solve the EM score equations, and no convergence proof is supplied. The paper's own simulation results, particularly the degradation with increased quadrature points in Appendix C, are more consistent with the proposed procedure solving a different optimization problem than with it approximating the MML solution. The didactic portions and the empirical comparison are useful, but the load-bearing theoretical contribution is unsupported.","major_comments":[{"comment":"The derivation is invalid at the step from Eq. (22) to Eq. (26). The score equation (22) is a weighted sum over quadrature points equal to zero; the MML solution requires only that this sum vanish. Eq. (26) imposes the much stronger condition that every summand N^{1,(n)}_{jt} - N^{(n)}_t P_j(\\theta_t, \\Gamma^{(n)}_j) vanish separately. Eq. (24) only shows that, under asymptotic frequency assumptions and in the limit \\Gamma^{(n)} \\to \\Gamma, the ratio N^{1,(n)}_{jt}/N^{(n)}_t approaches P_j(\\theta_t, \\Gamma_j); it does not imply that the term-wise condition holds at the fixed point of the proposed iteration. Consequently, the OLS estimates in Eqs. (31) and (32) are not roots of the score equations and do not maximize Q(\\Gamma|\\Gamma^{(n)}).","section":"Section 4.2, Eq. (26)"},{"comment":"The statement that 'by construction, the sequence of estimators ... converges to the true values' is not supported by any argument. The proposed algorithm is a fixed-point iteration whose M-step is an OLS fit to a dependent variable y^{(n)}_{jt} that itself depends on the current parameter estimates through the posterior in Eq. (18). No proof of convergence, contraction, or monotonicity of the Q-function or the observed likelihood is provided, and no theorem is cited that would apply to this modified EM. The convergence claim is therefore an assertion, not a result, and it is load-bearing for the paper's main thesis.","section":"Section 4.2, after Eq. (32)"},{"comment":"The numerical evidence in Appendix C is inconsistent with the paper's central claim. If Eqs. (31) and (32) were solving the M-step of an EM algorithm for the quadrature-approximated observed likelihood, increasing the number of quadrature points should improve the approximation and should not systematically degrade the estimates. The reported increase in bias and RMSE with the number of quadrature points suggests that the OLS step is solving a different problem, one that depends on the range of the quadrature points rather than on the likelihood. The authors' explanation in terms of practical identifiability does not account for this systematic pattern.","section":"Appendix C"},{"comment":"The construction is circular in a way that undermines the 'closed-form' claim. The latent response variable y^{(n)}_{jt} is defined as the log-odds of the posterior expected proportions N^{1,(n)}_{jt}/N^{(n)}_t, and those proportions are computed from the current item parameters \\Gamma^{(n)}. Regressing this variable on the quadrature points to recover a and \\tau therefore amounts to a self-consistency equation rather than an independent regression of an observed outcome on ability. The paper does not show that the solution of this self-consistency equation coincides with the marginal maximum likelihood estimator.","section":"Section 4.2, Eqs. (28)-(29)"}],"minor_comments":[{"comment":"There are several typographical errors: 'Gauss-Hermitee' appears in the reference list, 'the the 2PL model' appears in the Table 3 caption, and 'n quads' is used in Figures 4-6 and Appendix C without a definition.","section":"Throughout"},{"comment":"The notation for the total expected counts is inconsistent: Eq. (21) defines N^{(n)}_t with a superscript, but Eqs. (26)-(30) use N_t without the superscript. This makes it harder to follow which quantities are iteration-dependent.","section":"Eqs. (21), (26)-(30)"},{"comment":"The comparison with mirt does not state the quadrature settings, convergence criteria, or starting values used by mirt. Without these details, the reported accuracy and timing comparisons are difficult to interpret.","section":"Section 5"},{"comment":"The manuscript states that the R script is 'available upon request'; for a reproducibility-oriented journal, a public repository would be preferable.","section":"Section 5, code availability"},{"comment":"Equations (28) and (29) both define y^{(n)}_{jt} with the same left-hand side, but only Eq. (29) introduces an error term. The relation between the two equations should be stated more explicitly.","section":"Section 4.2, Eqs. (28)-(29)"}],"recommendation":"reject","confidential_remarks":"The paper's central theoretical claim is not supported by the derivation in Section 4.2. The step from the score equation (22) to the term-wise condition (26) is a non sequitur, and the claimed convergence of the OLS sequence is asserted rather than proven. This is a load-bearing error that cannot be repaired within the scope of the manuscript; a revised paper would need to reframe the procedure as a heuristic or study its fixed points as a separate contribution. The didactic review and simulation study may be useful, but they do not establish the stated result."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a clear, well-written presentation of an EM-based OLS estimator for 2PL item parameters, but the central claim is not established. The genuinely new part is the algorithm: use EM posterior proportions to build log-odds at each quadrature point, then update discrimination and difficulty by OLS. The review of complete-data likelihood and the connections to Lawley and Takane-De Leeuw are accurate and useful.\n\nWhere it stumbles is the proof. In Eq. (22), the M-step score equation is a weighted sum of phi terms over quadrature points set to zero. The paper then asserts, without proof, that the solution corresponds to every phi term being zero, and replaces the sum with the term-wise condition in Eq. (26). This is a much stronger requirement, and for T > 2 the system is over-determined, so the OLS step minimizes a sum of squared log-odds residuals rather than maximizing Q. The paper's claim that the sequence converges to the true values \"by construction\" is not supported. Eq. (24) only shows that if the parameter sequence happens to converge to the truth, the observed frequency ratio converges to the IRF; it does not prove that the fixed point of the OLS iteration solves the original score equations. The degradation with more quadrature points in Appendix C looks like a symptom of this mismatch, not a tuning issue.\n\nThe simulations are only a proof of concept, with quadrature counts chosen to look good, and biases appear for extreme parameters. The speed comparison with mirt is nice but secondary.\n\nStill, the idea is coherent and the write-up is honest about limitations like practical identifiability. It deserves a serious referee because it is a novel variant and the proof gap is exactly what peer review should surface. I would not cite it as a reliable method, but I might cite it as an interesting attempt. For a reading group on EM pitfalls, it would be a good case study.\n\nRecommendation: send it out, but expect the referee to ask for a real convergence analysis or a clear statement that this is a heuristic. As is, I would not accept.","headline":"A clearly written EM+OLS variant for 2PL whose convergence-to-MML claim is asserted, not proven, because it replaces the M-step score sum with a stronger term-wise zero condition.","tokens_in":17140,"tokens_out":2942,"would_cite":false,"duration_ms":28409,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F10","62P15"],"pacs":[],"model":"deepseek-v4-flash","headline":"In 2-parameter IRT models, EM iteration plus ordinary least squares yields closed-form estimators of discrimination and difficulty, so gradient search may be unnecessary.","keywords":["item response theory","2PL model","EM algorithm","closed-form estimation","ordinary least squares","marginal maximum likelihood","quadrature","latent response variable"],"falsifier":"Simulate a large-sample 2PL dataset with known parameters, run the EM-OLS routine and a standard gradient-search MML estimator from the same starting values, and compare the converged values; if the two fixed points differ by more than sampling error—especially for discrimination—the claimed equivalence between the pointwise-zero condition and the likelihood equations is refuted.","tokens_in":16104,"feed_emoji":"📊","tokens_out":6603,"duration_ms":56471,"temperature":0.7,"pith_summary":"Item response theory models with two parameters (discrimination and difficulty) are usually calibrated by numerical gradient search because maximum-likelihood item parameters have no closed form. This paper argues that if the EM algorithm is used, a closed form does exist: the discrimination and difficulty estimates at each M-step are simply the ordinary least squares regression of a latent response variable on ability. The latent response variable is built from the E-step proportions of correct responses at each quadrature point, converted to log-odds. If the argument is correct, 2PL item calibration reduces to iterating an E-step and a linear regression, and the sequence of OLS estimates converges to the true parameter values. A proof-of-concept simulation shows the resulting estimates are comparable to those from a standard calibration routine.","feed_headline":"EM plus OLS estimates 2PL item parameters without gradient search","feed_subtitle":"Iterated EM steps plus OLS give closed-form item estimates comparable to standard software.","key_machinery":"The load-bearing object is the pointwise residual $N_{jt}^{1,(n)}-N_t^{(n)}P_j(\\theta_t,\\Gamma_j^{(n)})$: the M-step's score equation is a weighted sum of such residuals across quadrature points, while the paper replaces that sum with the stronger requirement that every residual vanishes. That replacement yields the latent response variable $y_{jt}^{(n)}$ and the linear regression $y_{jt}^{(n)}=a_j^{(n)}\\theta_t+\\tau_j^{(n)}+\\epsilon_{jt}^{(n)}$, whose OLS solution is the closed-form estimator. The $\\phi$-terms introduced in Equation (22) are shown to converge to zero at the true parameters, which is what bridges the EM proportions to the item response function.","core_discovery":"The paper's central claim is that within an EM framework for 2-parameter logistic IRT models, the item parameters can be obtained in closed form by ordinary least squares. At iteration $n$, the E-step produces $N_{jt}^{1,(n)}$, the expected number of correct responses to item $j$ at quadrature point $\\theta_t$, and $N_t^{(n)}$, the expected number of respondents at that point, conditional on provisional parameters. The method forms the latent response variable $y_{jt}^{(n)}=\\log\\big(N_{jt}^{1,(n)}/(N_t^{(n)}-N_{jt}^{1,(n)})\\big)$ and regresses it on $\\theta_t$, giving OLS estimates for discrimination $a_j$ and threshold $\\tau_j$ (and hence difficulty $b_j=-\\tau_j/a_j$) via Equations (31) and (32). The authors argue that this sequence converges to the true parameters because, at the true solution, each term in the first-order conditions vanishes individually, so the M-step's usual weighted sum can be replaced by pointwise residuals that OLS minimizes. They call this a 'trivial' solution of the partial derivatives and note that it makes gradient search unnecessary.","pith_inferences":["If the fixed point really coincides with the MLE, the same E-step-plus-regression trick could generalize to multidimensional IRT models by replacing the univariate regression with a multivariate OLS on several ability dimensions.","The closed-form M-step opens a natural route to standard errors: one could apply OLS-based inference to the final latent response regression, a step the paper does not take.","A direct diagnostic would be to compare the response-pattern frequencies predicted by the OLS fixed point with those predicted by a gradient-search MML solution; the paper's practical-identifiability discussion suggests they need not be identical in finite samples."],"forward_implications":["2PL and 1PL item parameters can be estimated by iterating an EM E-step with an OLS M-step, with no Newton-Raphson or other gradient search.","Each EM iteration produces a simple linear regression, so the per-iteration computational cost is small; the paper's timing comparisons suggest faster runs than the reference routine, though the code is not optimized.","The choice of the number of quadrature points matters: in the simulations, two points work best for the 1PL and three to four for the 2PL, with larger counts adding bias and variance.","Because the estimators are closed-form at each step, they expose practical identifiability issues directly: with finite noisy data the sequence can converge to an alternative parameter set that generates frequencies close to the observed ones."],"supporting_citations":[{"why":"Supplies the quadrature-based EM procedure whose E-step proportions the paper reuses.","marker":"Bock & Aitkin, 1981"},{"why":"Introduces marginal maximum likelihood estimation for item parameters, the target that the closed-form sequence is claimed to match.","marker":"Bock & Lieberman, 1970"},{"why":"Provides the complete-data log-likelihood decomposition into $Q_1$ and $Q_2$ that the derivation starts from.","marker":"Woodruff & Hanson, 1996"},{"why":"Establishes the IRT-factor-analysis equivalence that motivates treating log-odds proportions as a latent response variable.","marker":"Takane & De Leeuw, 1987"},{"why":"The reference software whose item parameter estimates are compared in the simulation study.","marker":"Chalmers, 2012"},{"why":"Supplies the warning about instability in 3PL estimation that the paper invokes when discussing practical identifiability.","marker":"Mislevy, 1986"}],"fun_headline_variants":["Closed-form EM-OLS for 2PL IRT parameters","2PL item parameters via EM-OLS closed form","EM yields OLS closed-form for 2PL IRT","No gradient search: EM-OLS closed-form for IRT","OLS solves 2PL IRT inside EM iterations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole construction rests on assuming that requiring the model's item response function to match the observed proportion correct at every single ability-grid point, rather than only in the likelihood's weighted sum, still converges to the same maximum-likelihood estimates.","fun_headline_variants_meta":{"raw":{"variants":["Closed-form EM-OLS for 2PL IRT parameters","2PL item parameters via EM-OLS closed form","EM yields OLS closed-form for 2PL IRT","No gradient search: EM-OLS closed-form for IRT","OLS solves 2PL IRT inside EM iterations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000201,"raw_usage":{"total_tokens":1349,"prompt_tokens":888,"completion_tokens":461,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":504,"completion_tokens_details":{"reasoning_tokens":379}},"tokens_in":504,"tokens_out":461,"duration_ms":4409,"temperature":1.0,"reasoning_tokens":379,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:16:58.570512+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate a large-sample 2PL dataset with known parameters, run the EM-OLS routine and a standard gradient-search MML estimator from the same starting values, and compare the converged values; if the two fixed points differ by more than sampling error—especially for discrimination—the claimed equivalence between the pointwise-zero condition and the likelihood equations is refuted.","supporting_citations":[{"cited_title":"D., & Aitkin, M","cited_arxiv_id":null,"evidence_quote":"Supplies the quadrature-based EM procedure whose E-step proportions the paper reuses."},{"cited_title":"D., & Lieberman, M","cited_arxiv_id":null,"evidence_quote":"Introduces marginal maximum likelihood estimation for item parameters, the target that the closed-form sequence is claimed to match."},{"cited_title":"J., & Hanson, B","cited_arxiv_id":null,"evidence_quote":"Provides the complete-data log-likelihood decomposition into $Q_1$ and $Q_2$ that the derivation starts from."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the IRT-factor-analysis equivalence that motivates treating log-odds proportions as a latent response variable."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The reference software whose item parameter estimates are compared in the simulation study."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the warning about instability in 3PL estimation that the paper invokes when discussing practical identifiability."}],"review_version":1}