REVIEW 4 major objections 3 minor 2 references
A new three-step N-EM algorithm maximizes likelihoods containing integrals without hand-crafted latent variables, and it subsumes classic EM as a special case.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 03:38 UTC pith:BAKKMJ42
load-bearing objection N-EM is a repackaging of EM with the integration variable as missing data; the specific updates look useful, but the paper's central novelty claim and several technical details need major correction. the 4 major comments →
The normalized expectation-maximization (N-EM) algorithm
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
For likelihoods of the form f(x;θ)=c(x,θ)∫_S h(s|x,θ)ds with h≥0 and finite integral, the paper shows that normalizing h by its integral at the current parameter estimate yields a density g; using this g as the mixing measure makes Q(θ|θ^(t)) = Σ log c(x_i,θ) + Σ ∫ log h(s|x_i,θ) g(s|x_i,θ^(t)) ds a surrogate that lies below the log-likelihood and touches it at θ=θ^(t). Maximizing this Q therefore ascends the likelihood monotonically. The paper further shows this g is the best possible choice among all candidate densities, in the sense that it produces the optimal lower bound, and that the classical EM surrogate is recovered when the integrand is a complete-data joint density.
What carries the argument
The normalized density function (ndf): g(s|x,θ)=h(s|x,θ)/∫_S h(s'|x,θ) ds'. It converts the integral term into an expectation under a proper probability density, makes Jensen's inequality tight at the current iterate, and thereby turns maximum likelihood into the EM-style iteration θ^(t+1)=argmax_θ Q(θ|θ^(t)). It carries the whole argument: normalization is what manufactures the missing-data-like structure automatically.
Load-bearing premise
The convergence theorems rely on the surrogate Q-functions being strictly concave and differentiable, the negative log-likelihood being coercive, and stationary points being isolated; the paper asserts these hold for its examples without verifying them, and if any Q lacks strict concavity the claimed linear convergence does not follow.
What would settle it
Run the N-EM updates in (3.30) on a dataset drawn from the skew-normal-normal model and record the log-likelihood; if any iteration decreases the log-likelihood (beyond numerical tolerance), the ascent property is violated. Alternatively, numerically check strict concavity of Q(θ|θ^(t)) at a non-stationary point for the truncated-normal example; if a direction of zero curvature exists, Proposition 1's rate claim is void.
If this is right
- For any likelihood of the displayed integral form, an N-EM algorithm can be written down mechanically from the integrand, without designing latent variables.
- Classical EM surrogates are a special case: the conditional predictive density of the missing data is exactly the ndf of the complete-data joint density.
- The ascent property guarantees the observed log-likelihood never decreases across iterations, so the iteration is monotone under standard compactness/continuity conditions.
- The same framework handles finite homogeneous sums, which are discrete integrals, so models whose likelihood contains such sums also fit the recipe.
- If the local convergence conditions hold, the iteration converges linearly to a local maximizer with rate given by the spectral radius of the algorithm's map.
Where Pith is reading between the lines
- A natural test of the framework is whether the N-step can be applied to likelihoods whose integral is only computable by Monte Carlo; the paper does not develop this, but the surrogate construction would remain valid with an estimated g.
- Because the paper's optimality argument for the ndf is variational, the same Lagrange-multiplier device could be adapted to choose not only g but also data-dependent transformations of the integrand, potentially tightening the lower bound further.
- The N-EM view reframes EM's 'art' as a normalization step; if that view is right, it suggests textbook EM derivations can be taught as a mechanical integral-normalization exercise rather than as case-by-case tricks.
- For models with multiple or nested integral terms, the N-step could be applied componentwise; the paper does not explore this, but the convexity argument appears to carry through.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes an 'N-EM' algorithm for maximum likelihood estimation when the likelihood has the form f(x;θ)=c(x,θ)∫_S h(s|x,θ)ds. At iteration t one forms the normalized density g(s|x_i,θ^(t))=h(s|x_i,θ^(t))/∫_S h(s'|x_i,θ^(t))ds', constructs Q(θ|θ^(t))=∑_i log c(x_i,θ)+∑_i E_g log h(s|x_i,θ), and maximizes Q. The paper proves an ascent property via Jensen's inequality, argues that this choice of g is optimal among a class of lower-bound densities, and applies the recipe to several models: gamma-integral, skew normal-normal, skew t-normal, skew Laplace-normal, two-sided truncated normal, multivariate Laplace-type models, PIG data, and grouped data. Numerical experiments and convergence results are presented.
Significance. The core lower-bound derivation in Section 2.2 is correct and self-contained: Jensen's inequality produces a minorizing function, and the normalized g at θ^(t) achieves tangency, so the ascent property holds for the generic construction. The framework is a legitimate pedagogical unification of EM-type constructions. However, the construction is mathematically identical to ordinary EM in which the integration variable s is treated as the latent variable, so the paper's central selling point—that N-EM solves problems EM cannot—is not valid. In addition, the displayed surrogate Q1 in Eq. (3.29) contains sign errors that violate the tangency condition, so one flagship application does not satisfy even the basic ascent property. The numerical experiments are extensive but do not compare with baseline EM/MM algorithms, and the convergence claims rest on unverified regularity assumptions.
major comments (4)
- [Sections 1, 2.1, 2.4, and 3] The central claim that N-EM solves problems EM cannot is contradicted by the construction itself. For likelihood (2.1), define the complete-data density f(x,s;θ)=c(x,θ)h(s|x,θ). Then ∫_S f(x,s;θ)ds=f(x;θ), and the conditional density is f(s|x,θ^(t))=h(s|x,θ^(t))/∫_S h(s'|x,θ^(t))ds', which is exactly the N-step ndf (2.3). The Q-function in (2.4) is then exactly the standard EM Q-function (2.16) for this complete-data model. Thus every 'first time' application in Section 3 is an ordinary EM algorithm with latent variable s, and the abstract's assertion that N-EM solves problems 'which cannot be addressed by the EM algorithm' is unsupported. This invalidates the paper's stated principal contribution.
- [Section 3.2.1, Eq. (3.29)] Equation (3.29) does not satisfy the tangency condition (2.7). Evaluating Q1 at θ=θ^(t) does not reproduce ℓ1: the term -nσ_*^{2(t)} lacks the denominator 2σ_*²; the term n(μ-μ^(t))²/(2σ_*²) has the wrong sign; and the boundary correction term should involve (x_i-μ^(t)) rather than (x_i+μ^(t)-2μ), with additional σ_*^{4(t)}g_1^2 terms from the truncated-normal variance. The exact truncated-normal expectation gives -nσ_*^{2(t)}/(2σ_*²) - n(μ-μ^(t))²/(2σ_*²) + (1/(2σ_*²))∑_i σ_*^{2(t)}(x_i-μ^(t))g_1(...) + (1/(2σ_*²))∑_i σ_*^{4(t)}g_1(...)^2. As printed, Q1 is not tangent to ℓ1, so the monotone ascent property (2.9) is not guaranteed for the SNND update (3.30), and the numerical results for SNND in Table 1 are not backed by the stated theory.
- [Section 6, Propositions 1–2 and Corollaries 1–2] The convergence results are conditional on unverified hypotheses. The manuscript asserts that the surrogate functions in (3.29), (3.34), (3.39), (3.43), (4.45), (4.52), (4.58) and (4.55) are strictly concave, that -ℓ is coercive, and that stationary points are isolated, but no verification is supplied for any of the eight examples. For instance, for the skew-normal surrogate, even after correcting (3.29), global strict concavity in (μ,σ²,δ) through σ_*=σ/δ is not established; for the skew-t surrogate, the weights w_i^(t) and the log terms make strict concavity similarly non-obvious. The statement that 'the local convergence results follow directly by Proposition 1' is therefore not supported by the manuscript.
- [Section 5 (empirical comparisons)] The numerical experiments contain no comparison with existing EM algorithms, MM algorithms, or numerical quadrature/Newton-type methods for the same problems. Consequently, the claims that N-EM has 'superiority in relieving computational burden' or is efficient are not demonstrated. The simulations show convergence of the proposed updates, but they do not establish any advantage over existing methods, especially given that the updates are standard EM updates for the same latent-variable representations.
minor comments (3)
- [General organization] Section numbering is inconsistent: the applications in Section 3 are also referred to as Section 4 in the numerical section, and the finite homogeneous summation promised in the abstract and introduction does not appear as a developed section. The folded normal distribution (FND) is mentioned in Section 5 but not derived in the provided text.
- [Typos] There are numerous typos: 'constrcuting' (Section 1), 'calcualte' (Section 4.1.1), 'Tpye' (Table 2), 'in of' (Section 4.1.3), and 'regraded' (Section 2.4). These should be corrected.
- [Section 2.5.2] The constants c_0^(t), c_01^(t), c_02^(t) in inequalities (2.20)–(2.22) are not explicitly defined. This makes the display harder to check; please provide their definitions.
Circularity Check
Central novelty claim reduces to standard EM by the paper's own Section 2.4; the lower-bound derivation itself is not circular.
specific steps
-
renaming known result
[Abstract; Section 2.4, Eqs. (2.16)-(2.17); Section 3 introduction]
"we take g(z|Yobs,θ) as the normalized density of the joint density f(Yobs,z|θ) of the complete data, i.e., g(z|Yobs,θ)= f(Yobs,z|θ)/∫Z f(Yobs,z′|θ)dz′ = f(z|Yobs,θ), (2.17) which is exactly the conditional predictive density."
This identifies the N-step ndf with EM's conditional predictive density, and by (2.4) the N-EM Q-function with Q_EM in (2.16). Section 2.4 then states 'the original EM algorithm is a special case of the proposed N-EM algorithm.' Therefore every Section 3 example, presented as 'solved for the first time' and as evidence that N-EM 'can solve some problems which cannot be addressed by the EM algorithm,' is exactly ordinary EM with the integration variable s as the latent variable. The claimed new capability is a relabeling of the standard missing-data construction, not an independent result.
full rationale
The core N-EM construction is not circular in the usual prediction/fitting sense: Eq. (2.11)-(2.13) derive a Jensen lower bound and verify tangency algebraically, and the 'best ndf' claim follows from a genuine variational optimization over q in (2.14)-(2.15). That part is self-contained. However, the paper's central novelty claim is undermined by its own reduction: Eq. (2.17) equates the N-step g with EM's conditional predictive density, and (2.4) with (2.17) becomes the EM Q-function (2.16). Section 2.4 explicitly says EM is a special case of N-EM, so the Section 3 'solved for the first time' examples are standard EM algorithms with the integration variable as missing data. The abstract's assertion that N-EM solves problems EM cannot is therefore a renaming of known latent-variable EM rather than a new capability. I also note two non-circular rigor issues: Section 6 asserts strict concavity, coerciveness, and isolated stationary points for all eight examples without verification, and the printed Q1 in Eq. (3.29) appears to violate the tangency condition (2.7); these are correctness risks, not circularity.
Axiom & Free-Parameter Ledger
axioms (5)
- domain assumption The observed likelihood can be written as f(x;θ)=c(x,θ)∫_S h(s|x,θ)ds with h≥0 and finite integral (Eq. 2.1).
- standard math Jensen's integral inequality (2.10) and the supporting hyperplane inequality (2.18) are valid for the functions involved.
- domain assumption For convergence, surrogate Q functions are strictly concave and differentiable, and -ℓ is coercive with isolated stationary points (Propositions 1-2).
- domain assumption For skew-t-normal, the degrees of freedom ν is known (Section 3.2.2).
- domain assumption The normalized density g defined by (2.3) is a probability density, requiring ∫_S h(s|x_i,θ)ds < ∞ and h≥0.
read the original abstract
Although the $\textit{expectation-maximization}$ (EM) algorithm is a powerful optimization tool in statistics, it can only be applied to missing/incomplete data problems or to problems with a latent-variable structure. It is well known that the introduction of latent variables (or the data augmentation) is an art; i.e., it could only be done case by case. In this paper, we propose a new algorithm, a so-called $\textit{normalized EM}$ (N-EM) algorithm, for a class of log-likelihood functions with integrals. As an extension of the original EM algorithm, the N-EM algorithm inherits all advantages of EM-type algorithms and consists of three steps: normalization step (N-step), expectation step (E-step) and maximization step (M-step), where the N-step is to construct a $\textit{normalized density function}$ (ndf), the E-step is to compute a well-established surrogate $Q$-function and the M-step is to maximize the $Q$-function as in the original EM algorithm. The ascent property, the best choice of the ndf, and those N-EM algorithms with a difficult M-step are also explored. By multiple real applications, we have shown that the N-EM algorithm can solve some problems which cannot be addressed by the EM algorithm. Next, for problems to which the EM can be applied (often case by case), the N-EM algorithm can be employed in a unified framework. Numerical experiments are performed and convergence properties are also established.
Reference graph
Works this paper leans on
-
[1]
and Sullivan, B.J
Brailean, J.C., Little, D., Giger, M.L., Chen, C.T. and Sullivan, B.J. (1992). Application of the EM algorithm to radiographic images.Medical Physics19(5), 1175–1182. Cappe, O. (2011). Online EM algorithm for hidden Markov models.Journal of Computa- tional and Graphical Statistics20(3), 728–749. Casella, G. and George, E.I. (1992). Explaining the gibbs sa...
1992
-
[373]
and Fatehi, A
Ranjan, R., Huang, B. and Fatehi, A. (2016). Robust Gaussian process modeling using EM algorithm.Journal of Process Control42, 125-136. Ripley, B.D. (2009).Stochastic simulation.John Wiley & Sons, New York. Rubin, D.B. (1987). The calculation of posterior distributions by data augmentation, Com- ment, A non-iterative sampling/importance resampling alterna...
2016
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.