REVIEW 3 major objections 5 minor 32 references
Maximum Likelihood for Logistic Regression Model with Incomplete and Hybrid-Type Covariates
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims a stochastic-approximation EM algorithm for logistic regression with missing mixed-type covariates estimates regression coefficients and covariate distributions jointly, and beats standard imputation pipelines on bias…
desk verdict A plausible SAEM recipe for logistic regression with missing mixed-type covariates, backed by code and simulations, but the headline claim of consistent superiority is contradicted by its own tables and the key independence assumption is untested. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is SAEM (Stochastic Approximation Expectation-Maximization), driven by the complete-data factorization $p(y_i, x_i; \theta) = p(y_i \mid x_i; \beta) \, p(x_i^c; \theta_c) \, p(x_i^d; \theta_d)$. The S-step samples missing continuous covariates by Metropolis-Hastings from the conditional distribution that sums over missing discrete values; the SA-step updates a stochastic approximation of the expected complete log-likelihood component-wise with smoothing parameters $\delta_t$; and the M-step maximizes each component separately. The workhorse identities are the closed-form updates for the Gaussian sufficient statistics $T_1$ and $T_2$ and for the multinomial probabilities, which keep the stochastic approximation exact in those blocks.
What would settle it
Generate a dataset in which the discrete and continuous covariates are strongly correlated, delete values under a mechanism that depends on both, and compare SAEM against MICE on many replications; if SAEM's bias and RMSE are not lower than MICE's in that setting, the independence factorization is the point of failure. A simpler check is to fit the paper's model to a real mixed dataset and compare the implied joint distribution of the covariates with the empirical distribution.
Extended reading notes
Core claim
The central claim is that the complete-data likelihood separates into a logistic term for the binary outcome, a multivariate Gaussian term for the continuous covariates, and a product of independent multinomial terms for the discrete covariates. Under this factorization, the E-step of EM can be replaced by a stochastic approximation in which missing continuous values are drawn from their conditional distribution by Metropolis-Hastings, missing discrete values are averaged with posterior weights, and the M-step splits into three separate updates: gradient ascent for the regression coefficients, closed-form sufficient-statistic updates for the Gaussian mean and covariance, and analytic updates for the multinomial probabilities. The paper asserts that this algorithm, run with a decreasing smoothing sequence, yields maximum-likelihood-type estimates under ignorable missingness and that in simulations it consistently has lower bias and RMSE for the coefficients and better test-set AUC, accuracy, and Brier scores than MICE, MissForest, mean/mode imputation, and complete-case analysis. Prediction with missing test covariates follows the same model by marginalizing over the missing parts, using Monte Carlo draws for continuous values and exact sums for discrete ones.
Load-bearing premise
The load-bearing premise is that, conditional on the regression, each sample's continuous and discrete covariates are independent of each other and the discrete covariates are mutually independent, so the joint covariate distribution factorizes into one Gaussian and a product of multinomials.
Editorial extensions
If this is right
- For MCAR and MAR missingness in mixed covariates, the regression coefficient estimates from SAEM have lower bias and RMSE than MICE, MissForest, mean/mode imputation, and complete-case analysis in the simulated settings tested.
- Prediction on test data with missing covariates can be made directly from the fitted joint model, without a separate imputation step, by marginalizing over the missing values.
- The discrete-covariate parameters (multinomial probabilities) and continuous-covariate parameters (mean and covariance) are estimated in the same run at negligible extra cost, because their M-steps are closed form.
- The method remains usable at high missingness rates: reported results include 50% and 60% MCAR and 30% to 40% MAR, with SAEM maintaining the highest AUC, accuracy, and lowest Brier score in those comparisons.
- Testing-phase computation for SAEM is much faster than MissForest in the reported timings, for example about two seconds versus thirty-eight seconds in the largest configuration.
Reading between the lines
- Editorial inference: because the covariate-distribution block does not depend on the form of the outcome model, the same SAEM skeleton should transfer to probit, Poisson, or other generalized linear models; only the update for the outcome coefficients would change.
- Editorial inference: under the independence factorization, the method's advantage should shrink when discrete and continuous covariates are strongly dependent; a user with such data would need a coupled covariate model, which the paper does not provide.
- Editorial inference: a practical benchmark implied by the paper is that gains over MissForest should be largest at high missingness rates and with informative covariates; users with weak covariates and low missingness may see near-tied performance.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a Stochastic Approximation Expectation-Maximization (SAEM) algorithm for maximum-likelihood estimation in logistic regression with missing covariates of mixed continuous and discrete type. The complete-data model assumes multivariate Gaussian continuous covariates, independent multinomial discrete covariates, and full independence between the continuous and discrete blocks. The E-step uses Metropolis–Hastings to sample missing continuous covariates, the SA-step updates stochastic sufficient statistics, and the M-step updates the logistic coefficients, Gaussian parameters, and multinomial probabilities component-wise. The method is compared with MICE, MissForest, mean/mode imputation, and complete-case analysis on synthetic and four real datasets, with the abstract and conclusion claiming consistent superiority in accuracy and reliability.
Significance. If the method performed as claimed, it would provide a useful joint-modeling alternative to separate imputation plus logistic regression for hybrid missing covariates under ignorable missingness. The manuscript has strengths: the source code is provided, the simulation studies compare against ground truth and external baselines, and the likelihood objective is standard and clearly stated. However, the paper's central claim of consistent outperformance is not supported by its own numerical results, and the core independence assumption between continuous and discrete covariates is never tested in settings where that assumption fails. The contribution is therefore only partially validated and needs substantial revision before the performance claims can be accepted.
major comments (3)
- [§IV and Supplementary Table III] The abstract and conclusion claim that the proposed SAEM 'consistently outperforms traditional methods.' This is contradicted by the paper's own numbers: in Supplementary Table III, under 50% MCAR, SAEM has RMSE 0.4397 for β1, while MissForest achieves 0.3323 and the full-data reference 0.2618; SAEM also loses to MissForest on RMSE for β3 (0.1354 vs 0.0906), β4 (0.0860 vs 0.0689), and β7 (0.0961 vs 0.0909). SAEM has smaller bias for β1 but larger variance, so the broad claim is already overstated in the favorable independent-covariate setting. Please revise the superiority claims or provide a statistical comparison across the 100 runs that supports the stated conclusion.
- [§II and Supplementary Eq. (18)] The complete-data model factorizes as p(y_i, x_i; θ) = p(y_i | x_i; β) p(xc_i; θc) p(xd_i; θd), which imposes independence between continuous and discrete covariates and mutual independence among discrete covariates. This factorization drives the E-step in Eq. (4), the Metropolis–Hastings target in Supplementary Eq. (15), the discrete posterior weights, the stochastic-approximation sufficient statistics, and the prediction rule in Section III-B. The synthetic experiments in Section IV generate discrete and continuous covariates independently, so they provide no evidence about the realistic regime in which, for example, a discrete covariate such as sex or smoking status affects the mean of a continuous biomarker. Under such dependence, the MH sampler targets p(xc_mis | xc_obs, y) rather than p(xc_mis | xc_obs, xd_obs, y), and the discrete imputation weights ignore observed continuous covariates; both can bias β even under MCAR/MAR missingness. Please add simulation scenarios with dependence between the covariate blocks and a sensitivity analysis for the real data.
- [§IV, smoothing schedule] The description of the SAEM step-size sequence is internally inconsistent. The text states that δ_t = (t + t1)^{-τ} with t1 = 50, but also states that δ_t = 1 during the first 50 iterations. At t = 50, the formula gives (100)^{-1} = 0.01, not 1. Please specify the exact schedule, including how the transition from the warm-up phase to the decreasing phase is implemented, so that the experiments are reproducible.
minor comments (5)
- [Abstract and §V] The phrases 'consistently outperforms' and 'demonstrate that the SAEM approach outperforms classical techniques' should be tempered to reflect the scenarios actually tested, in light of the RMSE results in Supplementary Table III.
- [Eq. (2)] The displayed complete log-likelihood appears to have an extra closing parenthesis at the end of the Gaussian term; please check the formula and correct the typographical error.
- [Supplementary Table III caption] The caption states that the best values are highlighted in bold, but the table as printed contains no bold entries; this should be fixed or the caption amended.
- [§IV] The phrase 'almost 30 faster' should read 'almost 30 times faster'.
- [Supplementary Materials, dataset descriptions] The French phrase 'dataset utilis´e dans R' is a leftover fragment; it should be translated or removed.
Circularity Check
No significant circularity: the derivation follows from an explicitly stated complete-data factorization and is validated against external baselines, not from self-citation or fitted inputs.
full rationale
The central derivation is self-contained. Equation (18) states the complete-data factorization p(y_i,x_i;theta)=p(y_i|x_i;beta)p(xc_i;theta_c)p(xd_i;theta_d) as an explicit modeling assumption, and the E-step (Eq. 4), MH target (Supplementary Eq. 15), component-wise stochastic approximation updates (Eqs. 6-11), and prediction rule (Section III-B) all follow algebraically from that factorization plus standard SAEM convergence results cited to external works [21,25]. The performance claim is tested against synthetic data with known beta and against MICE, MissForest, mean/mode, and complete-case baselines on real datasets; no fitted parameter from those experiments is renamed as a prediction, nor is any external benchmark imported from the authors' own prior work. The only self-citations ([17] in a literature survey and [28] for source code) are not load-bearing: the code link is for reproducibility, and the convergence of SAEM is attributed to Delyon et al. and Kuhn and Lavielle, not to the authors' own papers. The factorization in (18) is a substantive assumption that may fail on dependent hybrid data, but that is a misspecification and generalizability risk, not a circular step: the paper does not define its target in terms of its output. The simulations being generated under the method's own independence model is a standard validation design, and it does not make the estimator's output equal to its input. Accordingly, no circular step is identified.
Assumptions & free parameters
free parameters (3)
- SAEM smoothing schedule (t1, tau, warm-up) =
t1=50, tau=1, delta_t=1 for first 50 iterations
- Metropolis-Hastings chain length and burn-in S =
not reported
- Prediction Monte Carlo sample size S =
not reported
assumptions (5)
- domain assumption Missingness mechanism is ignorable (MCAR/MAR)
- domain assumption Continuous and discrete covariates are independent, and discrete covariates are mutually independent
- domain assumption Continuous covariates follow a multivariate Gaussian and each discrete covariate follows an independent multinomial distribution
- standard math SAEM convergence conditions from Delyon et al. and Kuhn-Lavielle hold for this model with the MH sampler
- domain assumption Training and test covariates share the same distribution
Cite this review
Pith. "Pith review of Maximum Likelihood for Logistic Regression Model with Incomplete and Hybrid-Type Covariates." pith.science (2026). https://pith.science/paper/AOBEWV5I
@misc{pith2026250603445,
author = {Pith},
title = {Pith review of: Maximum Likelihood for Logistic Regression Model with Incomplete and Hybrid-Type Covariates},
year = {2026},
howpublished = {\url{https://pith.science/paper/AOBEWV5I}},
note = {Machine review of arXiv:2506.03445}
}
read the original abstract
Logistic regression is a fundamental and widely used statistical method for modeling binary outcomes based on covariates. However, the presence of missing data, particularly in settings involving hybrid covariates (a mix of discrete and continuous variables), poses significant challenges. In this paper, we propose a novel Expectation-Maximization based algorithm tailored for parameter estimation in logistic regression models with missing hybrid covariates. The proposed method is specifically designed to handle these complexities, delivering efficient parameter estimates. Through comprehensive simulations and real-world application, we demonstrate that our approach consistently outperforms traditional methods, achieving superior accuracy and reliability.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[16]
W. Jiang, J. Josse, M. Lavielle, T. Group et al., “Logistic regression with missing covari- ates—parameter estimation, model selection and predictio n within a joint-modeling framework,” Computational Statistics & Data Analysis, vol. 145, p. 106907, 2020
work page 2020
-
[1]
J. W. Osborne, Best practices in logistic regression. Sage Publications, 2014
work page 2014
-
[2]
A logistic regress ion approach to field estimation using binary measurements,
A. S. Leong, M. Zamani, and I. Shames, “A logistic regress ion approach to field estimation using binary measurements,” IEEE Signal Processing Letters, vol. 29, pp. 1848–1852, 2022
work page 2022
-
[3]
Modified multi-directi on iterative algorithm for separable nonlinear models with missing data,
J. Chen, M. Hu, Y . Mao, and Q. Zhu, “Modified multi-directi on iterative algorithm for separable nonlinear models with missing data,” IEEE Signal Processing Letters, vol. 29, pp. 1968–1972, 2022
work page 1968
-
[4]
What do we do with missing data? some o ptions for analysis of incomplete data,
T. E. Raghunathan, “What do we do with missing data? some o ptions for analysis of incomplete data,” Annu. Rev. Public Health, vol. 25, no. 1, pp. 99–117, 2004
work page 2004
-
[5]
Robust subspace tracking with missing data and outliers: Novel algorithm with convergence guarantee,
N. V . Dung, N. L. Trung, K. Abed-Meraim et al., “Robust subspace tracking with missing data and outliers: Novel algorithm with convergence guarantee,” IEEE Transactions on Signal Processing, vol. 69, pp. 2070–2085, 2021
work page 2021
-
[6]
P . D. Allison, “Missing data,” The SAGE handbook of quantitative methods in psychology, vol. 23, pp. 72–89, 2009
work page 2009
-
[7]
A. Hippert-Ferrer, M. El Korso, A. Breloy, and G. Ginolha c, “Robust low-rank covariance matrix September 16, 2025 DRAFT 11 estimation with a general pattern of missing values,” Signal Processing, vol. 195, p. 108460, 2022
work page 2025
Show all 32 references
-
[8]
A principal compon ent method to impute missing values for mixed data,
V . Audigier, F. Husson, and J. Josse, “A principal compon ent method to impute missing values for mixed data,” Advances in Data Analysis and Classification, vol. 10, pp. 5–26, 2016
2016
-
[9]
Multivariate correlation model s with mixed discrete and continuous variables,
I. Olkin and R. F. Tate, “Multivariate correlation model s with mixed discrete and continuous variables,” The Annals of Mathematical Statistics, pp. 448–465, 1961
1961
-
[10]
Maximum likelihood e stimation for mixed continuous and categorical data with missing values,
R. J. Little and M. D. Schluchter, “Maximum likelihood e stimation for mixed continuous and categorical data with missing values,” Biometrika, vol. 72, no. 3, pp. 497–512, 1985
1985
-
[11]
Maximum lik elihood from incomplete data via the em algorithm,
A. P . Dempster, N. M. Laird, and D. B. Rubin, “Maximum lik elihood from incomplete data via the em algorithm,” Journal of the royal statistical society: series B (methodological), vol. 39, no. 1, pp. 1–22, 1977
1977
-
[12]
Parameter e stimation of heavy-tailed ar model with missing data via stochastic EM,
R. Zhou, J. Liu, S. Kumar, and D. P . Palomar, “Parameter e stimation of heavy-tailed ar model with missing data via stochastic EM,” IEEE Transactions on Signal Processing, vol. 67, pp. 2159–2172, 2019
2019
-
[13]
Missforest—non-parametric missing value imputation for mixed- type data,
D. J. Stekhoven and P . B¨ uhlmann, “Missforest—non-parametric missing value imputation for mixed- type data,” Bioinformatics, vol. 28, no. 1, pp. 112–118, 2012
2012
-
[14]
Logistic regression with missing res ponses and predictors: a review of existing approaches and a case study,
S. Martins, J. ´Alvarez, and I.-P . M, “Logistic regression with missing res ponses and predictors: a review of existing approaches and a case study,” arXiv, p. 2302.03435, 2024
2024 arXiv
-
[15]
On the application of generali zed linear mixed models for predicting path loss in lte networks,
A. Cohen and Y . Alqudah, “On the application of generali zed linear mixed models for predicting path loss in lte networks,” Journal on Advances in Signal Processing, p. 6, 2023
2023
-
[17]
Robust inference with incompleteness for logistic regression model,
M. Cherifi, M. El Korso, S. Fortunati, A. Mesloub, and L. F erro-Famil, “Robust inference with incompleteness for logistic regression model,” Signal Processing, p. 110027, 2025
2025
-
[18]
High-dimensional logisti c regression with missing data: Imputation, regularization, and universality,
K. Aladin and A. Montanari, “High-dimensional logisti c regression with missing data: Imputation, regularization, and universality,” arXiv, p. 2410.01093, 2025
2025 arXiv
-
[19]
Deeply lear ned generalized linear models with missing data,
D. Lim, N. Rashid, J. Oliva, and J. Ibrahim, “Deeply lear ned generalized linear models with missing data,” Journal of Computational and Graphical Statistics, 2024
2024
-
[20]
Improving penaliz ed logistic regression model with missing values in high-dimensional data,
A. Alharthi, M. Lee, and Z. Algamal, “Improving penaliz ed logistic regression model with missing values in high-dimensional data,” International Journal of Online & Biomedical Engineering, 2022
2022
-
[21]
Convergence o f a stochastic approximation version of the em algorithm,
B. Delyon, M. Lavielle, and E. Moulines, “Convergence o f a stochastic approximation version of the em algorithm,” Annals of statistics, pp. 94–128, 1999
1999
-
[22]
Student’s t var modeling with missing data via September 16, 2025 DRAFT 12 stochastic em and gibbs sampling,
R. Zhou, J. Liu, S. Kumar, and D. P . Palomar, “Student’s t var modeling with missing data via September 16, 2025 DRAFT 12 stochastic em and gibbs sampling,” IEEE Transactions on Signal Processing, vol. 68, pp. 6198– 6211, 2020
2025
-
[23]
Dynamic expectation-maximization algorithms for mixed-type data,
S. Pruilh and S. Allassonni` ere, “Dynamic expectation-maximization algorithms for mixed-type data,” 2024
2024
-
[24]
Missing co variates in generalized linear models when the missing data mechanism is non-ignorable,
J. G. Ibrahim, S. R. Lipsitz, and M.-H. Chen, “Missing co variates in generalized linear models when the missing data mechanism is non-ignorable,” Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 61, no. 1, pp. 173–190, 1999
1999
-
[25]
Coupling a stochastic approxi mation version of emwith an mcmc procedure,
E. Kuhn and M. Lavielle, “Coupling a stochastic approxi mation version of emwith an mcmc procedure,” ESAIM: Probability and Statistics, vol. 8, pp. 115–131, 2004
2004
-
[26]
mice: Multi variate imputation by chained equations in r,
S. V an Buuren and K. Groothuis-Oudshoorn, “mice: Multi variate imputation by chained equations in r,” Journal of statistical software, vol. 45, pp. 1–67, 2011
2011
-
[27]
Missing data imputation: focusing on single imputation,
Z. Zhang, “Missing data imputation: focusing on single imputation,” Annals of translational medicine, vol. 4, no. 1, p. 9, 2016
2016
-
[28]
Saem al gorithm for mixed data imputation (binary/categorical/continuous),
M. Cherifi, X. Zhu, M. N. El Korso, and A. Mesloub, “Saem al gorithm for mixed data imputation (binary/categorical/continuous),” 2025 , release v1.0. [Online]. Available: https://github.com/wolf90704/saem-mixed-imputation
2025
-
[29]
Heart Disease,
A. Janosi, W. Steinbrunn, M. Pfisterer, and R. Detrano, “ Heart Disease,” UCI Machine Learning Repository, 1989, DOI: https://doi.org/10.24432/C52P4X
1989 doi
-
[30]
Algerian Forest Fires,
F. Abid, “Algerian Forest Fires,” UCI Machine Learning Repository, 2019, DOI: https://doi.org/10.24432/C5KW4N
2019 doi
-
[31]
Titanic: Machine learning from dis aster,
Kaggle Community, “Titanic: Machine learning from dis aster,” 2012, accessed: 2025-04-07. [Online]. Available: https://www.kaggle.com/c/titanic /data
2012
-
[32]
The 1974 motor trend car r oad tests,
H. Henderson and P . V elleman, “The 1974 motor trend car r oad tests,” Motor Trend Magazine, 1974, dataset utilis´ e dans R sous le nom mtcars. September 16, 2025 DRAFT 13 Supplementary Materials S.I. M ETROPOLIS –H ASTINGS ALGORITHM In logistic regression for mixed data, it ...
1974
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.