REVIEW 3 major objections 6 minor 17 references
Deep Generalised Mixed Models: a Novel Neural Network Structure for Analysing Hierarchical Data
T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read The Deep Generalised Mixed Model estimates both fixed and random effects with neural networks and imputes missing responses from the model's posterior predictive distribution, keeping inference valid under missing-at-random dropout.
desk verdict A useful neural extension of mixed models with a data-augmentation step for MAR dropout, but the headline claim of valid inference under MAR is asserted rather than demonstrated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the three-network DGMM: a fixed-effects network $h$, an encoder network $e$, and a decoder network $d$. The encoder outputs the means and variances of a normal variational posterior for the random effects, the decoder maps a draw $z$ (held constant for a subject) together with measurement time into the subject-specific part $\mu_2$, and the fixed plus random parts are summed and passed through link functions to form predictions for each outcome. Estimation maximizes the ELBO, with the KL term available in closed form because both the variational posterior and the prior are normal. For missing data, the load-bearing mechanism is the data augmentation algorithm in Section 2.3: each epoch imputes the missing responses by sampling from $p(Y_i^m \mid Y_i^o; \theta)$, implemented with an adaptive Metropolis-Hastings step, and then retrains on the completed input.
What would settle it
Simulate many datasets from a known linear mixed model with a documented MAR dropout rule, fit the DGMM with several random seeds and large sample sizes, and check whether the estimated fixed-effect curves and random-effect posteriors concentrate around the true values as subjects and within-subject measurements grow; if repeated runs remain scattered or imputation error does not approach the true model's imputation error, the valid-inference-under-MAR claim is refuted.
Extended reading notes
Core claim
The paper's claim is that the DGMM—a network in which $\mu_1(X_i, T_i)$ is a neural-net fixed-effects function and $\mu_2(b_i, T_i)$ is a neural-net random-effects function, with random effects $b_i \sim N(0,I)$—can serve as a generalised mixed model for longitudinal outcomes with arbitrary response distributions. Because the marginal likelihood is intractable, the model maximizes an evidence lower bound: Monte-Carlo approximation of the expected log-likelihood minus the closed-form KL divergence between the variational posterior $N(\mu_b(H_i;\phi), \Sigma_b(H_i;\phi))$ and the standard normal prior. Missing-at-random dropout is handled by a data augmentation loop that alternates between optimizing this bound on the currently imputed data and drawing new imputations of the missing responses from the posterior predictive distribution using a random-walk Metropolis-Hastings sampler. The paper asserts that this gives semi-parametric modelling of the mean and correlation structure, scalability to high-dimensional ESM data, and valid inference under MAR missingness.
Load-bearing premise
The load-bearing premise is that imputing missing responses from the model's own posterior predictive distribution, together with a normal approximation to the random-effects posterior, forms a valid EM-type estimation procedure under missing-at-random data, and the paper provides no convergence proof for this loop.
Editorial extensions
If this is right
- A single DGMM architecture can model longitudinal outcomes that follow different distributions (Gaussian, Student-$t$, gamma, beta, binomial, Poisson, negative binomial), removing the need to hand-specify a separate model for each outcome.
- The encoder pools all available measurements per subject, so the model can generate subject-specific dynamic predictions past the last observation while the data augmentation is designed to keep those predictions unbiased under MAR dropout.
- When training converges, the simulations indicate the DGMM recovers true trajectories with lower dynamic bias than an RNN under MAR mechanisms and can approach the predictive accuracy of the true data-generating mixed model.
- The architecture is not yet stable enough for routine use: simulation runs show large variance with occasional under- or overfitting, posterior collapse is a known failure mode, and on the GrowIt! data neither the DGMM nor the RNN forecasted individual negative states well.
- Because the decoder is a generic network, future users can replace its feed-forward layers with recurrent or convolutional layers, extending the same formulation to other forms of nested or hierarchical data.
Reading between the lines
- The normal variational posterior for the random effects is probably the weakest statistical link; replacing it with a heavier-tailed or more flexible variational family (for example a Student-$t$ or a normalizing-flow posterior) could reduce bias for the strongly skewed ESM outcomes.
- The large run-to-run variance suggests the stochastic EM-style update lacks a convergence diagnostic; pooling imputations across multiple seeds or adding a proper convergence check on the imputed-data bound could make the method usable before further architectural changes.
- The encoder's latent means can be read as low-dimensional embeddings of individual trajectories, so the same architecture could be extended to joint modelling of longitudinal and time-to-event data or to clustering patients by trajectory shape.
- A natural testable extension is to benchmark the DGMM against standard missing-data tools such as multiple imputation by chained equations or joint mixed models on the same MAR data, since those are established comparators the paper does not include.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes the Deep Generalised Mixed Model (DGMM), a variational-autoencoder-based neural network that generalizes mixed effects models by representing both fixed and random effects with neural networks. For missing longitudinal outcomes, the authors introduce a stochastic data-augmentation algorithm (Algorithm 1) that imputes missing values from the model's posterior predictive distribution, motivated by missing-at-random (MAR) drop-out in the GrowIt! experience sampling study. The method is applied to the GrowIt! data and evaluated in simulations against an LSTM RNN and the true linear mixed model, using RMSPE and dynamic bias. The paper reports that the DGMM shows potential but suffers from model instability and suboptimal performance in the motivating application.
Significance. If the advertised property of valid inference under MAR were rigorously established, the DGMM would be a meaningful contribution at the intersection of mixed models and deep learning, extending Mandel et al. (2021) by allowing both fixed and random effect parts to be nonlinear and by providing a principled missing-data mechanism. The paper is commendably honest about its limitations, explicitly reporting instability, hyperparameter sensitivity, and underperformance in the GrowIt! analysis. However, the significance is currently limited by (i) the absence of theoretical support for the central MAR inference claim, (ii) an evaluation that measures only predictive accuracy rather than inferential properties such as coverage, and (iii) the lack of uncertainty quantification in the reported comparisons. The core ideas are promising, but the evidence presented does not yet justify the abstract's strongest claims.
major comments (3)
- [Section 2.3, Eq. (2)] Equation (2) is internally inconsistent as printed: it states p(Y^m,_i^{(v)} | Y^o_i; θ^{(v)}) = ∫ p(Y^m,_i^{(v)} | Y^o_i; θ^{(v)}) p(b_i | Y^o_i; θ^{(v)}) db_i, meaning the left-hand side appears as a factor inside the integrand. The intended expression is presumably p(Y^m_i | Y^o_i; θ) = ∫ p(Y^m_i | b_i; θ) p(b_i | Y^o_i; θ) db_i, using the conditional-independence assumption of Eq. (1). Since Eq. (2) defines the imputation distribution used in Step 2 of Algorithm 1—the core of the MAR claim—this must be corrected and reconciled with the sampling algorithm in Web Appendix C.
- [Abstract and Section 5] The Abstract's claim that the DGMM 'provides valid inference when data are missing-at-random' is not established by the paper. Algorithm 1 is a stochastic EM-type loop, but its Step 1 performs a variational (approximate) M-step, and no convergence result is supplied to show that the fixed point of the loop maximizes the observed-data ELBO or marginal likelihood under MAR. In addition, the normal variational family for p(b_i | Y_i; θ) is only exact for Gaussian linear models; for generic outcome distributions F_k in Eq. (1) the approximation error is not quantified. The paper should either provide a theorem or a precise convergence argument, or the claim should be softened to predictive performance rather than valid frequentist inference.
- [Section 4] The simulation studies evaluate RMSPE and dynamic bias for point predictions (Eq. (4)) but never assess inferential properties such as parameter bias, interval coverage, or confidence-interval width for the fixed effects or random-effect covariance. Thus the headline claim of valid inference under MAR is never empirically tested. Moreover, Figure 4 shows large run-to-run variance, and the line graphs in Figure 5 are presented without confidence bands or standard errors, making it difficult to judge whether DGMM–RNN differences are meaningful. I recommend adding uncertainty measures for all reported metrics and, if the 'valid inference' claim is retained, including inferential evaluation such as coverage of the true trajectory or of model parameters.
minor comments (6)
- [Section 2.2] The closed-form KL divergence sums over k with terms μ_il and σ^2_il, but the latent dimension is u; please use a consistent index (e.g., l = 1, ..., u) and avoid reusing k, which already denotes the outcome index.
- [Section 2.4] In the dynamic prediction formula, the inner integral is over p(b_j | Y_j(t); θ) but written with db_i, and the outer integral over p(θ | D_n) uses dθ; please make the integration variables consistent (db_j and dθ) for readability.
- [Section 3] The application uses a simplified architecture that omits the fixed-effects network h and adds baseline covariates to the decoder, which differs from the general model in Section 2 and Figure 1. Please explain this simplification and its implications for identifiability of fixed versus random effects in the reported results.
- [Figures 2, 3, and 5] Several figure labels and captions contain garbled text (e.g., 'Sim)la(on S()dy I Res)l(s', 'T me ( u )'), which appears to be a rendering artifact; these need to be repaired before publication.
- [Web Appendices] Key implementation details, such as the inner steps of the Random Walk Metropolis–Hastings algorithm and the Robbins–Monro process in Step 2 of Algorithm 1, are relegated to Web Appendix C, which is not included in the manuscript version under review; the main text should summarize these steps or the appendices should be available for verification.
- [References] Reference 34 is cited as 'Adam, K. D. B. J. et al.'; the correct citation is Kingma, D. P. and Ba, J., 'Adam: A method for stochastic optimization,' which should be corrected.
Circularity Check
No significant circularity; the DGMM derivation is self-contained, with the unsupported MAR-inference claim being a correctness gap rather than a circular step.
full rationale
The paper's derivation chain is self-contained against its own inputs. The model is defined in Eq. (1) as a generalised mixed model with NN fixed and random effect functions; the ELBO in Section 2.2 is derived from the marginal likelihood by standard variational arguments; the normal variational family is stated as an approximation with its exactness condition given; and the missing-data routine in Algorithm 1 is a stochastic-EM-style self-consistency loop that imputes missing values from the model's own posterior predictive distribution. This is not circular: the imputed values are auxiliary training inputs, not the quantities being predicted or evaluated. Predictions are assessed on independent test sets via RMSPE and bias, so no fitted parameter is renamed as a prediction. No uniqueness theorem is imported from the authors' prior work, and the cited prior work by Mandel et al. is external to the authors. The equality of the ELBO with the marginal log-likelihood when q(b_i) equals the true posterior is a mathematical identity, not a reduction of the result to its inputs. Equation (2) as printed repeats the left-hand side inside the integrand, but this is a typographical error in the stated factorisation rather than a circular definition; the intended conditional independence factor p(Y^m_i | b_i; theta) p(b_i | Y^o_i; theta) is standard. The abstract's claim of 'valid inference when data are missing-at-random' is under-supported, because Algorithm 1 lacks a convergence proof and the simulations evaluate point predictions rather than interval coverage or estimator bias, but an unsupported claim is a correctness risk, not circularity. Accordingly, no specific circular step can be quoted and exhibited, and the appropriate score is 0.
Assumptions & free parameters
free parameters (4)
- Latent dimension u =
5 (GrowIt! analysis)
- Encoder architecture =
4 hidden layers, 64 units each
- Decoder architecture =
2 hidden layers, 48 units each
- KL annealing schedule =
Cyclic KL-annealing with beta-VAE weighting; warm-up before imputation
assumptions (5)
- domain assumption Outcomes are conditionally independent given random effects bi
- domain assumption Random effects follow N(0, I)
- domain assumption Missingness is missing-at-random (MAR) and the set of time points is pre-specified
- standard math Posterior p(bi | Yi; theta) can be approximated by a multivariate normal (Bayesian CLT)
- standard math ELBO maximization is equivalent to maximizing the marginal log-likelihood
Cite this review
Pith. "Pith review of Deep Generalised Mixed Models: a Novel Neural Network Structure for Analysing Hierarchical Data." pith.science (2026). https://pith.science/paper/Y3PUEHG3
@misc{pith2026260805930,
author = {Pith},
title = {Pith review of: Deep Generalised Mixed Models: a Novel Neural Network Structure for Analysing Hierarchical Data},
year = {2026},
howpublished = {\url{https://pith.science/paper/Y3PUEHG3}},
note = {Machine review of arXiv:2608.05930}
}
read the original abstract
The experience sampling method (ESM) is a longitudinal research design where participants report their thoughts, emotional states and behaviours multiple times a day. Our work is motivated by such data collected by the GrowIt! app, which was released to investigate daily emotions among adolescents during the COVID-19 pandemic. Current procedures to analyse ESM data face various challenges. While standard statistical techniques may not scale well to a high-dimensional setting, machine learning procedures can give biased results due to selection bias introduced by missingness. In our motivating dataset, adolescents dropped out due to previous strong feelings of negative emotions. Hence, the implied missing data are of the missing-at-random type that standard machine learning procedures cannot accommodate. We develop a novel neural network architecture that generalises mixed effects models to deep learning to overcome these challenges. It allows semi-parametric and flexible modelling of data's mean and correlation structure through fixed and random effects. For estimation, we use an adaptation of variational auto-encoders and a Bayesian data augmentation algorithm. Through this approach, the model can accommodate longitudinal outcomes following generic distributions, scale well to high-dimensional settings and provide valid inference when data are missing-at-random. We applied the Deep Generalised Mixed Model to the GrowIt! study and various simulations. The results show potential for the Deep Generalised Mixed Model, yet suboptimal performance due to model instability.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
The Grow It! app— longitudinal changes in adolescent well-being during the COVID-19 pandemic: a proof-of-concept study
(1) Dietvorst, E; Legerstee, J.; Vreeker, A; Koval, S; Mens, M.; Keijsers, L; Hillegers, M. The Grow It! app— longitudinal changes in adolescent well-being during the COVID-19 pandemic: a proof-of-concept study. European Child & Adolescent Psychiatry 2023, 32, 1097–1107. 18 (2) Dietvorst, E.; Hillegers, M. H.; Legerstee, J. S.; Vries, L. P. D.; Vreeker, A...
2023
-
[4]
New developments in experience sampling methodology
(11) Tuerlinckx, F.; Kuppens, P.; Ariens, S.; Cloos, L.; Dejonckheere, E.; Lafit, G.; Niemeijer, K.; Revol, J.; Schat, E.; Schreuder, M., et al. New developments in experience sampling methodology. British Journal of Mathematical and Statistical Psychology 2026, 79, 46–65. (12) Prince, S. J., Understanding deep learning ; MIT press:
2026
-
[8]
A solution for missing data in recurrent neural networks with an application to blood glucose prediction
(17) Tresp, V.; Briegel, T. A solution for missing data in recurrent neural networks with an application to blood glucose prediction. Advances in Neural Information Processing Systems 1997,
1997
-
[10]
Speech recognition with missing data using recurrent neural nets
(18) Parveen, S.; Green, P. Speech recognition with missing data using recurrent neural nets. Advances in Neural Information Processing Systems 2001,
2001
-
[14]
C.; Kale, D.; Wetzel, R
(19) Lipton, Z. C.; Kale, D.; Wetzel, R. In Machine learning for healthcare conference , 2016, pp 253–270. (20) Choi, E.; Bahadori, M. T.; Schuetz, A.; Stewart, W. F.; Sun, J. In Machine learning for healthcare conference, 2016, pp 301–318. (21) Le Morvan, M.; Josse, J.; Scornet, E.; Varoquaux, G. What’s a good imputation to predict with missing values? A...
2016
-
[80]
Recurrent neural networks for missing or asynchronous data
(16) Bengio, Y.; Gingras, F. Recurrent neural networks for missing or asynchronous data. Advances in neural information processing systems 1995,
1995
-
[1412]
Cyclical annealing schedule: A simple approach to mitigating kl vanishing
(35) Fu, H.; Li, C.; Liu, X.; Gao, J.; Celikyilmaz, A.; Carin, L. Cyclical annealing schedule: A simple approach to mitigating kl vanishing. arXiv preprint arXiv:1903.10145
arXiv 1903
-
[1979]
A new class of stochastic EM algorithms
(32) Allassonnière, S.; Chevallier, J. A new class of stochastic EM algorithms. Escaping local maxima and handling intractable sampling. Computational Statistics & Data Analysis 2021, 159, 107159. (33) Garthwaite, P. H.; Fan, Y.; Sisson, S. A. Adaptive optimal scaling of Metropolis–Hastings algorithms using the Robbins–Monro process. Communications in Sta...
2021
Show all 17 references
-
[2006]
M.; Slasor, P
(7) Cnaan, A.; Laird, N. M.; Slasor, P. Using the general linear mixed model to analyse unbalanced repeated measures and longitudinal data. Statistics in medicine 1997, 16, 2349–2380. (8) Parekh, P.; Fan, C. C.; Frei, O.; Palmer, C. E.; Smith, D. M.; Makowski, C.; Iversen, J. ...
1997
-
[2013]
20 (30) Kullback, S.; Leibler, R. A. On information and sufficiency. The annals of mathematical statistics 1951, 22, 79–86. (31) Cox, D. R.; Hinkley, D. V., Theoretical statistics; CRC Press:
1951
-
[2016]
P.; Welling, M
(29) Kingma, D. P.; Welling, M. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114
-
[2018]
Long short-term memory
(37) Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural computation 1997, 9, 1735–1780. (38) Wang, L.; Miller, L. C. Just-in-the-moment adaptive interventions (JITAI): a meta-analytical review. Health Communication 2020, 35, 1531–1544. (39) Wang, Y.; Blei, D.; Cunn...
1997 arXiv
-
[2019]
P.; Higgins, I.; Pal, A.; Matthey, L.; Watters, N.; Desjardins, G.; Lerchner, A
(36) Burgess, C. P.; Higgins, I.; Pal, A.; Matthey, L.; Watters, N.; Desjardins, G.; Lerchner, A. Understanding disentangling in β-V AE.arXiv preprint arXiv:1804.03599
-
[2023]
Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network
(13) Sherstinsky, A. Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network. Physica D: Nonlinear Phenomena 2020, 404, 132306. (14) Chen, Y.; Kang, Y.; Chen, Y.; Wang, Z. Probabilistic forecasting with temporal convolutional neural network. Ne...
2020
-
[2024]
T.; Li, H
(9) Li, S.; Cai, T. T.; Li, H. Inference for high-dimensional linear mixed-effects models: A quasi-likelihood approach. Journal of the American Statistical Association 2022, 117, 1835–1846. (10) Ong, P.; Haußmann, M.; Lönnroth, O.; Lähdesmäki, H. Latent mixed-effect models for...
2022 arXiv
-
[5111]
(34) Adam, K. D. B. J. et al. A method for stochastic optimization. arXiv preprint arXiv:1412.6980 2014,
2014 arXiv
-
[6085]
P.; Barnett, I
(26) Mandel, F.; Ghosh, R. P.; Barnett, I. Neural networks for clustered and longitudinal data using mixed effects models. Biometrics 2023, 79, 711–721. (27) Kingma, D. P.; Welling, M., et al. An introduction to variational autoencoders. Foundations and Trends® in Machine Lear...
2023
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.