{"id":"a4cbee96-7200-4728-8e61-32337a9e93f9","arxiv_id":"2608.05930","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"The Deep Generalised Mixed Model extends mixed-effects models to deep learning for longitudinal data with missing-at-random dropout, but current results are unstable and often behind a simpler RNN.","lead":"Researchers built a neural network that combines mixed-effects models with deep learning to analyze longitudinal data with missing values. The method shows promise in simulations but is currently unstable and underperforms a recurrent neural network on real app data.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The advertised 'valid inference under MAR' is the load-bearing claim, and it is unsupported: Algorithm 1 has no convergence guarantee for its stochastic VAE data augmentation, and the paper never evaluates inference (bias, coverage), only point predictions.","rationale":"This is the single most load-bearing concern because the paper's contribution over prior mixed-model deep learning (Mandel et al.) is precisely the missing-data generalisation, and the Abstract's strongest statement is 'valid inference when data are missing-at-random.' If the data augmentation does not converge to the observed-data likelihood, the method is a heuristic imputation scheme, not a valid-inference procedure. The reader's weakest assumption pointed at Section 2.3, and I agree that the augmentation is fragile; I would push one step further. Even if the algorithm is accepted as intended, the paper provides no evidence about inferential validity: no standard errors, no coverage, no confidence intervals, no convergence diagnostics. The simulations show large variance and under/over-fitting, which the authors themselves acknowledge. I also note the typographical error in Eq. (2), a concrete flaw in the presentation of the key imputation identity. I nevertheless do not recommend rejection: the architecture is clearly described, the discussion is candid about instability, and the problems are in principle addressable with a convergence proof or with a small simulation study evaluating coverage. The conditional-accept verdict is therefore unchanged. I would make the condition explicit: the 'valid inference' claim must be either removed from the abstract or backed by such a study.","tokens_in":14173,"tokens_out":7008,"duration_ms":74417,"concrete_test":"Run the following check on a simulated LMM dataset with n=600, 20 time points, and cut-off MAR drop-out (the paper's Simulation Study II setting): fit the DGMM with 10 independent random seeds for a fixed large number of epochs and, at the final epoch, record the imputed values for the missing outcomes in a held-out validation set. Compare the pooled empirical distribution of these imputed values to the true posterior predictive p(Y^m | Y^o) under the known data-generating model, checking 95% interval coverage. If coverage deviates materially from nominal (e.g. <90% or >98%), or if between-seed variation in the imputation intervals is large, then Algorithm 1 is not sampling from the target posterior predictive and the 'valid inference under MAR' claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is in the Abstract: the DGMM 'provide[s] valid inference when data are missing-at-random.' Everything novel beyond Mandel et al. rests on Algorithm 1 in Section 2.3. For that claim to hold, (i) the stochastic data augmentation must converge to the maximizer of the observed-data ELBO/marginal likelihood, and (ii) the resulting estimator must have valid frequentist properties. Neither is shown. Eq. (2) as printed contains an internal inconsistency: it states p(Y^m_i | Y^o_i; theta^(v)) = integral p(Y^m_i | Y^o_i; theta^(v)) p(b_i | Y^o_i; theta^(v)) db_i, i.e. the integrand repeats the left-hand side; the intended integrand is p(Y^m_i | b_i; theta^(v)) under conditional independence. More substantively, Step 1 maximizes the VAE ELBO for the complete data using one imputed draw per epoch, while the encoder q(b_i | Y^o_i, Y^m_i; phi) treats the imputed values as observed. Standard stochastic-EM convergence requires exact complete-data M-steps or sufficient statistics; here the M-step is itself a variational approximation, so the fixed point need not be the observed-data MLE, and the normal variational family for p(b_i | Y_i; theta) is only exact for Gaussian linear models. No theorem is supplied. Empirically, the paper reports large run-to-run variance (Figure 4) and iterations with under/overfitting, and the simulations evaluate RMSPE and bias of point predictions, not parameter bias, interval coverage, or hypothesis-test behavior. Hence the abstract's strongest sentence is neither proved nor tested as stated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the Deep Generalised Mixed Model (DGMM), a variational-autoencoder-based neural network that generalizes mixed effects models by representing both fixed and random effects with neural networks. For missing longitudinal outcomes, the authors introduce a stochastic data-augmentation algorithm (Algorithm 1) that imputes missing values from the model's posterior predictive distribution, motivated by missing-at-random (MAR) drop-out in the GrowIt! experience sampling study. The method is applied to the GrowIt! data and evaluated in simulations against an LSTM RNN and the true linear mixed model, using RMSPE and dynamic bias. The paper reports that the DGMM shows potential but suffers from model instability and suboptimal performance in the motivating application.","tokens_in":14664,"tokens_out":6293,"duration_ms":65618,"significance":"If the advertised property of valid inference under MAR were rigorously established, the DGMM would be a meaningful contribution at the intersection of mixed models and deep learning, extending Mandel et al. (2021) by allowing both fixed and random effect parts to be nonlinear and by providing a principled missing-data mechanism. The paper is commendably honest about its limitations, explicitly reporting instability, hyperparameter sensitivity, and underperformance in the GrowIt! analysis. However, the significance is currently limited by (i) the absence of theoretical support for the central MAR inference claim, (ii) an evaluation that measures only predictive accuracy rather than inferential properties such as coverage, and (iii) the lack of uncertainty quantification in the reported comparisons. The core ideas are promising, but the evidence presented does not yet justify the abstract's strongest claims.","major_comments":[{"comment":"Equation (2) is internally inconsistent as printed: it states p(Y^m,_i^{(v)} | Y^o_i; θ^{(v)}) = ∫ p(Y^m,_i^{(v)} | Y^o_i; θ^{(v)}) p(b_i | Y^o_i; θ^{(v)}) db_i, meaning the left-hand side appears as a factor inside the integrand. The intended expression is presumably p(Y^m_i | Y^o_i; θ) = ∫ p(Y^m_i | b_i; θ) p(b_i | Y^o_i; θ) db_i, using the conditional-independence assumption of Eq. (1). Since Eq. (2) defines the imputation distribution used in Step 2 of Algorithm 1—the core of the MAR claim—this must be corrected and reconciled with the sampling algorithm in Web Appendix C.","section":"Section 2.3, Eq. (2)"},{"comment":"The Abstract's claim that the DGMM 'provides valid inference when data are missing-at-random' is not established by the paper. Algorithm 1 is a stochastic EM-type loop, but its Step 1 performs a variational (approximate) M-step, and no convergence result is supplied to show that the fixed point of the loop maximizes the observed-data ELBO or marginal likelihood under MAR. In addition, the normal variational family for p(b_i | Y_i; θ) is only exact for Gaussian linear models; for generic outcome distributions F_k in Eq. (1) the approximation error is not quantified. The paper should either provide a theorem or a precise convergence argument, or the claim should be softened to predictive performance rather than valid frequentist inference.","section":"Abstract and Section 5"},{"comment":"The simulation studies evaluate RMSPE and dynamic bias for point predictions (Eq. (4)) but never assess inferential properties such as parameter bias, interval coverage, or confidence-interval width for the fixed effects or random-effect covariance. Thus the headline claim of valid inference under MAR is never empirically tested. Moreover, Figure 4 shows large run-to-run variance, and the line graphs in Figure 5 are presented without confidence bands or standard errors, making it difficult to judge whether DGMM–RNN differences are meaningful. I recommend adding uncertainty measures for all reported metrics and, if the 'valid inference' claim is retained, including inferential evaluation such as coverage of the true trajectory or of model parameters.","section":"Section 4"}],"minor_comments":[{"comment":"The closed-form KL divergence sums over k with terms μ_il and σ^2_il, but the latent dimension is u; please use a consistent index (e.g., l = 1, ..., u) and avoid reusing k, which already denotes the outcome index.","section":"Section 2.2"},{"comment":"In the dynamic prediction formula, the inner integral is over p(b_j | Y_j(t); θ) but written with db_i, and the outer integral over p(θ | D_n) uses dθ; please make the integration variables consistent (db_j and dθ) for readability.","section":"Section 2.4"},{"comment":"The application uses a simplified architecture that omits the fixed-effects network h and adds baseline covariates to the decoder, which differs from the general model in Section 2 and Figure 1. Please explain this simplification and its implications for identifiability of fixed versus random effects in the reported results.","section":"Section 3"},{"comment":"Several figure labels and captions contain garbled text (e.g., 'Sim)la(on S()dy I Res)l(s', 'T me ( u )'), which appears to be a rendering artifact; these need to be repaired before publication.","section":"Figures 2, 3, and 5"},{"comment":"Key implementation details, such as the inner steps of the Random Walk Metropolis–Hastings algorithm and the Robbins–Monro process in Step 2 of Algorithm 1, are relegated to Web Appendix C, which is not included in the manuscript version under review; the main text should summarize these steps or the appendices should be available for verification.","section":"Web Appendices"},{"comment":"Reference 34 is cited as 'Adam, K. D. B. J. et al.'; the correct citation is Kingma, D. P. and Ba, J., 'Adam: A method for stochastic optimization,' which should be corrected.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is an honest report of a promising but under-validated method, and I appreciate the authors' transparent discussion of instability and subpar performance. However, the gap between the abstract's 'valid inference under MAR' claim and the actual theoretical and empirical evidence is substantial: there is no convergence analysis of the stochastic EM loop with variational M-step, and the simulations do not evaluate inferential properties. A major revision that either supplies rigorous support for the MAR claim or scales the claim back to prediction, and that adds inferential simulation results, would be needed. I also note that no code is mentioned in the manuscript; for a deep-learning method, code availability would materially aid reproducibility and could be considered during revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, it does something genuinely new: it extends Mandel et al.'s neural-network fixed effects to random effects via a VAE, and adds a data-augmentation step aimed at MAR missingness. Second, the central claim in the abstract—\"provide valid inference when data are missing-at-random\"—is not supported by the theory or the simulations. The authors are honest about instability, but the claim overreaches.\n\nWhat the paper does well: the model formulation is clear and the ELBO derivation is standard and correct. The simulation setup is sensible, and the application to the GrowIt! ESM study is real, with a thoughtful discussion of why ESM data may not suit prediction. The authors openly list limitations, including posterior collapse and hyperparameter sensitivity. That transparency is a credit.\n\nThe soft spots are real but fixable. First, Algorithm 1 is a stochastic EM where the M-step is itself a variational approximation; there is no convergence theorem, so the fixed point need not be the observed-data MLE. Second, the normal approximation to p(b_i | Y_i) is only exact for linear Gaussian models, yet it is used generally. Third, the simulations evaluate RMSPE and point-prediction bias, not parameter bias, interval coverage, or hypothesis-test behavior. So the phrase \"valid inference\" is never actually tested. There is also a typo in Eq. (2): the integrand repeats the left-hand side; it should be p(Y^m_i | b_i; theta). That is minor, but it makes you wonder if the missing-data derivation was checked carefully. Finally, no code is released; the data are restricted, but code could be shared. The run-to-run variance in Figure 4 is worrying; the authors concede it, but it means the method is not yet reliable.\n\nWho this is for: people working on ESM/mHealth missing data and on neural extensions of mixed models. I would send it to a serious referee, because the idea is promising and the problems are addressable. My recommendation is to accept it for peer review with an expectation of major revision: either supply a convergence guarantee or sharply weaken the claim, and add simulations that report parameter coverage and interval properties.","headline":"A useful neural extension of mixed models with a data-augmentation step for MAR dropout, but the headline claim of valid inference under MAR is asserted rather than demonstrated.","tokens_in":15062,"tokens_out":2208,"would_cite":false,"duration_ms":22927,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The Deep Generalised Mixed Model estimates both fixed and random effects with neural networks and imputes missing responses from the model's posterior predictive distribution, keeping inference valid under missing-at-random dropout.","keywords":["mixed effects models","neural networks","longitudinal data","missing-at-random","variational auto-encoders","experience sampling method","data augmentation","dynamic predictions"],"falsifier":"Simulate many datasets from a known linear mixed model with a documented MAR dropout rule, fit the DGMM with several random seeds and large sample sizes, and check whether the estimated fixed-effect curves and random-effect posteriors concentrate around the true values as subjects and within-subject measurements grow; if repeated runs remain scattered or imputation error does not approach the true model's imputation error, the valid-inference-under-MAR claim is refuted.","tokens_in":14015,"feed_emoji":"🧠","tokens_out":10170,"duration_ms":94450,"temperature":0.7,"pith_summary":"The paper proposes the Deep Generalised Mixed Model (DGMM), a neural-network architecture that extends mixed effects models to deep learning by replacing both the fixed-effects and random-effects parts with feed-forward neural networks. The central claim is that this architecture can analyse intensive longitudinal data from generic outcome distributions, scale to high-dimensional settings, and give valid inference and dynamic predictions when dropout is missing at random (MAR). Estimation uses a variational auto-encoder: an encoder maps each subject's observed data to a variational posterior over random effects, and a decoder turns draws from that posterior, together with time, into subject-specific mean shifts. Missing responses are imputed during training by sampling from the model's posterior predictive distribution, following a stochastic EM-style data augmentation algorithm. On simulations under MAR the method can recover true trajectories with less dynamic bias than a recurrent neural network when it converges, but the paper reports substantial run-to-run instability, posterior collapse, and a failure to forecast individual states well on the motivating GrowIt! data.","feed_headline":"Neural mixed model claims valid inference under MAR dropout","feed_subtitle":"The DGMM learns fixed and random effects with neural nets and imputes dropped-out responses, though results vary by run.","key_machinery":"The central object is the three-network DGMM: a fixed-effects network $h$, an encoder network $e$, and a decoder network $d$. The encoder outputs the means and variances of a normal variational posterior for the random effects, the decoder maps a draw $z$ (held constant for a subject) together with measurement time into the subject-specific part $\\mu_2$, and the fixed plus random parts are summed and passed through link functions to form predictions for each outcome. Estimation maximizes the ELBO, with the KL term available in closed form because both the variational posterior and the prior are normal. For missing data, the load-bearing mechanism is the data augmentation algorithm in Section 2.3: each epoch imputes the missing responses by sampling from $p(Y_i^m \\mid Y_i^o; \\theta)$, implemented with an adaptive Metropolis-Hastings step, and then retrains on the completed input.","core_discovery":"The paper's claim is that the DGMM—a network in which $\\mu_1(X_i, T_i)$ is a neural-net fixed-effects function and $\\mu_2(b_i, T_i)$ is a neural-net random-effects function, with random effects $b_i \\sim N(0,I)$—can serve as a generalised mixed model for longitudinal outcomes with arbitrary response distributions. Because the marginal likelihood is intractable, the model maximizes an evidence lower bound: Monte-Carlo approximation of the expected log-likelihood minus the closed-form KL divergence between the variational posterior $N(\\mu_b(H_i;\\phi), \\Sigma_b(H_i;\\phi))$ and the standard normal prior. Missing-at-random dropout is handled by a data augmentation loop that alternates between optimizing this bound on the currently imputed data and drawing new imputations of the missing responses from the posterior predictive distribution using a random-walk Metropolis-Hastings sampler. The paper asserts that this gives semi-parametric modelling of the mean and correlation structure, scalability to high-dimensional ESM data, and valid inference under MAR missingness.","pith_inferences":["The normal variational posterior for the random effects is probably the weakest statistical link; replacing it with a heavier-tailed or more flexible variational family (for example a Student-$t$ or a normalizing-flow posterior) could reduce bias for the strongly skewed ESM outcomes.","The large run-to-run variance suggests the stochastic EM-style update lacks a convergence diagnostic; pooling imputations across multiple seeds or adding a proper convergence check on the imputed-data bound could make the method usable before further architectural changes.","The encoder's latent means can be read as low-dimensional embeddings of individual trajectories, so the same architecture could be extended to joint modelling of longitudinal and time-to-event data or to clustering patients by trajectory shape.","A natural testable extension is to benchmark the DGMM against standard missing-data tools such as multiple imputation by chained equations or joint mixed models on the same MAR data, since those are established comparators the paper does not include."],"forward_implications":["A single DGMM architecture can model longitudinal outcomes that follow different distributions (Gaussian, Student-$t$, gamma, beta, binomial, Poisson, negative binomial), removing the need to hand-specify a separate model for each outcome.","The encoder pools all available measurements per subject, so the model can generate subject-specific dynamic predictions past the last observation while the data augmentation is designed to keep those predictions unbiased under MAR dropout.","When training converges, the simulations indicate the DGMM recovers true trajectories with lower dynamic bias than an RNN under MAR mechanisms and can approach the predictive accuracy of the true data-generating mixed model.","The architecture is not yet stable enough for routine use: simulation runs show large variance with occasional under- or overfitting, posterior collapse is a known failure mode, and on the GrowIt! data neither the DGMM nor the RNN forecasted individual negative states well.","Because the decoder is a generic network, future users can replace its feed-forward layers with recurrent or convolutional layers, extending the same formulation to other forms of nested or hierarchical data."],"supporting_citations":[{"why":"Supplies the starting point: mixed effects with a neural-network fixed-effects part, which the DGMM extends by making random effects neural too.","marker":"[26]"},{"why":"Provides the variational auto-encoder formalism that underlies the encoder–decoder structure.","marker":"[27]"},{"why":"Introduces the evidence lower bound / auto-encoding variational Bayes objective the DGMM maximizes.","marker":"[29]"},{"why":"Gives the Bayesian central limit theorem used to justify the normal approximation to the random-effects posterior.","marker":"[31]"},{"why":"Borrowed as the stochastic expectation-maximisation framework for the missing-data imputation loop.","marker":"[32]"},{"why":"Supplies the Robbins–Monro process used to adapt the Metropolis-Hastings sampler inside the imputation step.","marker":"[33]"},{"why":"The closest prior RNN treatment of missing longitudinal values, which the DGMM aims to surpass by avoiding its restrictive decay assumption.","marker":"[25]"},{"why":"Documents that GrowIt! participants drop out following strong negative emotions, establishing the MAR missingness that motivates the method.","marker":"[24]"},{"why":"Cyclic KL-annealing is used to counter posterior collapse in the VAE, a stability technique the DGMM relies on.","marker":"[35]"},{"why":"Provides the beta-VAE reweighted KL objective also used to stabilize training against posterior collapse.","marker":"[36]"}],"fun_headline_variants":["Neural mixed model tackles MAR dropout but runs unstable","Could a neural net replace mixed models? DGMM says maybe, but unstable","Neural mixed model imputes MAR dropouts, but runs vary","Deep mixed model: handles MAR data, but unstable inference"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that imputing missing responses from the model's own posterior predictive distribution, together with a normal approximation to the random-effects posterior, forms a valid EM-type estimation procedure under missing-at-random data, and the paper provides no convergence proof for this loop.","fun_headline_variants_meta":{"raw":{"variants":["Neural mixed model tackles MAR dropout but runs unstable","Could a neural net replace mixed models? DGMM says maybe, but unstable","Neural mixed model imputes MAR dropouts, but runs vary","Deep mixed model: handles MAR data, but unstable inference"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001048,"raw_usage":{"total_tokens":4439,"prompt_tokens":1013,"completion_tokens":3426,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":629,"completion_tokens_details":{"reasoning_tokens":3353}},"tokens_in":629,"tokens_out":3426,"duration_ms":24957,"temperature":1.0,"reasoning_tokens":3353,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T04:24:49.534759+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate many datasets from a known linear mixed model with a documented MAR dropout rule, fit the DGMM with several random seeds and large sample sizes, and check whether the estimated fixed-effect curves and random-effect posteriors concentrate around the true values as subjects and within-subject measurements grow; if repeated runs remain scattered or imputation error does not approach the true model's imputation error, the valid-inference-under-MAR claim is refuted.","supporting_citations":[],"review_version":2}