{"id":"5025d338-d1e4-459a-9f28-fd93a10572d8","arxiv_id":"2607.04647","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Two-stage neural-encoder GLMMs with variance-corrected SGLD recover full-data MCMC posteriors and quantify modality importance at population and subject levels without losing predictive accuracy.","lead":"The paper shows how to attach neural encoders for images and other high-dimensional inputs to Bayesian generalized linear mixed models, then sample the mixed-model parameters with variance-corrected stochastic-gradient MCMC after freezing the encoders. The design keeps interpretable fixed and random effects while scaling to large longitudinal multimodal clinical datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Conditional-Bayes UQ is only validated under oracle-aligned features; residual encoder error is not stress-tested for interval calibration.","rationale":"I agree with the reader’s weakest assumption and verdict. The paper’s strongest evidence is recovery of means/variances after correction when the feature map is effectively oracle-aligned; that is necessary but not sufficient for the scientific claim that modality-level posterior intervals are trustworthy once the encoder is frozen. App. B is a local stability argument, not a coverage guarantee under realistic Phase-1 error, and the multi-encoder ABCD setting amplifies the issue. The proposed coverage experiment under imperfect frozen features is the single check that would settle whether this soft spot actually undermines the claim. No stronger internal inconsistency is apparent; the method is coherent as conditional Bayes. Verdict remains CONDITIONAL pending that stress test (and the reader’s already-noted data/code cleanup).","tokens_in":19267,"tokens_out":562,"duration_ms":5621,"concrete_test":"In the high-dim GLMM sim (Eq. 6 / App. D.3), freeze Phase-1 encoders at several early epochs (or inject controlled feature noise of size r_N matching App. B), then run full Phase-2 SGLD + Lyapunov correction and report empirical 95% coverage of true (β, log σ²) over ≥50 seeds. If coverage falls materially below ~95% while oracle-z MCMC remains calibrated, the load-bearing UQ claim fails for realistic residual representation error.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that after Phase-1 freezing and Lyapunov correction, SGLD recovers trustworthy posterior means/variances for fixed effects and variance components (and thus modality-importance summaries). The reader correctly flags the frozen-encoder premise (§3.3, App. B). The softest place is more specific: simulation benchmarks (App. D.1–D.3, Fig. 2 bottom / Fig. 3) compare against Gibbs/MCMC that use the true latent z, not the learned encoder map. App. B only gives a local mode/covariance stability bound under small score/curvature perturbation r_N; it does not establish frequentist coverage of corrected intervals when the encoder is imperfect, when Phase-1 is short, or when multiple modality encoders are standardized jointly (as in ABCD Eq. 7). Real-data UQ is assessed mainly by predictive AUC parity, not by parameter-level calibration under representation error. If residual encoder misspecification is non-negligible, corrected intervals can look well-calibrated against an oracle-feature benchmark while miscovering the scientifically relevant conditional posterior.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes a two-stage Multimodal GLMM that couples modality-specific neural encoders with a classical mixed-effects linear predictor, then performs variance-corrected SGLD on the GLMM parameters after freezing the learned representation. The architecture retains fixed effects for structured covariates and fixed/random effects (including modality-level random slopes) for standardized encoder outputs, with sign-invariant standardized magnitude summaries justified by Proposition 3.1. Simulations report recovery of posterior means and corrected variances relative to full-data MCMC/Gibbs benchmarks, and two longitudinal applications (glaucoma visual fields; ABCD multi-domain mental-health risk) report held-out AUC comparable to deterministic neural baselines while interpreting population-level and subject-specific modality importance.","tokens_in":19664,"tokens_out":1475,"duration_ms":21137,"significance":"If the conditional-Bayes target after encoder freezing is scientifically adequate and the corrected SGLD posteriors are well calibrated, this is a useful and timely contribution: it extends scalable Bayesian GLMM inference (building on prior SGLD-GLMM work) to high-dimensional multimodal longitudinal data without burying subject-level effects inside a black-box network. Strengths include an explicit identifiability analysis with a proof (Prop. 3.1 / Appendix A), an honest division of labor that avoids overclaiming full Bayesian neural-weight posteriors, a Lyapunov post-hoc variance correction with a stated algorithm, and clinically motivated applications that separate average modality effects from between-subject heterogeneity. These elements make the work more than a pure engineering hybrid of CNNs/MLPs and mixed models.","major_comments":[{"comment":"Central UQ claim vs. simulation design (Abstract; §4; Appendix D.1–D.3; Fig. 2 bottom / Fig. 3): the Abstract states that the method recovers posterior means and variances from full-data MCMC benchmarks and evaluates parameter-level interval coverage. The reported benchmarks, however, appear to condition on the true latent feature z (or its known functional form), not on the frozen learned map ĥ_φ used by the proposed sampler. High feature correlations (~0.997–0.999) make this nearly equivalent in the reported LMM cases, but it does not directly validate calibration of the actual target π_φ̂(Θ|D) under residual representation error, short Phase-1 training, or multi-encoder standardization as in ABCD Eq. (7). A load-bearing addition is MCMC (or long-run HMC) conditional on the same frozen learned features, plus explicit frequentist coverage of corrected intervals under imperfect encoders—","section":"§4 / Appendix D / Abstract"},{"comment":"Appendix B (Prop. B.1) only gives a local mode/covariance stability bound under small score and Hessian perturbations r_N. That is useful but insufficient for the paper’s stronger operational claim that Lyapunov-corrected SGLD intervals are trustworthy for scientific estimands (β, Σ, and ω_pop/ω_het). The manuscript should either (i) state more carefully that reported UQ is conditional-Bayes given a point-estimated encoder and is not claimed to have frequentist coverage under encoder misspecification, or (ii) provide simulation evidence that corrected credible intervals achieve near-nominal coverage for fixed effects and variance components when the encoder is learned from finite data (including noise dimensions as in §4.2).","section":"Appendix B / §3.3"},{"comment":"Uncertainty evaluation on real data (§5.1–5.2) is largely predictive parity (AUC ≈ 0.76 glaucoma; AUC/accuracy ≈ 0.78 ABCD) versus deterministic baselines. Predictive AUC does not establish calibration of parameter-level posteriors or of modality-importance rankings (e.g., Sleep vs. NSES in §5.2). Given that the paper’s scientific selling point is nuanced modality importance with uncertainty, the applications need at least one stronger UQ check: predictive calibration (reliability / Brier / PIT-style summaries on held-out subjects), sensitivity of ω_pop/ω_het rankings to Phase-1 freeze epoch and encoder architecture, or leave-one-domain-out stability of the reported domain orderings. Without this, the real-data UQ narrative rests mainly on interval widths around point estimates whose calibration is untested outside oracle-feature simulations.","section":"§5.1–§5.2"}],"minor_comments":[{"comment":"Abstract claims “parameter-level interval coverage in simulations and predictive calibration on held-out data,” but the main text emphasizes mean/variance recovery and AUC parity. Align abstract language with the actual reported metrics, or add the missing coverage/calibration displays.","section":"Abstract"},{"comment":"Appendix F.1 still says “a XX cohort study.” Fill in the cohort name/source and any access constraints.","section":"Appendix F.1"},{"comment":"Figures 4–5 are referenced as full posterior distributions for glaucoma and ABCD, but the manuscript text does not quantify mixing diagnostics (ESS, split-R̂, or MH acceptance) for the SGLD chains after correction. A short diagnostics paragraph would strengthen credibility of the reported CIs.","section":"Appendix H–I / §5"},{"comment":"Notation: z_ij is both the standardized encoder output and, in simulations, the true latent function; a consistent distinction (e.g., z* vs. ẑ) would reduce confusion when discussing non-identifiability and benchmark construction.","section":"§3.1 / §4"},{"comment":"Related work could more clearly position against other hybrid mixed-effects neural models already cited (NME, MC-GMENN, GDMix) on the specific axis of modality-level random slopes plus corrected SG-MCMC, rather than only MAP/VI limitations.","section":"§2"},{"comment":"Typographical: “stochasticgradient” missing space in the Abstract; “Preprint.” / arXiv line formatting is fine for arXiv but should be cleaned for journal submission.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The paper is a natural and competent extension of the authors’ prior scalable SGLD-GLMM line; novelty is real but incremental, and the main risk for the journal is overstated UQ relative to oracle-feature validation. If the authors add learned-feature conditional benchmarks and tone the Abstract to match what is actually shown, this could become a solid methods paper rather than a borderline one. Scope fit for a stat.ML / biostatistics methods venue is good."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The thing worth knowing is that this is a practical assembly job done carefully: modality-specific encoders plus a GLMM layer, trained jointly for a while, then frozen so variance-corrected SGLD can target a fixed conditional posterior. That is not a new theory of Bayesian neural nets, but it is a finished procedure that recovers full-data MCMC means and corrected variances in simulation and keeps held-out AUC on two longitudinal clinical problems.\n\nWhat is actually new is the conditional-Bayes split and the standardized modality summaries (population and heterogeneity magnitudes and shares). Prop. 3.1 is short and correct about sign/scale non-identifiability. The SGLD + Fisher-identity gradient + Lyapunov correction is their prior work; the contribution is making that machinery run on learned multimodal features without turning the whole network into a Bayesian object. Simulations are the strongest part: LMM latent recovery is near-perfect after standardization, and the high-dimensional logistic GLMM with random slopes recovers fixed effects and variance components against a Gibbs/MCMC benchmark after correction. Real data (glaucoma VF CNN + age; ABCD multi-domain MLPs) give sensible domain rankings—sleep dominant on average, neighborhood SES weak on average but high random-slope variance—and predictive parity with deterministic baselines. Citations are honest about Shi, Mandel, Tschalzev, and their own SGLD-GLMM line.\n\nSoft spots, in proportion. The stress-test note is right that the main simulation benchmarks use true latent z, not the learned encoder, so interval calibration under residual representation error is not fully stress-tested; Appendix B only gives a local stability bound. That is a real limitation of the UQ claim, not a collapse of the method: they are explicit that they do not propagate uncertainty over neural weights, and predictive calibration is checked on held-out data. Hyperparameters (step size, freeze epoch, MH budget, κ) are free and tuned; that is normal for SGLD. Cohort construction still has a placeholder (“XX cohort”) and code is not shipped here—those are fixable before acceptance, not conceptual holes. Sign of learned latents remains arbitrary; they correctly push interpretation onto magnitudes and rankings.\n\nThis is for people who need interpretable mixed-effects UQ on images/text longitudinal data and already accept empirical-Bayes-style conditioning on a trained encoder. I would send it to peer review. Engage if you work on clinical multimodal longitudinal models; cite it when you need a scalable neural-GLMM baseline with corrected SGLD.","headline":"Clean, usable two-stage neural-encoder GLMM with real MCMC recovery after correction; the soft spot is that UQ is conditional on a frozen encoder and is mainly stress-tested against oracle features.","tokens_in":20217,"tokens_out":612,"would_cite":true,"duration_ms":6831,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Neural encoders can be fused into Bayesian mixed models so high-dimensional modalities get population and patient-specific importance with calibrated uncertainty.","keywords":["generalized linear mixed models","neural encoders","stochastic gradient Langevin dynamics","multimodal longitudinal data","uncertainty quantification","modality importance","random slopes"],"falsifier":"On a high-dimensional multimodal simulation where full MCMC on the true latent features is feasible, check whether corrected SGLD credible intervals fail to cover the true fixed effects or variance components at their nominal rate, or whether the recovered modality-importance rankings reverse relative to the known data-generating process.","tokens_in":20181,"feed_emoji":"🧠","tokens_out":609,"duration_ms":5944,"temperature":0.7,"pith_summary":"Bayesian mixed models already give uncertainty for clustered longitudinal outcomes, but they usually take only low-dimensional tabular covariates. This paper shows how to attach modality-specific neural encoders for images, text, or other high-dimensional blocks, train those encoders jointly with the mixed-model objective, freeze the learned features, and then sample the mixed-model parameters with variance-corrected stochastic-gradient MCMC. The design keeps ordinary fixed and random effects for structured covariates and for each learned modality, including subject-specific random slopes that measure how much a given modality matters for each person. Simulations recover full-data MCMC posterior means and variances after a covariance correction, and two clinical applications (glaucoma visual fields and multi-domain adolescent mental-health risk) show that modality rankings can be read at both population and individual levels without loss of held-out predictive accuracy.","feed_headline":"Neural mixed models rank modalities for each patient","feed_subtitle":"Encoders plus corrected SGLD recover MCMC posteriors and keep predictive accuracy on clinical data.","key_machinery":"Conditional-Bayes two-stage pipeline: Phase-1 joint supervised learning of standardized neural features inside the GLMM likelihood, then Phase-2 SGLD targeting the fixed conditional posterior of the GLMM parameters, followed by a Lyapunov-equation post-hoc variance correction that undoes inflation from mini-batches and Langevin noise.","core_discovery":"A two-stage conditional-Bayes procedure—joint supervised training of modality-specific neural encoders, followed by variance-corrected SGLD on the GLMM layer with those features frozen—recovers accurate posterior means and variances for fixed effects and variance components while yielding identifiable standardized summaries of each modality’s population-level and subject-level importance, all without sacrificing predictive performance on large longitudinal multimodal data.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Neural encoders plus SGLD rank multimodal effects per patient","Conditional Bayes recovers posteriors for multimodal GLMMs","Joint neural-GLMM training ranks modalities by subject","Variance-corrected SGLD scales Bayesian multimodal mixed models","Neural features preserve GLMM interpretability on longitudinal data"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"Once the neural encoder is frozen after supervised training, the remaining uncertainty over its weights can be ignored and the corrected GLMM posterior is still scientifically trustworthy.","fun_headline_variants_meta":{"raw":{"variants":["Neural encoders plus SGLD rank multimodal effects per patient","Conditional Bayes recovers posteriors for multimodal GLMMs","Joint neural-GLMM training ranks modalities by subject","Variance-corrected SGLD scales Bayesian multimodal mixed models","Neural features preserve GLMM interpretability on longitudinal data"]},"model":"grok-4.5","effort":"low","cost_usd":0.006784,"raw_usage":{"total_tokens":1692,"prompt_tokens":746,"num_sources_used":0,"completion_tokens":81,"cost_in_usd_ticks":67840000,"prompt_tokens_details":{"text_tokens":746,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":865,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":746,"tokens_out":81,"duration_ms":6762,"temperature":1.0,"reasoning_tokens":865,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T15:48:34.427337+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On a high-dimensional multimodal simulation where full MCMC on the true latent features is feasible, check whether corrected SGLD credible intervals fail to cover the true fixed effects or variance components at their nominal rate, or whether the recovered modality-importance rankings reverse relative to the known data-generating process.","supporting_citations":[],"review_version":1}