{"id":"b0f7992c-abc8-461d-aa0b-3614782615f5","arxiv_id":"2607.14122","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"GNDR treats a trained network's last layer as a classical generalized linear model on learned features, yielding maximum-likelihood confidence intervals for distributional parameters.","lead":"GNDR is a two-step recipe that fits a neural network as part of a classical probability model, then freezes all but the final layer and uses maximum-likelihood theory to produce per-observation confidence bands. It is demonstrated on hospital-stay counts, breast-cancer survival from gene expression, and age estimation from facial images.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1 conditions on the learned feature extractor, so the reported covariance omits first-step estimation error; the resulting intervals are conditional, not unconditional, and can undercover.","rationale":"The reader's weakest assumption correctly identifies that the frozen deep feature extractor is treated as a fixed, known basis and its estimation error is not propagated. My analysis sharpens this: it is not merely a nuisance-parameter omission, but a structural break in the likelihood argument. Because v(x) is trained on the same responses y, conditioning on v(x) does not yield a standard fixed-design VGLM; the second-step 'MLE' is an M-estimator with an endogenous, data-dependent design, and the regularity conditions in Appendix A are not satisfied for the unconditional procedure. Equation (8) is a conditional observed information, not the full covariance of the two-step estimator, so the Delta-method intervals in Eqs. (13)–(16) are not valid unconditional prediction intervals. The observed 89.35% out-of-sample coverage is consistent with this. This does not invalidate the conditional theorem as a mathematical statement, but it undermines the paper's advertised 'rigorous uncertainty quantification' for new observations. The reader's CONDITIONAL verdict already captures the need to make this caveat explicit and to validate the UQ claims; I am not proposing a different verdict. Hence UNCHANGED.","tokens_in":22107,"tokens_out":7259,"duration_ms":80838,"concrete_test":"Simulate a neural-network DGP with known fixed weights: x ~ Uniform(-2,2), y|x ~ Normal(F(x;ω*), exp(G(x;ω*))^2), where F and G are two-hidden-layer networks with known weights. For R=500 replicated training sets of sizes n=2000 and n=5000, fit the paper's two-step GNDR with the same architecture (e.g., last hidden layer of 2 units), compute 95% predictive intervals via Eqs. (8)–(16), and evaluate coverage on a fresh test set of 1000 points. Also fit an oracle version that fixes ω̂_deep to the true ω*. If the oracle achieves ~95% coverage while the estimated-feature procedure yields average coverage significantly below 95% (e.g., <93%), the missing first-step correction is confirmed. Additionally, compute the ratio of the mean Hessian-based standard error to the empirical standard deviation of θ̂_last across replications; a ratio substantially below 1 (e.g., <0.8) would demonstrate that","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central UQ claim rests on Theorem 1 (§3.2), which states √n(θ̂_last − θ_last^(0)) → N_q(0, I(θ_last^(0))^{-1}) 'conditional on the fixed latent feature column-vector v(x)'. But v(x_i) is defined (§3.1) as F_deep(x_i; ω̂_deep), where ω̂_deep is obtained in the first step by stochastic optimization with early stopping on the same training responses. Conditioning on v(x) does not remove the fact that the feature map is endogenous: it is a function of the training y's. The second-step log-likelihood is therefore not the true conditional likelihood of y given v(x) for a known feature map; the design matrix is stochastic and correlated with the response. The Appendix's proof verifies regularity conditions for a fixed-design VGLM, but those conditions are not met by the actual two-step procedure, because the design is learned from the same data. Eq. (8) estimates the conditional Fisher information given the realized features, not the total estimator variance that accounts for feature-learning randomness. The paper then uses this covariance for observation-specific intervals (Eqs. 13–16) without any first-step correction (e.g., a semiparametric two-step variance adjustment). The empirical out-of-sample coverage of 89.35% in §4.4 is exactly the direction expected from this omission: the intervals are too narrow because they ignore uncertainty in the feature extractor.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Generalized Neural Distributional Regression (GNDR), a two-step estimation framework for parametric distributional regression with deep neural networks. In the first step, the full network (including the terminal layer) is trained by stochastic optimization with early stopping. In the second step, all internal layers are frozen and only the terminal dense layer and classical structured effects are re-estimated by full-data maximum likelihood in a reduced Vector Generalized Linear Model (VGLM). Theorem 1 claims conditional asymptotic normality of this terminal-layer estimator, with covariance given by the inverse Fisher information; the Delta method is then used to build observation-specific confidence bands for distributional parameters, quantiles, and survival curves. The framework is demonstrated on four applications: Dutch boys' BMI with a Box-Cox t distribution, hospital stay counts with a Beta-binomial distribution, TCGA-BRCA survival with various parametric and mixture-cure models, and age prediction from UTKFace images with a zero-truncated normal distribution. The paper also presents the open-source package 'thetaflow'.","tokens_in":22505,"tokens_out":4874,"duration_ms":50348,"significance":"If the conditional-normality claim could be upgraded to a valid unconditional two-step inference statement, GNDR would be a practically valuable bridge between deep representation learning and classical parametric inference, enabling uncertainty quantification on unstructured/high-dimensional data. The manuscript has several strengths: the identifiability discussion is explicit, the reduction to a VGLM is transparent, the Hessian is computed analytically via automatic differentiation, and the authors provide reproducible code and a package. The empirical scope is broad, covering tabular, genomic, and image data. However, the central UQ promise is not yet justified: the asymptotic covariance in Eq. (8) ignores the randomness of the first-step feature extractor, and the model-selection protocol in Section 3.3 makes the reported test-set calibration partly self-referential. These issues are load-bearing for the paper's main claims.","major_comments":[{"comment":"Theorem 1 conditions on v(x)=F_deep(x; \\hat\\omega_deep), but \\hat\\omega_deep is estimated from the same training responses and is correlated with the second-step likelihood. The covariance in Eq. (8) is the conditional Fisher information given the realized features, not the total variance of \\hat\\theta_last or of the Delta-method bands in Eqs. (11)–(16). Standard two-step inference (e.g., Murphy–Topel or Newey–McFadden corrections) is not applied. The empirical test-set coverage of 89.35% in §4.4 is exactly the direction expected from this omission. The manuscript should either provide a corrected covariance that propagates first-step uncertainty, or explicitly reframe the UQ as conditional on the estimated features and temper the coverage claims.","section":"§3.1–3.2, Theorem 1, Eq. (8)"},{"comment":"The model-selection rule in §3.3 selects the model minimizing the Anderson–Darling statistic on the test set. The applications then report the test-set AD/calibration of the selected model as evidence of 'superior distributional calibration' (e.g., Table 2 and the claim that model (d) 'emerges as the superior architecture'). This is self-referential: the test set is used for selection, so the reported test performance is optimistically biased. A separate validation set for selection, or nested cross-validation, is needed for honest out-of-sample evaluation.","section":"§3.3, §4.2, §4.3"},{"comment":"The proof of Theorem 1 verifies the VGLM regularity conditions by assertion rather than by argument. In particular, Regularity Condition II requires the concatenated design matrix [z, v(x)] to have full column rank, but v(x)=F_deep(x; \\hat\\omega_deep) is itself random and estimated, so the fixed-design VGLM conditions of Yee (2015) are not automatically satisfied. The proof also does not clarify the definition of the 'true' θ_last^0 when the feature map is estimated. A rigorous proof requires either an explicit asymptotic framework for the two-step estimator (treating \\hat\\omega_deep as a nuisance function) or a sample-splitting scheme that makes the feature extractor independent of the second-step likelihood.","section":"Appendix A"}],"minor_comments":[{"comment":"Typo: 'packagethetaflow' should read 'package thetaflow'.","section":"Abstract"},{"comment":"Heading typo: 'identifiabiliy' should be 'identifiability'.","section":"§2.2"},{"comment":"The notation in (16) writes the limiting distribution as N_p, but f maps R^p to R^q; the dimension should be q or the notation should be clarified.","section":"Eq. (16)"},{"comment":"The Box-Cox t log-likelihood uses f_T and F_T without explicitly stating that T denotes a Student-t distribution with τ_i degrees of freedom; please define these symbols.","section":"§4.1"},{"comment":"The 'logit(λ/40)' link for the Cure Mixture Weibull scale imposes an identifiability bound; it would help to state explicitly that this is a modeling assumption and how sensitive results are to the choice of 40.","section":"Table 3"},{"comment":"The sentence about AIC/BIC being invalid 'except they are comparing the exact same distribution, as it happens in cross validation' is confusing; please clarify the intended scope of the advice.","section":"§3.3"}],"recommendation":"major_revision","confidential_remarks":"The conditional/unconditional gap in the UQ pipeline is the central issue; it is fixable in principle but requires substantive additional theory or a clear reframing. The test-set model selection also undermines the empirical calibration claims as currently reported. I would encourage the authors to address these two points before the paper is suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Simone, this one is worth a careful read, but the headline claim needs a qualifier.\n\nThe genuinely new piece is the two-step estimator: train the full network, then freeze all internal layers and re-estimate the terminal head by full-batch MLE, with an analytical observed Fisher information from the reduced VGLM. The Delta-method machinery for observation-specific intervals is clean, and the paper actually ships code (thetaflow) and tests the approach on three quite different data modalities, including images. That breadth is not common in this literature.\n\nTheorem 1 is a standard conditional MLE result, and the authors are unusually transparent about the conditioning — the statement says “conditional on the fixed latent feature column-vector v(x)”. The problem is that v(x) is itself estimated from the same training responses, and the Appendix verifies regularity conditions for a fixed-design VGLM, not for the two-step procedure. So the covariance in Eq. (8) is conditional on the realized features, and the intervals built from it are conditional intervals. The paper then presents them as confidence bands and tolerance intervals without any adjustment for first-step uncertainty. The reported out-of-sample coverage of 89.35% in §4.4 is exactly the direction you’d expect from that omission. This is the central soft spot, and it is real.\n\nThere are two secondary weaknesses. First, model selection is done by minimizing the Anderson-Darling statistic on the test set (Section 3.3 and Table 2). That makes the reported test-set calibration self-referential; the selected model is the one that looked best on that same test set. Second, deepregression (Rügamer et al., 2020) is cited but never used as a baseline, so the “superior” calibration claim is not supported by a head-to-head comparison.\n\nNone of this makes the paper unserious. The conditional theorem is sound as a conditional statement, the software is a real contribution, and the survival and image applications are well executed. But the UQ promise overreaches. The fix is a two-step variance correction (or a bootstrap over the first step) plus a genuinely independent holdout for model selection. As it stands, the paper deserves a serious referee — I would send it out — but with the expectation of major revisions.","headline":"A genuinely useful two-step MLE framework with a real conditional-inference caveat: the reported intervals ignore first-step feature-estimator uncertainty, and the test-set model selection weakens the empirical claims.","tokens_in":22975,"tokens_out":2593,"would_cite":false,"duration_ms":25888,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F12","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"By freezing a deep network's hidden layers as a fixed basis expansion, the terminal layer becomes a classical MLE with recoverable Fisher information — observation-specific confidence bands follow for any distributional parameter.","keywords":["neural distributional regression","uncertainty quantification","Fisher information matrix","two-step estimation","Delta method","asymptotic normality","survival analysis","deep learning"],"falsifier":"Simulate data from a GNDR model whose true parameters are known, with a wide terminal bottleneck so the learned features carry nontrivial estimation error. Fit the two-step procedure across many Monte Carlo replicates and compare the empirical standard deviation of θ̂_last with the mean inverse observed Hessian: if the ratio deviates from one, or if Delta-method interval coverage falls below nominal while a full two-step bootstrap holds nominal coverage, then Theorem 1's conditional characterization is incomplete.","tokens_in":22001,"feed_emoji":"📊","tokens_out":12281,"duration_ms":117202,"temperature":0.7,"pith_summary":"The paper proposes Generalized Neural Distributional Regression (GNDR), a way to embed deep neural networks inside classical parametric distributions, modeling each distributional parameter as a link-transformed network output. Its central claim is that the non-identifiability that normally blocks statistical inference in deep networks can be sidestepped by a two-step scheme: first train the whole network, then freeze all hidden layers and re-fit only the final output layer with full-batch maximum likelihood. In the second step the network is mathematically a vector generalized linear model over a learned feature map, so the estimator is asymptotically normal and the observed Hessian is a valid covariance matrix. This makes it possible to attach valid, per-observation confidence intervals to parameters, quantiles, and survival curves even when the input is an image or a high-dimensional gene-expression vector. Empirical results show the framework matching or improving on classical additive distributional-regression baselines in calibration while processing unstructured data.","feed_headline":"Freeze the deep net, and its last layer becomes a proper MLE","feed_subtitle":"Analytical Fisher information from the terminal layer gives every deep prediction a confidence band","key_machinery":"The load-bearing device is the two-step estimation split ω = {ω_deep, ω_last}. Step 1 optimizes the full network by stochastic gradient methods; step 2 permanently freezes ω̂_deep, so the final hidden layer becomes a deterministic learned basis expansion v(x) = F_deep(x; ω̂_deep), and the network output collapses to the linear form F(x; ω) = W_lastᵀ v(x) + b_last. With the structured effects ξ, the model is then exactly a vector generalized linear model — a linear model for several distribution parameters at once — over the identifiable parameter vector θ_last = (ξ, vec(W_last), b_last), whose design matrix [z, v(x)] is full rank thanks to the disjoint-covariate condition. The observed Fishe","core_discovery":"Conditional on the frozen latent feature vectors v(x_i) produced by the upstream network, the GNDR estimator θ̂_last is asymptotically normal: √n(θ̂_last − θ_last⁰) → N_q(0, Σ_last) with Σ_last = I⁻¹(θ_last⁰) (Theorem 1). Because the second training step drives the restricted log-likelihood to a strict root of the score equation over a fixed, finite-dimensional parameter space, standard regularity conditions for vector generalized linear models apply, and the inverse observed Hessian (Eq. 8) is a machine-precision-accurate covariance estimate. From this anchor, the paper chains three Delta-method applications — through the data-dependent projection P(x), the predictor mapping h, and the inve","pith_inferences":["The intervals in the paper are conditional intervals: they treat the learned features from the first step as fixed, so the honest test of the method is to compare their frequentist coverage with a full two-step bootstrap; the paper's own image experiment (89.35% out-of-sample coverage for nominal 95%) suggests the conditioning gap is material at that scale.","The width of the terminal bottleneck plays a dual role — representation capacity in step one, statistical degrees of freedom in step two — which points toward a sieve-style asymptotic theory in which M grows slowly with n; that formalization is implicit in the paper but not developed.","The freeze-and-refit device should transfer to other inference paradigms: Bayesian terminal-layer inference (where the observed Hessian doubles as a Laplace posterior covariance), neural random effects for clustered data, and conformal wraps around the Delta-method intervals as a finite-sample correction.","A direct calibration experiment is easy to run: simulate from a known GNDR model, refit the full two-step procedure many times, and check whether the inverse observed Hessian tracks the Monte Carlo variance of the terminal-layer estimator — if it does not, the gap identifies exactly where the conditional-normality premise leaks."],"forward_implications":["For any base distribution with a smooth, twice-differentiable log-likelihood, the framework yields observation-specific confidence bands for parameters, quantiles, survival curves, and tolerance intervals, all computed from one analytical covariance matrix (Eqs. 8, 13–16).","GNDR reduces to classical additive distributional regression when the neural component is zeroed out, while the neural version can capture interaction surfaces that independent penalized splines miss; the paper's hospital-stay application shows the neural interaction model giving the best out-of-sample calibration.","The theory changes model-selection practice: in-sample AIC/BIC and global deviance are declared invalid for comparing deep distributional architectures, replaced by out-of-sample Anderson-Darling statistics on normalized quantile residuals — with the caveat, in censored survival data, that residual diagnostics be paired with visual checks of predicted survival curves.","Architecture design follows from the theory: the terminal hidden layer should be an aggressive bottleneck (width M ≤ 4 in the applications), because the second-step parameter count — b + M p_F + p_F — is what determines statistical degrees of freedom and variance inflation."],"fun_headline_variants":["Freeze deep features, get exact confidence bands","Two-step fit gives deep nets a statistical backbone","Analytical Fisher info from a frozen neural net","Per-prediction confidence bands from deep features","GNDR: uncertainty quantification for deep regression"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The entire inferential output treats the deep features from the first training step as fixed and known, so all confidence intervals are conditional on them — the paper's own facial-image experiment finds 89.35% out-of-sample coverage for nominal 95%, which is the size of the leak when that premise is strained.","fun_headline_variants_meta":{"raw":{"variants":["Freeze deep features, get exact confidence bands","Two-step fit gives deep nets a statistical backbone","Analytical Fisher info from a frozen neural net","Per-prediction confidence bands from deep features","GNDR: uncertainty quantification for deep regression"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000175,"raw_usage":{"total_tokens":1095,"prompt_tokens":689,"completion_tokens":406,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":433,"completion_tokens_details":{"reasoning_tokens":335}},"tokens_in":433,"tokens_out":406,"duration_ms":4638,"temperature":1.0,"reasoning_tokens":335,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T10:59:44.695933+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate data from a GNDR model whose true parameters are known, with a wide terminal bottleneck so the learned features carry nontrivial estimation error. Fit the two-step procedure across many Monte Carlo replicates and compare the empirical standard deviation of θ̂_last with the mean inverse observed Hessian: if the ratio deviates from one, or if Delta-method interval coverage falls below nominal while a full two-step bootstrap holds nominal coverage, then Theorem 1's conditional characterization is incomplete.","supporting_citations":[],"review_version":1}