{"id":"b0a6b3ca-3f50-428d-9918-ead1ee578667","arxiv_id":"2505.01995","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Double-NN combines extended fiducial inference with deep neural networks to produce individual treatment effect prediction intervals, with a consistency proof allowing model size O(n^ζ) for ζ<1.","lead":"The authors propose a method called Double-NN that uses three neural networks to estimate individual treatment effects and their uncertainty through extended fiducial inference. The paper claims the model size can grow almost as fast as the sample size while the inference stays valid, and it reports shorter prediction intervals than conformal methods in simulations.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No theory connects Theorem 3.1's consistency result to the advertised uncertainty-quantification guarantee; coverage of EFI ITE intervals is asserted but unproven.","rationale":"The reader correctly identified Assumption 6 as an unverified, load-bearing premise for Theorem S2 and Theorem 3.1; that remains a legitimate concern. I see an even more direct gap: the paper's advertised uncertainty-quantification guarantee is not a theorem in the manuscript. Section 3.1 proves consistency of an inverse mapping, and the transition from consistency to valid EFD-based intervals is only a heuristic 'Slutsky' propagation. If the paper were revised to verify Assumption 6, the central claim would still lack proof. The simulations in §5 and §4 are informative and the code is available, but they use non-neural data generators and thus do not exercise the theorem's own condition that the true model be a neural network. Because the gap is addressable by either adding a calibration/coverage theorem or softening the abstract's claim, a conditional recommendation is appropriate rather than rejection.","tokens_in":32368,"tokens_out":8104,"duration_ms":83967,"concrete_test":"Attempt to derive a formal coverage statement for the §5.1 interval from Theorem 3.1 and equations (19)-(20): specify the convergence rate and limiting distribution of \\barθ*, and prove that the α/2 and 1-α/2 quantiles of the EFD yield asymptotic coverage 1-α for a known DNN target model under Assumptions 1-6. If the derivation requires additional conditions (e.g., a posterior CLT or Donsker-type condition) not stated in the paper, the headline claim should be narrowed to consistency of the inverse mapping.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The abstract and Section 6 claim that EFI maintains proper quantification of uncertainty for models of dimension O(n^ζ). The formal result, Theorem 3.1, only establishes that G*(Y_n,X_n,Z_n) = (1/n)∑_i \\hat g(y_i,x_i,z_i,w*_n) is a consistent estimator of the inverse function. The supplement (§1.1) then asserts that 'by Slutsky's theorem' the uncertainty of Z_n propagates to θ, and defines an EFD over θ via (S7)/(20). Consistency of a point estimator does not by itself make the distribution of \\barθ* over imputed z a valid confidence or prediction distribution. No theorem in the paper bounds the gap between the EFI-based ITE intervals of §5.1 and the nominal coverage level 1-α, gives a CLT or concentration result for \\barθ* in the growing-dimension regime, or handles misspecification when c(·), τ(·) are not neural networks—the case used in every simulation while §3.2 explicitly assumes the true model is a neural network. Even if Assumption 6 is granted, the central advertised contribution, valid uncertainty quantification for large DNNs, does not follow from the provided theory. This is a gap in the argument, not an issue of presentation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the Double-NN method, an extended fiducial inference (EFI) procedure for individual treatment effect (ITE) estimation, in which deep neural networks model the control response and treatment effect functions and an additional neural network approximates the inverse mapping from data and imputed latent errors to the model parameters. The central theoretical claim is that the dimension of the target DNN model may grow as O(n^ζ) for any 0 ≤ ζ < 1 while the method still provides proper uncertainty quantification, extending beyond the classical O(n^{1/2}) CLT range. The paper proves consistency of the learned inverse mapping under stated assumptions, presents an illustrative linear example, compares prediction intervals against conformal quantile regression on two simulations and two real datasets, and makes code publicly available.","tokens_in":32682,"tokens_out":8094,"duration_ms":89041,"significance":"If the advertised result were fully established, it would be a notable contribution to uncertainty quantification for large neural-network models and would position EFI as a theoretically grounded alternative to conformal methods for ITE inference. The paper includes reproducible code, detailed experimental settings, and a sensible benchmark against CQR, which are strengths. However, the main significance claim—proper uncertainty quantification for models of dimension O(n^ζ) with ζ approaching 1—is not actually derived in the theorems: Theorem 3.1 establishes only consistency of the inverse mapping, and the bridge to coverage guarantees is an informal Slutsky argument. The proof also relies on an unverified contractive-mapping assumption. These gaps currently limit the significance of the theoretical contribution.","major_comments":[{"comment":"The abstract and Section 6 claim that the paper theoretically proves proper quantification of uncertainty for model dimension O(n^ζ), but Theorem 3.1 only proves that G*(Y_n,X_n,Z_n) is a consistent estimator of the inverse function. The supplement's §1.1 invokes Slutsky's theorem to propagate uncertainty from Z_n to θ, but consistency of a point estimator does not imply that the empirical fiducial measure (S7) yields intervals with nominal coverage. No theorem in the paper provides a CLT or concentration result for the fiducial samples in the growing-dimension regime, and no theorem bounds the coverage error of the §5.1 prediction intervals. This is a load-bearing gap between the formal results and the central advertised contribution.","section":"Section 3.1, Theorem 3.1; Section 6; Supplement §1.1"},{"comment":"The proof of Theorem S2, and therefore of Theorem 3.1, requires that the imputation-regularized optimization mapping M(w_n) be differentiable and contractive with largest singular value bounded by λ* < 1 uniformly over W_n for sufficiently large n and almost every dataset. The paper states that this is 'reasonable to assume' and cites Nielsen (2000) and Liang et al. (2018a), but it does not verify the contractivity condition for the DNN/EFI setting. If this assumption fails, the convergence of \\hat{w}_n^{(t)} to \\tilde{w}_n^* is not established. A proof or a concrete sufficient condition for Assumption 6 is needed.","section":"Supplement §1.2.2, Assumption 6"},{"comment":"The theoretical development assumes that the true data-generating model is a neural network, thereby omitting approximation error. In contrast, all simulation studies use non-neural functions: Example 1 uses a linear control response with a sigmoid-transformed treatment effect, and Example 2 uses a rational control response. The paper asserts robustness to approximation error without proof. As a result, the numerical evidence does not directly validate the conditions of Theorem 3.1, and the claim of broad applicability requires either a misspecification analysis or a clear statement that the theory covers only correctly specified neural-network models.","section":"Section 3.2 and Section 5.2"},{"comment":"For the ReLU activation function, the proof of Lemma S3 relies on the informal assertion that a neuron belonging to the true neuron set cannot be constantly zero over all samples, and on the existence of a threshold q_min for the expected squared gradient. This threshold is not stated as a formal condition in Assumptions 1–6 or in Theorem S1/S2, leaving another unverified step in the chain of arguments leading to the consistency of \\hat{w}_n^*.","section":"Supplement, Lemma S3 proof"}],"minor_comments":[{"comment":"The text says convergence holds if max{σ_1,...,σ_H} ≺ τ(d_1,...,d_H) and \"ϵ→∞\"; this should read \"ϵ is sufficiently small\" or \"ϵ↓0\" to be consistent with the rest of the paper.","section":"Supplement §1.2.1, after equation (S10)"},{"comment":"The phrase \"priro distribution\" should be corrected to \"prior distribution\".","section":"Assumption 5"},{"comment":"The heading \"NSLM\" should be \"NLSM\" to match the dataset name used in the text and the reference to the National Study of Learning Mindsets.","section":"Section 5.3.2"},{"comment":"The spelling \"LaLonde\" is used in the section heading while \"Lalonde\" appears in the body text; please standardize. In addition, Figure 4 is based on a single train/test split, so the comparison shown there would be more informative with repeated splits or error bars.","section":"Section 5.3.1 and Figure 4"},{"comment":"The text sometimes refers to the intervals as \"confidence intervals\" and at other times as \"prediction intervals\"; since the target is the predictive ITE, the terminology should be made consistent.","section":"Section 5.1"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern is valid: the abstract and Section 6 overclaim relative to Theorem 3.1, and the Slutsky argument in the supplement is not a substitute for a coverage theorem. The paper also relies heavily on the authors' prior results for core lemmas, which is acceptable if the imported assumptions are verified; here, the contractive-mapping assumption (Assumption 6) is especially fragile and needs either proof or a clear sufficient condition. If the authors can supply a formal uncertainty-quantification theorem or substantially temper the UQ claim, and address Assumption 6, the paper could become publishable; otherwise the manuscript in its current form should not be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The headline claim is stronger than the theorems. Theorem 3.1 proves consistency of the EFI inverse mapping estimator, not that the resulting fiducial intervals have valid coverage. The supplement's Slutsky step propagates uncertainty from Z_n to θ, but consistency of a point estimator does not make the distribution of averaged imputations a calibrated confidence or prediction distribution. There is no theorem bounding the gap between the EFI ITE intervals and their nominal level, no growing-dimension CLT or concentration for the fiducial samples, and no misspecification theory for the non-neural data-generating processes used in every simulation. That is a genuine gap between the abstract and the results.\n\nWhat is actually new and good: the Double-NN construction—modeling c(x) and τ(x) with DNNs while learning the inverse map with another DNN—is a sensible extension of EFI, and the proof machinery extending consistency to dim(θ)=O(n^ζ) for ζ<1 via the StoNet equivalence is a real technical step beyond the earlier fixed-dimension framework. The simulations are carefully run against CQR and show the method often producing shorter intervals with coverage near nominal. The code is posted. The authors also openly admit the true-model-is-neural-network assumption and the heuristic nature of robustness to approximation error.\n\nSoft spots, in proportion: the coverage gap is the load-bearing one and it is serious enough that the phrase \"proper quantification of uncertainty\" in the abstract should be changed or backed by a theorem. Assumption 6, the contractive mapping condition, is asserted as \"reasonable\" and is never verified; the consistency proof of Theorem S2 depends on it. A referee should ask for either a check of that condition in the simulations or a proof that it holds for the DNNs used. The misspecification issue (theory assumes a neural network, simulations use non-neural functions) is acknowledged by the authors, so it is a limitation rather than a hidden flaw, but it further weakens the advertised claim.\n\nWho this is for: statisticians and ML researchers working on uncertainty quantification for neural networks and causal inference. The paper deserves a serious referee—the framework is novel, the simulations are informative, and the gap between claim and proof is addressable. I would send it out with instructions to focus on the consistency-to-coverage step and on Assumption 6.","headline":"The paper sells consistency as calibration; the underlying Double-NN method is real and worth refereeing, but the advertised UQ guarantee for O(n^ζ) models is not proven.","tokens_in":33184,"tokens_out":1867,"would_cite":false,"duration_ms":23440,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F99","62G08","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"A Double-NN method uses deep networks to estimate individual treatment effects with calibrated uncertainty, and proves the model size may grow as O(n^ζ) for any ζ < 1.","keywords":["extended fiducial inference","individual treatment effects","deep neural networks","uncertainty quantification","causal inference","Double-NN","stochastic gradient MCMC","large models"],"falsifier":"Estimate the largest singular value of $\\partial M(w_n)/\\partial w_n$ by finite differences during the IRO/EFI iterations on actual datasets from the paper's Examples 1 and 2; if the largest value reaches or exceeds 1 at any iterated $w_n$, the contraction premise behind Theorem 3.1 fails for that setting. Separately, run the method with $\\dim(\\theta) = O(n^{0.9})$ and heavy-tailed or dependent errors, and check whether the fiducial intervals keep nominal coverage; a coverage breakdown would show the stated large-model guarantee does not extend beyond the Gaussian-noise setting.","tokens_in":32139,"feed_emoji":"🎯","tokens_out":7273,"duration_ms":68923,"temperature":0.7,"pith_summary":"This paper introduces Double-NN, a method that estimates individual treatment effects and quantifies their uncertainty by combining deep neural networks with extended fiducial inference (EFI). The central claim is that the inverse mapping that recovers the parameters of the treatment and control networks remains a consistent estimator even when the model dimension grows with the sample size $n$ as $O(n^{\\zeta})$ for any $0 \\leq \\zeta < 1$, provided the EFI network's layers are narrow in the sense that $\\sum_{l=1}^H d_l \\prec n$. If true, this would extend trustworthy uncertainty quantification to large deep models, where classical central-limit-theorem guarantees stop at $\\zeta < 1/2$ and Bayesian posterior consistency is currently proved only up to $\\zeta < 1/4$. The paper also reports that Double-NN prediction intervals for individual treatment effects are shorter and closer to nominal coverage than conformal quantile regression intervals on simulated and real data.","feed_headline":"Deep models can grow nearly as fast as the data, with valid intervals","feed_subtitle":"Double-NN estimates individual treatment effects with deep nets and proves their uncertainty stays calibrated.","key_machinery":"The load-bearing object is the estimated inverse mapping $G^*(Y_n,X_n,Z_n)$, approximated by a DNN $\\hat{g}(y,x,z,w_n)$ with an output layer matching $\\dim(\\theta)$. EFI treats the unobserved errors $z_i$ as latent variables, defines the energy $U_n = \\sum_i d(y_i,x_i,z_i,\\bar{\\theta}) + \\eta \\sum_i \\|\\hat{\\theta}_i - \\bar{\\theta}\\|^2$, and with $\\epsilon \\downarrow 0$ the extended fiducial density of $\\theta$ is obtained by pushing the conditional distribution of $Z_n$ through $G^*$. To prove consistency for large models, the paper replaces the EFI network by an auxiliary stochastic neural network (StoNet) with small injected noises; each layer then forms a low-dimensional sparse regression, and a high-dimensional regression lemma plus an imputation-regularized optimization convergence argument yield consistency of $w^*_n$. The contractivity of the resulting iteration mapping $M(w_n)$, stated as Assumption 6, is what turns per-layer estimation error into a global consistent inverse mapping.","core_discovery":"The paper's central claim is that, under assumptions stated in the supplement, the EFI inverse mapping $G^*(Y_n,X_n,Z_n) = \\frac{1}{n}\\sum_{i=1}^n \\hat{g}(y_i,x_i,z_i,w^*_n)$ is a consistent estimator of the inverse function $\\theta = G(Y_n,X_n,Z_n)$ when $\\sum_{l=1}^H d_l \\prec n$ and the model dimension is $\\dim(\\theta) = d_H = O(n^{\\zeta})$ for $0 \\leq \\zeta < 1$. From this consistency, the extended fiducial density propagates uncertainty from the imputed latent errors $Z_n$ to $\\theta$, so fiducial samples of $\\bar{\\theta}$ produce prediction intervals for the predictive individual treatment effect $\\tilde{\\tau}_i$. The paper therefore claims a rigorous framework for uncertainty quantification of deep neural networks whose size follows a neural scaling law $\\dim(\\theta) \\prec n$, and supports the claim with simulations in which Double-NN intervals are shorter and better calibrated than conformal quantile regression intervals.","pith_inferences":["A direct test of the paper's main premise would be to compute the largest singular value of $\\partial M(w_n)/\\partial w_n$ on the simulation datasets; if it is not below 1 along the algorithm's trajectory, the consistency theorem's Assumption 6 is not satisfied in practice.","Because the EFI uncertainty is carried by imputed errors rather than by the approximating function class, the same Double-NN construction could be extended to average treatment effects and to propensity-score-adjusted estimators by changing only the function $b(\\theta)$ in the extended fiducial density integral; the paper notes this extension but does not implement it.","The paper's link to conformal prediction through generalized fiducial inference suggests EFI intervals might eventually be made distribution-free by an appropriate calibration step, though that is not established here.","For over-parameterized models with $\\dim(\\theta) \\succeq n$, imposing sparsity on $\\theta$ and applying post-selection EFI is a natural next step; the paper lists it as future work, so any such guarantee would be an extension, not a claim of this paper."],"forward_implications":["If Theorem 3.1 is correct, fiducial-based uncertainty quantification remains valid for deep models whose parameter count grows like $O(n^{\\zeta})$ for every $\\zeta < 1$, not just the $\\zeta < 1/2$ allowed by classical CLT arguments.","The Double-NN construction gives prediction intervals for predictive individual treatment effects in all three test-set situations: control-observed, treatment-observed, and covariates-only subjects.","The same inverse-mapping consistency supports inference for the conditional average treatment effect function $\\tau(x)$, not only point estimates, because each fiducial $\\bar{\\theta}$ sample yields a $\\tau$-network estimate.","The method targets the zero-energy manifold as $\\epsilon \\downarrow 0$, and the reported simulations recover the true latent errors even when the true model is not a neural network; the paper presents this as empirical robustness, not a proved theorem.","Under the empirical neural scaling law $\\dim(\\theta) = O(n^{\\zeta})$ with $\\zeta \\in [0.5,1)$, the theory gives a formal uncertainty-quantification framework for large-scale DNNs, provided a matching depth growth $H = O(n^{\\beta})$ with $\\beta < 1 - \\zeta$ is chosen."],"supporting_citations":[{"why":"Supplies the original EFI framework, including the energy function, extended fiducial density, and adaptive SGHMC training that Double-NN builds on.","marker":"Liang et al. (2024)"},{"why":"Provides the sparse deep learning consistency theory and the approximation-rate results that justify DNNs of size $O(n^{\\zeta})$ for many function classes.","marker":"Sun et al. (2022)"},{"why":"Establishes the imputation-regularized optimization convergence result used to prove consistency of the estimated EFI network weights.","marker":"Liang et al. (2018a)"},{"why":"Introduces the stochastic neural network (StoNet) whose asymptotic equivalence to the DNN carries the large-model consistency proof.","marker":"Liang et al. (2022b)"},{"why":"Provides Theorem 3.4, the sparse high-dimensional regression lemma used for each layer-wise regression in the StoNet proof.","marker":"Song and Liang (2023)"},{"why":"Defines conformal quantile regression for predictive individual treatment effects, the baseline method that Double-NN is compared against.","marker":"Lei and Candès (2021)"},{"why":"Documents the empirical neural scaling law $p = O(n^{\\zeta})$ with $0.5 \\leq \\zeta < 1$, which motivates the large-model regime the theory targets.","marker":"Hestness et al. (2017)"},{"why":"Shows that multilayer perceptrons of size $O(n^{\\zeta})$ accurately approximate Sobolev-class functions, supporting the choice of DNN size in Double-NN.","marker":"Farrell et al. (2021)"}],"fun_headline_variants":["Double-NN: deep nets scale fast, intervals stay valid","Deep nets can grow faster, still quantify uncertainty","Individual treatment effects with deep nets: valid intervals","Beyond classical limits: deep uncertainty quantification","Double-NN: scalable deep nets, reliable treatment effects"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof hinges on Assumption 6, which says the iteration map $M(w_n)$ is a strict contraction with largest singular value bounded below 1 uniformly; the paper calls this 'reasonable to assume' and cites earlier work but does not numerically verify it, and the theory also assumes the true data-generating process is a neural network while all simulations use non-neural functions.","fun_headline_variants_meta":{"raw":{"variants":["Double-NN: deep nets scale fast, intervals stay valid","Deep nets can grow faster, still quantify uncertainty","Individual treatment effects with deep nets: valid intervals","Beyond classical limits: deep uncertainty quantification","Double-NN: scalable deep nets, reliable treatment effects"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000288,"raw_usage":{"total_tokens":1723,"prompt_tokens":1013,"completion_tokens":710,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":629,"completion_tokens_details":{"reasoning_tokens":636}},"tokens_in":629,"tokens_out":710,"duration_ms":7828,"temperature":1.0,"reasoning_tokens":636,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:04:26.284886+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Estimate the largest singular value of $\\partial M(w_n)/\\partial w_n$ by finite differences during the IRO/EFI iterations on actual datasets from the paper's Examples 1 and 2; if the largest value reaches or exceeds 1 at any iterated $w_n$, the contraction premise behind Theorem 3.1 fails for that setting. Separately, run the method with $\\dim(\\theta) = O(n^{0.9})$ and heavy-tailed or dependent errors, and check whether the fiducial intervals keep nominal coverage; a coverage breakdown would show the stated large-model guarantee does not extend beyond the Gaussian-noise setting.","supporting_citations":[{"cited_title":"(2024), Exended Fiducial Inference: Toward an Automated Process of Statistical Inference, , in press","cited_arxiv_id":null,"evidence_quote":"Supplies the original EFI framework, including the energy function, extended fiducial density, and adaptive SGHMC training that Double-NN builds on."},{"cited_title":"and Liang, F","cited_arxiv_id":null,"evidence_quote":"Provides Theorem 3.4, the sparse high-dimensional regression lemma used for each layer-wise regression in the StoNet proof."},{"cited_title":"and Candès, E","cited_arxiv_id":null,"evidence_quote":"Defines conformal quantile regression for predictive individual treatment effects, the baseline method that Double-NN is compared against."},{"cited_title":"(2021), Deep Neural Networks for Estimation and Inference, Econometrica, 89, 181--213","cited_arxiv_id":null,"evidence_quote":"Shows that multilayer perceptrons of size $O(n^{\\zeta})$ accurately approximate Sobolev-class functions, supporting the choice of DNN size in Double-NN."}],"review_version":1}