{"id":"11c8633c-cca7-475c-9e74-794fa1a98e72","arxiv_id":"2505.20536","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"CoDEAL imputes missing counterfactuals in staggered panel data using DNN covariate adjustment and multi-output autoencoder factors, then estimates unit-specific treatment effects.","lead":"This paper proposes CoDEAL, a deep learning method that estimates unit-specific treatment effects in panel data with staggered policy adoption by imputing missing counterfactual outcomes with nonlinear factor models and neural network covariate adjustment. It is relevant because it extends causal panel analysis beyond linear factor models and average effects to heterogeneous, nonlinear settings, with a stated convergence guarantee.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 2.1's bound omits the treatment-effect contamination in the AE encoder input: Algorithm 1 step 5 feeds the encoder with columns containing observed treated outcomes, so the theorem's link to the algorithm is unproven.","rationale":"The reader's weakest-assumption flag on Assumption 3 (pervasiveness) is a legitimate identifiability concern: even with a perfect algorithm, the latent factors must be recoverable from observed control columns. However, I see a more immediate gap between the theorem and the procedure as written. The prediction step in Algorithm 1 feeds the autoencoder with columns that contain the observed treated outcomes, including the large static treatment effects τ_i, while the training loss (6) only uses control entries. The theorem's error bound has no τ-related term, so as stated it cannot be a bound for the estimator described unless the proof silently assumes that the encoder input is de-contaminated. This is not merely a technicality: in the simulations τ_i has mean 12 and standard deviation 5, far larger than the factor/covariate signal, so the encoder's latent code at treated times is potentially dominated by the treatment shift. The strong simulation results suggest the implementation either masks treated entries or the AE inadvertently averages them out; both possibilities need to be stated and analyzed. This concern reinforces the reader's CONDITIONAL verdict: the paper should be accepted only after the algorithm description, the proof, and the code are reconciled. I therefore keep the verdict unchanged rather than escalating, because the concern is addressable and the empirical results, if reproducible, suggest the method itself is viable under some additional assumption about treatment invariance of the encoder.","tokens_in":25916,"tokens_out":19165,"duration_ms":207971,"concrete_test":"Re-run the Config.1 simulation exactly as in Section 3.2 with the algorithm as written (encoder input = full \\tilde{Y}_{·t}), and compare with a variant where treated entries are masked before encoding (e.g., set to the unit's pre-treatment mean). If the as-written MAE is materially larger than the reported ~0.65, the experiments or code must be using a different encoder input than the paper states; also inspect Appendix B to see whether the proof of Theorem 2.1 evaluates ρ on \\tilde{Y}_{·t} or on the uncontaminated φ*(F_t).","verdict_should_be":"UNCHANGED","load_bearing_attack":"The least secure point is the algorithm-theory link. In Algorithm 1 (step 5) and Algorithm 2, the trained encoder ρ_hat is evaluated on the full covariate-adjusted column \\tilde{Y}_{·t} = Y_{·t} - \\hat{g}_t(X). For treated periods this vector contains, for treated units, \\tilde{Y}_it = φ_i(F_t) + τ_i + ε_it up to covariate-adjustment error. The AE training loss (6) sums only over control entries, so nothing enforces the learned encoder to be insensitive to the τ_i components in its input. Theorem 2.1 bounds \\hat{Y}_it(0) - φ_i(F_t) - g_t(X_i) with a rate that contains no term involving τ_i. The proof must therefore either (i) assume the encoder is applied to de-contaminated columns (which the algorithm does not state), or (ii) establish that the AE is robust to O(1) treatment shifts on a subset of input coordinates. Neither is implied by Assumptions 1-3. In the simulations τ_i ~ N(12,5) is an order of magnitude larger than the factor/covariate signal; if the encoder used treated entries as signal, the latent code would be dominated by τ_i and the imputation badly biased. The reported MAEs around 0.6 are only consistent with the encoder having learned to cancel the treatment shift (e.g., by averaging over enough treated units), an effect not represented in the theorem's rate.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CoDEAL, a deep-learning method for estimating unit-specific average treatment effects on the treated (ATT) in staggered-adoption panel data. The model is Y_it = φ_i^*(F_t^*) + g_t^*(X_i) + τ_i^* W_it + ε_it, with a nonlinear factor structure φ_i^*(F_t^*) captured by a multi-output autoencoder and covariate effects g_t^*(X_i) captured by a deep ReLU network. The method first estimates covariate effects on untreated entries, forms covariate-adjusted outcomes, trains a multi-output autoencoder on untreated entries, and imputes missing counterfactuals by evaluating the trained autoencoder on the covariate-adjusted columns and adding back the covariate prediction. Theorem 2.1 claims an average squared error bound for imputed counterfactuals under Assumptions 1–3, and extensive simulations plus an OxCGRT COVID-19 policy application are reported.","tokens_in":26266,"tokens_out":3715,"duration_ms":38408,"significance":"If the theoretical guarantee and the empirical claims hold, CoDEAL would be a meaningful extension of factor-model-based causal matrix completion to nonlinear latent structures and unit-specific heterogeneity, with a practical contribution in covariate adjustment. The paper is clearly written and ships code with the supplementary material; the simulation study is extensive in breadth (four configurations, six covariate DGPs, four factor DGPs, 50 replications). The main weaknesses are that the central theorem is not verifiable from the submitted manuscript because the proof appendix is absent, and that the algorithm as written evaluates the learned encoder on inputs containing treated observations with O(1) treatment shifts, a setting the theorem does not cover.","major_comments":[{"comment":"The proof and the required parameter choices are deferred to \"Appendix B,\" but no appendix is included in the submitted manuscript. Since Theorem 2.1 is the central theoretical claim and its rate depends on unspecified DNN class parameters, the claim cannot be checked. A revision must include the full proof, the precise network architectures, and the explicit parameter settings that yield the stated rate.","section":"§2.3, Theorem 2.1"},{"comment":"There is a mismatch between the trained model and the input used at prediction time. The autoencoder is fitted using only control entries in the loss (6), but step 5 evaluates the learned encoder on the full covariate-adjusted column \\tilde{Y}_{·t}, which for treated periods contains entries of the form φ_i^*(F_t^*) + τ_i^* + ε_it for treated units. Theorem 2.1 bounds the imputation error with no term involving τ_i^*, so the rate cannot hold for the algorithm as stated unless one assumes the encoder is robust to O(1) treatment shifts on a subset of input coordinates, an assumption nowhere stated or implied by Assumptions 1–3. Please either modify the algorithm to de-contaminate the columns before encoding (for example by iterating imputation) or provide an explicit robustness argument with accompanying conditions.","section":"§2.2.1, Algorithm 1 steps 4–5 and Eq. (6)"},{"comment":"Assumption 3 introduces an auxiliary matrix M^* and reconstruction map ρ^* that the algorithm never estimates or uses, and the theorem's rate contains an integer n whose relationship to N and T is not specified beyond \"some diverging integer n > 0.\" Because the DNN encoder is trained by minimizing (6), it is not established that the learned encoder approximates ρ^* in the sense needed to convert Assumption 3 into the stated error bound. Please define n explicitly, state its relation to N and T, and show how the trained network class can realize the reconstruction within the error tolerance.","section":"§2.3, Assumption 3 and Theorem 2.1"},{"comment":"The text says CoDEAL \"consistently achieves the lowest MAE and MSE across nearly all settings,\" but in Table 4, Config. 3 with linear factors and r=5 or r=10, MC-NNM reports lower MAE (0.599 vs 0.614 and 0.616 vs 0.625, respectively) and lower MSE (0.685 vs 0.721 and 0.733 vs 0.754). The more cautious wording in the later paragraph, \"performance comparable to MC-NNM in linear settings,\" is accurate; the earlier claim should be revised to match the table.","section":"§3.4, Table 4 and accompanying text"}],"minor_comments":[{"comment":"Several entries have unmatched parentheses, e.g., \"1.517(0.030\" in the log/linear-factor NN column and \"1.216(0.013 2.790\" in the poly/Vert-Reg row; these should be fixed.","section":"§3.3, Table 2"},{"comment":"In the Config. 2, nonlinear-covariate, polynomial row, the MC-NNM MSE is reported as \"1.654(0.07)\" with one decimal in the standard error, inconsistent with the other entries that use three decimals; please standardize.","section":"§3.4, Table 4"},{"comment":"The definition of D(ξ0,η0) uses the index j for the time dimension while the surrounding text uses η; please replace j with η for consistency.","section":"§2.2.2, Eq. (7)"},{"comment":"The policy effect estimates are presented without uncertainty quantification; a brief discussion of sensitivity or a bootstrap interval would help readers assess the stability of the counterfactual estimates.","section":"§4, Real data application"}],"recommendation":"major_revision","confidential_remarks":"The paper's main theorem and methodological novelty hinge on a proof that is entirely absent from the submitted version. I would be open to reconsidering after a complete appendix is provided and the algorithm–theory gap regarding treated columns is closed. The empirical study is useful but does not compensate for the missing theoretical support."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper does something genuinely useful—wraps DNN covariate adjustment around a multi-output-autoencoder factor model to estimate unit-level ATT in staggered panels—and the simulations are extensive and honest. But the main theorem, Theorem 2.1, is not actually in the text: proof and parameter choices are deferred to an Appendix B that the arXiv version doesn't contain. And there's a specific gap between Algorithm 1 and the theorem that worries me more than the missing appendix.\n\nWhat is actually new: CoDEAL's model equation (1) combines a nonlinear factor component phi_i(F_t), a potentially nonlinear covariate effect g_t(X_i), and unit-specific decoders in a multi-output autoencoder. Prior causal matrix completion mostly uses linear factors or a single decoder, so this combination is a real step forward. The four-block and staggered-adoption construction follows Yan and Wainwright, but adapting it to an autoencoder with covariate adjustment is sensible. The simulations are the strongest part: 50 replications, four factor DGPs, six covariate DGPs, four configurations, and five benchmarks. CoDEAL consistently wins in nonlinear factor settings and ties MC-NNM in linear settings. That is credible evidence the method works in the DGPs considered. The OxCGRT application is a nice illustration.\n\nSoft spots, in proportion. First, Theorem 2.1 is the paper's central claim, but its proof lives in an absent appendix. A reader cannot verify the rate or the parameter choices; the current version is incomplete on its own terms. Second, there is a real algorithm-theory gap. In Algorithm 1, the encoder is trained with a loss over control entries only, but at imputation it is applied to full post-treatment columns that contain treated units' tau_i shifts. The theorem's error bound contains no term involving tau_i. The pervasiveness assumption says nothing about the AE being robust to O(1) shifts on a subset of coordinates. In the simulations tau_i ~ N(12,5), an order of magnitude larger than the signal, so the good MAE values imply the encoder is somehow canceling the shift; the paper doesn't explain why the learned encoder would be insensitive. If the appendix contains a lemma proving this, fine, but from the main text the link is unproven. Third, minor: no code or hyperparameters appear in the arXiv text despite a claim that code is in the supplementary material, and the real-data effect sizes have no confidence intervals. The citation pattern looks clean; related work is credited appropriately.\n\nThis paper is for applied causal panel researchers and matrix-completion people who want a nonlinear, covariate-adjusted alternative to linear factor methods. The method is promising and the simulations are the strongest part. If I were an editor, I'd send it to referees, but with a note that the current version cannot be evaluated on its own because the appendix is missing. The treatment-contamination gap is the substantive issue a referee should push on. The paper deserves serious review; it is not ready in this form.","headline":"A useful nonlinear factor panel method with strong simulations, but the central theorem is only stated, not proved in the available text, and the algorithm-theory link has a treatment-contamination gap that needs closing.","tokens_in":26755,"tokens_out":4303,"would_cite":true,"duration_ms":47557,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G08","62H25","62G05"],"pacs":[],"model":"deepseek-v4-flash","headline":"CoDEAL, a covariate-adjusted deep causal learning procedure, imputes missing control outcomes in staggered-adoption panels and estimates unit-specific treatment effects, with an error bound that vanishes as the panel grows.","keywords":["causal panel data models","counterfactual estimation","heterogeneous treatment effects","matrix completion","missing not at random","multi-output autoencoders","nonlinear factor models","deep neural networks"],"falsifier":"Construct a four-block panel under the paper's model with $K=3$ latent factors, but let one factor load only on a small subset of units so Assumption 3 fails while the rest of the design is unchanged; run CoDEAL and record the average squared imputation error as $N$ and $T$ grow. If the error still follows the Theorem 2.1 rate, pervasiveness is not needed in the stated form; if the error stops shrinking or degrades sharply, the assumption is doing real work.","tokens_in":25713,"feed_emoji":"📈","tokens_out":15022,"duration_ms":136683,"temperature":0.7,"pith_summary":"This paper tries to establish that a deep-learning estimator, CoDEAL, can impute the unobserved control outcomes in a staggered-adoption panel and thereby recover unit-specific average treatment effects on the treated. The payoff of the claim is practical: policies rolled out in different units at different times, such as vaccine mandates across U.S. states, can be evaluated without assuming linear factor structures, parallel trends, or missing-at-random outcomes. The paper models each observed outcome as a nonlinear covariate function plus a nonlinear latent factor term plus a unit-specific treatment effect, estimates the covariate function with a deep ReLU network and the factor structure with a multi-output autoencoder, and imputes missing counterfactuals by matrix completion. Theorem 2.1 states that the average squared imputation error is bounded by terms decaying in the time length, the number of units, and the covariate smoothness, up to logarithmic factors, which makes the unit-specific ATT estimator consistent.","feed_headline":"Neural method imputes the counterfactuals a staggered panel hides","feed_subtitle":"A neural covariate step plus a multi-output autoencoder impute control outcomes with a proven convergence rate.","key_machinery":"The load-bearing object is the multi-output autoencoder, a function class $G^{K_1}_{AE}$ made of one shared encoder $\\rho: \\mathbb{R}^N \\to \\mathbb{R}^{K_1}$ and $N$ independent decoders $\\phi_i: \\mathbb{R}^{K_1} \\to \\mathbb{R}$, fitted on the untreated entries of the covariate-adjusted matrix. It carries the argument by representing the nonlinear factor model $\\phi_i^*(F_t^*)$ as a low-dimensional reconstruction problem: the encoder maps each observed column to the latent factor space, and the unit-specific decoders absorb cross-unit heterogeneity that a single-output autoencoder cannot. The second key mechanism is Assumption 3, a pervasiveness condition requiring a sparse matrix $M^*$ and a Hölder reconstruction map $\\rho^*$ that invert the loading map on the factor range $[-B,B]^K$ up to order $n^{-1}$, which is what lets the autoencoder recover $F_t^*$. The covariate-adjustment step, equation (4), estimates $g_t^*$ on untreated entries before factor recovery so that covariate effects do not contaminate the latent structure.","core_discovery":"On the paper's own terms, the central discovery is that a panel outcome can be decomposed as $Y_{it} = \\phi_i^*(F_t^*) + g_t^*(X_i) + \\tau_i^*\\,\\mathbf{1}\\{W_{it}=1\\} + \\epsilon_{it}$, where $g_t^*$ is a $(\\beta,C)$-Hölder smooth function of covariates, $\\phi_i^*$ is a unit-specific nonlinear loading on a $K$-dimensional latent factor $F_t^*$, and $\\tau_i^*$ is the unit's ATT. CoDEAL removes $g_t^*$ with a fully connected ReLU network fitted on untreated entries, then fits a multi-output autoencoder -- one shared encoder and $N$ separate decoders -- on the covariate-adjusted control outcomes to recover the factors and loadings. The imputed control outcome is $\\hat Y_{it}(0) = \\hat\\phi_i(\\hat\\rho(\\tilde Y_{\\cdot t})) + \\hat g_t(X_i)$. Under the paper's pervasiveness assumption, the encoder can approximately invert the loading map, and Theorem 2.1 bounds the average squared distance between the imputed values and the true noiseless structure by $O_P\\big((T^{-2\\beta/(2\\beta+K)} + T^{-1}n + N^{-1}K_1 + n^{-1})\\log^4(NT) + N^{-2\\beta/(2\\beta+k)}\\log^5 N\\big)$. Consequently the plug-in estimator $\\hat\\tau_i = \\sum_{t:\\,W_{it}=1}\\big(Y_{it}(1)-\\hat Y_{it}(0)\\big)/\\sum_t W_{it}$ is consistent for $\\tau_i^*$ in the four-block and general staggered-adoption designs.","pith_inferences":["Editorial inference: Theorem 2.1's rate contains $T^{-1}n$ and $N^{-2\\beta/(2\\beta+k)}$ terms, so practical convergence is governed by how well Assumption 3's reconstruction map recovers the factors; a validation-time estimate of $\\sup_{x\\in[-B,B]^K}\\|\\rho^*(M^*\\phi^*(x))-x\\|$ would tell users how much of the guarantee is actually available for a given panel.","Editorial inference: the method fits $N$ separate decoders, so its parameter count grows linearly with the number of units; a natural extension the paper does not pursue is to share decoder strength across units through a covariate-dependent or hierarchical parameterization for very large panels.","Editorial inference: because covariate functions are estimated only on untreated entries, a hold-out falsification exercise--reserve a pre-treatment block, impute it as missing, and compare with the observed entries--would isolate the covariate-adjustment and factor-recovery steps from treatment-effect estimation."],"forward_implications":["Unit-specific ATTs are consistently estimated in staggered-adoption designs by reducing each treated block to a four-block subproblem and imputing its control outcomes, without requiring missing-at-random outcomes or a linear factor model.","In the four-block and staggered simulations, CoDEAL matches the best linear baseline in linear settings and reduces mean absolute error and mean squared error by roughly 10–40% when factor or covariate effects are nonlinear.","The nonlinear covariate and nonlinear factor components nest existing linear factor-model causal matrix completion methods as special cases, and the paper argues the same machinery extends to tensor or higher-order panel data.","In the real-data application, the estimated counterfactual says that without mandatory vaccination, confirmed cases in the policy states would have been about 12.2% higher and deaths about 34.6% higher by 11 December 2021."],"supporting_citations":[{"why":"formulates causal panel imputation as matrix completion under missing-not-at-random outcomes, the setting CoDEAL extends with nonlinear factor structures.","marker":"(Athey et al., 2021)"},{"why":"supplies the linear factor model and the pervasiveness condition that Assumption 3 generalizes to nonlinear loadings.","marker":"(Bai, 2003)"},{"why":"introduces multi-output autoencoders for nonlinear factor models, the architecture behind CoDEAL's unit-specific decoders.","marker":"(Xiu and Shen, 2024)"},{"why":"gives the four-block construction for staggered adoption that Algorithm 2 uses to reduce imputation to four-block designs.","marker":"(Yan and Wainwright, 2024)"},{"why":"establishes causal matrix completion under structured missingness, motivating CoDEAL's treatment of staggered adoption.","marker":"(Agarwal et al., 2023)"},{"why":"supplies the government-response policy panel used in the real-data application.","marker":"(Hale et al., 2021)"}],"fun_headline_variants":["Deep causal panel model fills missing control outcomes","Autoencoder and ReLU net impute counterfactual outcomes","CoDEAL autoencoder imputes panel counterfactuals","Covariate-adjusted deep net recovers control outcomes","Neural imputation of missing counterfactuals in panels"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is the pervasiveness condition in Assumption 3: a sparse linear combination of the unit loadings, with coefficients of order $1/n$ spread over about $n$ units, can be inverted by a smooth map to recover the latent factors to within order $1/n$; if some factors load only on a few units, the autoencoder cannot recover them and the Theorem 2.1 rate collapses.","fun_headline_variants_meta":{"raw":{"variants":["Deep causal panel model fills missing control outcomes","Autoencoder and ReLU net impute counterfactual outcomes","CoDEAL autoencoder imputes panel counterfactuals","Covariate-adjusted deep net recovers control outcomes","Neural imputation of missing counterfactuals in panels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001191,"raw_usage":{"total_tokens":5011,"prompt_tokens":1137,"completion_tokens":3874,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":753,"completion_tokens_details":{"reasoning_tokens":3795}},"tokens_in":753,"tokens_out":3874,"duration_ms":51847,"temperature":1.0,"reasoning_tokens":3795,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:52:40.396184+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a four-block panel under the paper's model with $K=3$ latent factors, but let one factor load only on a small subset of units so Assumption 3 fails while the rest of the design is unchanged; run CoDEAL and record the average squared imputation error as $N$ and $T$ grow. If the error still follows the Theorem 2.1 rate, pervasiveness is not needed in the stated form; if the error stops shrinking or degrades sharply, the assumption is doing real work.","supporting_citations":[{"cited_title":"Matrix completion methods for causal panel data models","cited_arxiv_id":null,"evidence_quote":"formulates causal panel imputation as matrix completion under missing-not-at-random outcomes, the setting CoDEAL extends with nonlinear factor structures."},{"cited_title":"Causal matrix completion","cited_arxiv_id":null,"evidence_quote":"establishes causal matrix completion under structured missingness, motivating CoDEAL's treatment of staggered adoption."},{"cited_title":"A global panel database of pandemic policies (oxford covid-19 government response tracker)","cited_arxiv_id":null,"evidence_quote":"supplies the government-response policy panel used in the real-data application."}],"review_version":1}