REVIEW 3 major objections 6 minor 41 references
Learning Who to Treat When Treatment is Missing
T0 review · 3 major / 6 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read When treatment assignment is partially missing, estimators that use the missing-at-random assumption are never less efficient—and usually more efficient—than complete-case estimators that discard partial observations, even when the stronger
desk verdict MAR beats complete-case under MCCAR is a real, non-obvious result, but the abstract oversells it and the key n^{-1/4} nuisance rate is unchecked in the main experiments. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central objects are two influence functions—φ_MAR and φ_MCCAR—for the value of a treatment policy under missing treatment data. Influence functions are bias-corrected scores that yield √n-consistent, asymptotically normal estimators when nuisance functions are estimated well. The proof shows Var(φ_MAR) = Var(φ_MCCAR) − g(P), isolating g(P) ≥ 0 as the efficiency gap that grows with the missingness rate, the augmented propensity score, and their correlation.
What would settle it
Simulate the synthetic MCCAR setting with known νd and ηd, plug in the true nuisance functions, and compare the empirical variance ratio of the MAR and MCCAR estimators to the formula (1 − g(P)/Var(φ_MCCAR))^{-1}; if the ratio differs by more than simulation error, the variance derivation is wrong.
Extended reading notes
Core claim
The paper proves that when treatment is missing in a way that depends only on observed covariates (MCCAR), the estimator derived under the weaker missing-at-random (MAR) assumption—which reuses partially observed units through an outcome-dependent treatment propensity—has asymptotic variance no larger than, and usually strictly smaller than, the complete-case estimator. The asymptotic relative efficiency is (1 − g(P)/Var(φ_MCCAR))^{-1}, where g(P) ≥ 0 is the missingness-weighted signal that the complete-case estimator throws away. Hence defaulting to complete-case analysis is wasteful even when the stronger assumptions that justify it hold.
Load-bearing premise
The efficiency gain requires the treatment-assignment-matching probability γd(X) = P(A=d(X)|X) to be estimated at nearly n^{-1/4} rate; if nuisance estimators converge more slowly in high-dimensional or complex settings, the MAR estimator may not reach √n-consistency and the advantage over complete cases can vanish.
Editorial extensions
If this is right
- Under both MAR and MCCAR missingness, the MAR-based estimator is valid; the MCCAR complete-case estimator is only valid under MCCAR.
- Even when MCCAR holds, the MAR estimator has asymptotic relative efficiency (1 − g(P)/Var(φ_MCCAR))^{-1} ≥ 1, so discarding partial observations is never the efficient choice.
- The efficiency advantage grows with the missingness rate, the probability that treatment matches the policy, and the correlation between these two quantities.
- Misspecified missingness mechanisms produce bias that does not disappear with sample size; correctly modeling the missingness mechanism is necessary for unbiased CATE estimates.
- In semi-synthetic policy experiments, MAR-based estimators maintain positive gains over random assignment with only 10–30% of treatments observed, while complete-case and propensity-based alternatives frequently fail.
Reading between the lines
- A data-driven version of the ARE formula could serve as a pre-analysis check: estimate g(P) and Var(φ_MCCAR) from the sample, and use the predicted efficiency gain to decide whether MAR-based estimation is worth the extra nuisance modeling.
- The same influence-function decomposition suggests that when γd is hard to estimate, an adaptive estimator that falls back on the complete-case influence function for high-dimensional strata might preserve some of the efficiency gain; the paper does not propose this.
- For threshold-based budget policies, the asymptotic normality under estimated policies implies that the MAR pseudo-outcomes could be plugged into ranking objectives rather than generic CATE regression; the paper leaves this as future work.
- The MCCAR-versus-MAR efficiency gap is a special case of a more general principle: discarding data that is only partially informative is wasteful whenever a valid imputation model exists; the same ARE argument may carry over to missing outcomes or missing confounders.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies policy learning when treatment indicators are missing. It proposes two families of estimators for the value of a fixed treatment policy and for the CATE: a complete-case MCCAR estimator based on the outcome regression νd(X) and a MAR estimator based on the nuisance functions λd, π, βd, γd from Kennedy (2020). The paper derives influence functions, proves √n-consistency and asymptotic normality under product-rate conditions (Propositions 3.2 and 3.3), and then proves an asymptotic relative efficiency result (Theorem 3.4) claiming that, under MCCAR, the MAR estimator is never less efficient than the MCCAR estimator, with a closed-form ARE. The methods are instantiated as DR-Learner variants for CATE estimation and evaluated on synthetic, high-dimensional, and semi-synthetic RCT datasets, with code provided.
Significance. If the efficiency theorem holds, the paper makes a practically important point: when treatment missingness is MCCAR, discarding partially observed units (complete-case analysis) is asymptotically suboptimal, and an MAR estimator that uses all units can be strictly more efficient. This is a substantive extension of Kennedy (2020) from ATE to policy value, and the paper supplies detailed von Mises expansions and variance derivations, as well as reproducible code. The empirical section is extensive and includes both synthetic rate studies and two realistic semi-synthetic benchmarks. The main caveats are that the efficiency claim is conditional on nuisance estimation rates and is proven for fixed policies rather than for the learned optimal policy; these caveats are not fully reflected in the abstract and contribution statements.
major comments (3)
- [Theorem 3.4 and Section 5] The abstract and Section 1 (contribution 2) state that the MAR estimator is 'both valid and more efficient than the MCCAR estimator when MCCAR assumptions hold.' However, Theorem 3.4 and Proposition 3.3 make the efficiency gain conditional on ∥bγd−γd∥ = o_P(n^{−1/4}) (Proposition 3.3, condition 3). In the high-dimensional experiment (Section 4.1.2), γd is estimated with Random Forests with d=50 covariates and no rate verification is provided. If γd converges slower than n^{−1/4}, bψMAR can carry first-order bias, its variance is not Var(φMAR), and the ARE formula does not apply. The Discussion (Section 5) acknowledges the condition, but the abstract and contribution statements do not. Please qualify the claims and, if the recommendation is to prefer MAR-based estimation broadly, provide empirical evidence or a formal argument that the rate condition is met in the regimes covered by the e
- [Section 3.2 and Theorems A.5–A.6] Theorem 3.4 establishes an efficiency comparison for the value of a fixed policy d. The paper's title and Section 3.2, however, frame the contribution as 'learning who to treat.' The asymptotic results for the data-dependent policy bd^∗ (Theorems A.5 and A.6) give only asymptotic normality, not a relative efficiency comparison between the MAR and MCCAR estimators of V(bd^∗). Thus the formal justification for preferring MAR-based estimation in policy learning is incomplete: the proven efficiency gain applies to fixed-policy evaluation, while the learned-policy setting is left without an efficiency statement. Either prove an efficiency result for the estimated policy under a margin condition, or temper the policy-learning claims to fixed-policy evaluation.
- [Remark 3.5] The convergence rates of the CATE estimators bτMAR and bτMCCAR are disposed of in Remark 3.5 by citing Kennedy (2020), Nie and Wager (2021), and Foster and Syrgkanis (2023). Since the CATE estimators are a main contribution (Section 3.2 and all of Section 4), the paper should state the precise assumptions under which these rates hold, including the smoothness/sparsity conditions on the CATE function and the rate conditions on the first-stage nuisances. As written, the empirical claims about CATE performance are not tied to a formal theorem, which weakens the theoretical support for the decision-rule part of the paper.
minor comments (6)
- [Abstract and Section 1] The phrase 'more efficient than the MCCAR estimator when MCCAR assumptions hold' should be qualified with 'under the regularity conditions of Proposition 3.3.'
- [Figure 1 caption] The right panel caption says MAR 'allows for dependence on the observed outcome Y either directly or through unobserved U.' Under the stated MAR assumption R⊥⊥A|(X,Y), dependence through an unobserved U is not automatically allowed unless U is conditionally independent of A given (X,Y); the phrase is confusing and should be clarified or the DAG adjusted.
- [Section 4.1.1] The text says the noise ε∼N(−n^{−α}, n^{−2α}) 'precisely control[s] the rate of convergence,' but the reported RMSE curves are median over 500 iterations; it would help to state whether the median is taken of the L2 error or of RMSE, and to give the standard error bands (or at least a mention of variability).
- [Appendix C, Algorithm 3] The nested cross-fitting loop is described in words but not with formal notation over the two inner folds; adding equations for bβ(j), bγ(j), and the averaging step would improve reproducibility.
- [Various] There are repeated typos: 'V oting' should be 'Voting'; 'R-Positivity' is sometimes written with a dash and sometimes without; 'Kennedy [2020]' is used both as a method and a reference, which can confuse the provenance of the MAR IF.
- [Section 2.2, Table 1] The asterisk footnote in Table 1 is helpful but the table layout makes the statement 'MAR is strictly weaker' hard to parse; consider adding a sentence explaining that the R-positivity under MAR is implied by MCCAR's R-ignorability and R-positivity, so the overall MAR assumption remains weaker.
Circularity Check
No significant circularity: the efficiency comparison is derived from the variance formulas, not assumed.
full rationale
The derivation chain is self-contained on the points that matter for circularity. The MAR estimand ψMAR=E[βd/γd] is identified from Assumptions 2.1/2.3 in Appendix A.1, and the policy-value influence function φMAR is adapted and its variance is computed in Appendix A.2.2. The remainder bound for bψMAR uses Lemma 1 of Kennedy (2020), a prior theorem by a co-author; that is independent theorem support with stated assumptions, not a definitional identity, and the paper separately derives the policy-value extension and the variance expressions. The central efficiency claim (Theorem 3.4) is proved by deriving Var(φMAR)=Var(ϕMCCAR)−g(P) with g(P)≥0, which is a direct algebraic comparison of the two variance formulas, not a fitted parameter renamed as a prediction. The n^{-1/4} condition on γd is a hypothesis of Proposition 3.3, and the Discussion explicitly acknowledges that the efficiency gain is asymptotic and requires nuisance convergence conditions; the abstract's unconditional phrasing is an overstatement of scope but not a circular step. The experiments in Fig. 2 are simulations that vary λd, π, and their correlation and evaluate the variance formulas; they do not fit the theorem's conclusion. No circular step is present.
Assumptions & free parameters
assumptions (6)
- domain assumption Standard causal assumptions: consistency, A-ignorability, A-positivity (Assumption 2.1)
- domain assumption MCCAR R-ignorability and R-positivity (Assumption 2.2)
- domain assumption MAR R-ignorability and R-positivity (Assumption 2.3)
- domain assumption Bounded outcome Y (Remark 3.1)
- standard math Nuisance functions estimated on independent sample (sample splitting) (Remark 3.1)
- domain assumption Margin condition and no-ties assumption for estimated policies (Assumption A.4 and Appendix A.3.1)
Cite this review
Pith. "Pith review of Learning Who to Treat When Treatment is Missing." pith.science (2026). https://pith.science/paper/AMA7LPNI
@misc{pith2026260714346,
author = {Pith},
title = {Pith review of: Learning Who to Treat When Treatment is Missing},
year = {2026},
howpublished = {\url{https://pith.science/paper/AMA7LPNI}},
note = {Machine review of arXiv:2607.14346}
}
read the original abstract
Policy learning methods are increasingly used to inform treatment allocation under budget constraints. Most proposed methods assume complete treatment data, yet applications frequently suffer from missingness that can bias estimates and lead to suboptimal policies. We address this gap by extending efficient estimators for average treatment effect (ATE) estimation to policy value and conditional average treatment effect (CATE) estimation under missing at random (MAR) and missing completely conditionally at random (MCCAR) treatment data. Through asymptotic efficiency analysis, we prove that the MAR estimator, which leverages partially-observed units, is both valid and more efficient than the MCCAR estimator when MCCAR assumptions hold. This result provides formal justification for preferring MAR-based estimation in policy learning under both missing data settings. Our comprehensive experiments using synthetic and semi-synthetic datasets confirm that correctly specifying the missingness mechanism is crucial: misspecified estimators remain biased regardless of sample size, while our estimators achieve near-oracle performance when assumptions are satisfied. Our work provides practitioners with theoretically grounded, empirically validated tools for robust policy learning in the presence of missing treatment data.
Figures
Figures from the paper (9 more)
Reference graph
Works this paper leans on
-
[1]
Journal of Educational Psychology , author =
Estimating causal effects of treatments in randomized and nonrandomized studies , volume =. Journal of Educational Psychology , author =. 1974 , pages =. doi:10.1037/h0037350 , number =
doi:10.1037/h0037350 1974
-
[2]
The international journal of biostatistics , volume=
Efficient nonparametric causal inference with missing exposure information , author=. The international journal of biostatistics , volume=. 2020 , publisher=
2020
-
[3]
Estimating
Kuzmanovic, Milan and Hatt, Tobias and Feuerriegel, Stefan , booktitle =. Estimating. 2023 , editor =
2023
-
[4]
Handbook of statistical methods for precision medicine , pages=
Semiparametric doubly robust targeted double machine learning: a review , author=. Handbook of statistical methods for precision medicine , pages=. 2024 , publisher=
2024
-
[5]
International Conference on Learning Representations , volume=
Comparing targeting strategies for maximizing social welfare with limited resources , author=. International Conference on Learning Representations , volume=
-
[6]
The International Journal of Biostatistics , author =
Optimal. The International Journal of Biostatistics , author =. 2016 , pages =. doi:10.1515/ijb-2015-0007 , language =
-
[7]
Who. Econometrica , author =. 2018 , keywords =. doi:10.3982/ECTA13288 , language =
-
[8]
The Annals of Statistics , author =
Optimal aggregation of classifiers in statistical learning , volume =. The Annals of Statistics , author =. 2004 , pages =. doi:10.1214/aos/1079120131 , number =
arXiv 2004
Show all 41 references
-
[9]
The Annals of Statistics , author =
Bayesian. The Annals of Statistics , author =. 1978 , pages =
1978
-
[10]
, year =
Tsiatis, Anastasios A. , year =. Semiparametric theory and missing data , isbn =
-
[11]
and Robins, James M
Van Der Laan, Mark J. and Robins, James M. , year =. Unified
-
[12]
The Econometrics Journal , author =
Double/debiased machine learning for treatment and structural parameters , volume =. The Econometrics Journal , author =. 2018 , pages =. doi:10.1111/ectj.12097 , language =
2018 doi
-
[13]
Statistical Methods in Medical Research , author =
Causal inference with missing exposure information:. Statistical Methods in Medical Research , author =. 2016 , pages =. doi:10.1177/0962280213513758 , language =
2016 doi
-
[14]
Statistics in Medicine , author =
Doubly robust estimators of causal exposure effects with missing data in the outcome, exposure or a confounder , volume =. Statistics in Medicine , author =. 2012 , pages =. doi:10.1002/sim.5643 , language =
2012 doi
-
[15]
Electronic Journal of Statistics , volume=
Towards optimal doubly robust estimation of heterogeneous causal effects , author=. Electronic Journal of Statistics , volume=. 2023 , publisher=
2023
-
[16]
Econometrica , volume=
Policy learning with observational data , author=. Econometrica , volume=. 2021 , publisher=
2021
-
[17]
Nonparametric
Curth, Alicia and Schaar, Mihaela van der , month = mar, year =. Nonparametric. Proceedings of
-
[18]
Advances in neural information processing systems , volume=
Why do tree-based models still outperform deep learning on typical tabular data? , author=. Advances in neural information processing systems , volume=
-
[19]
Journal of the American Statistical Association , author =
Experimental. Journal of the American Statistical Association , author =. 2023 , pages =. doi:10.1080/01621459.2021.1923511 , language =
2023
-
[20]
Proceedings of the National Academy of Sciences , author =
Meta-learners for. Proceedings of the National Academy of Sciences , author =. 2019 , note =. doi:10.1073/pnas.1804597116 , language =
2019 doi
-
[21]
International Conference on Machine Learning , pages=
Estimating individual treatment effect: generalization bounds and algorithms , author=. International Conference on Machine Learning , pages=. 2017 , organization=
2017
-
[22]
Biometrika , volume=
Quasi-oracle estimation of heterogeneous treatment effects , author=. Biometrika , volume=. 2021 , publisher=
2021
-
[23]
Journal of the American Statistical Association , volume=
Estimation and inference of heterogeneous treatment effects using random forests , author=. Journal of the American Statistical Association , volume=. 2018 , publisher=
2018
-
[24]
Biometrics , author =
Causal machine learning for heterogeneous treatment effects in the presence of missing outcome data , volume =. Biometrics , author =. 2025 , pages =. doi:10.1093/biomtc/ujaf098 , language =
2025 doi
-
[25]
Proceedings of the AAAI Conference on Artificial Intelligence , author =
Preventing. Proceedings of the AAAI Conference on Artificial Intelligence , author =. 2024 , pages =. doi:10.1609/aaai.v38i20.30246 , language =
2024 doi
-
[26]
International Journal of Epidemiology , author =
Machine-learning-based high-benefit approach versus conventional high-risk approach in blood pressure management , volume =. International Journal of Epidemiology , author =. 2023 , pages =. doi:10.1093/ije/dyad037 , number =
2023 doi
-
[27]
npj Digital Medicine , author =
Impact of a deep learning sepsis prediction model on quality of care and survival , volume =. npj Digital Medicine , author =. 2024 , note =. doi:10.1038/s41746-023-00986-6 , language =
2024 doi
-
[28]
De-Arteaga, Maria and Fogliato, Riccardo and Chouldechova, Alexandra , month = apr, year =. A. Proceedings of the 2020. doi:10.1145/3313831.3376638 , language =
2020
-
[29]
Journal of Big Data , author =
A survey on missing data in machine learning , volume =. Journal of Big Data , author =. 2021 , pages =. doi:10.1186/s40537-021-00516-9 , number =
2021 doi
-
[30]
Journal of the American Medical Informatics Association: JAMIA , author =
Missing clinical and behavioral health data in a large electronic health record (. Journal of the American Medical Informatics Association: JAMIA , author =. 2016 , pmid =. doi:10.1093/jamia/ocw021 , language =
2016 doi
-
[31]
Statistics in Medicine , author =
Missing data in the exposure of interest and marginal structural models:. Statistics in Medicine , author =. 2010 , pages =. doi:10.1002/sim.3801 , language =
2010 doi
-
[32]
Journal of Business & Economic Statistics , author =
Missing. Journal of Business & Economic Statistics , author =. 2010 , note =
2010
-
[33]
and Abbott, Rachel and Mucunguzi, Atukunda and Mwangwa, Florence and Kakande, Elijah and Charlebois, Edwin D
Landsiedel, Kirsten E. and Abbott, Rachel and Mucunguzi, Atukunda and Mwangwa, Florence and Kakande, Elijah and Charlebois, Edwin D. and Marquez, Carina and Kamya, Moses R. and Balzer, Laura B. , month = jun, year =. Causal. doi:10.48550/arXiv.2506.03336 , urldate =
-
[34]
American Political Science Review , author =
Social. American Political Science Review , author =. 2008 , keywords =. doi:10.1017/S000305540808009X , language =
2008 doi
-
[35]
Achilles, C. M. and Bain, Helen Pate and Bellott, Fred and Boyd-Zaharias, Jayne and Finn, Jeremy and Folger, John and Johnston, John and Word, Elizabeth , month = oct, year =. Tennessee's
-
[36]
Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining , pages=
Optuna: A next-generation hyperparameter optimization framework , author=. Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining , pages=
-
[37]
Machine Learning , author =
Random. Machine Learning , author =. 2001 , keywords =. doi:10.1023/A:1010933404324 , language =
2001 doi
-
[38]
Proceedings of the 22nd
Chen, Tianqi and Guestrin, Carlos , month = aug, year =. Proceedings of the 22nd. doi:10.1145/2939672.2939785 , language =
-
[39]
Higher order influence functions and minimax estimation of nonlinear functionals , volume =
Robins, James and Li, Lingling and Tchetgen, Eric and van der Vaart, Aad , month = jan, year =. Higher order influence functions and minimax estimation of nonlinear functionals , volume =. Probability and. doi:10.1214/193940307000000527 , pages =
-
[40]
Asymptotic. U.C. Berkeley Division of Biostatistics Working Paper Series , author =
-
[41]
The Annals of Statistics , volume=
Orthogonal statistical learning , author=. The Annals of Statistics , volume=
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.