Pith. sign in

REVIEW 3 major objections 3 minor 23 references

Causal machine learning for heterogeneous treatment effects in the presence of missing outcome data

T0 review · 3 major / 3 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Two modified learners estimate treatment effects without bias when outcomes are missing at random.

desk verdict The mDR-learner is a solid IPCW extension with a real excess-risk bound, but the mEP-learner's targeting step as written does not zero the EIF drift, so its oracle-efficiency claim is unsupported. read the letter →

arxiv 2412.19711 v2 pith:CQEUOEFO submitted 2024-12-27 stat.ML cs.LG

Matthew Pryce , Karla Diaz-Ordaz , Ruth H. Keogh , Stijn Vansteelandt This is my paper · ORCID
classification stat.MLcs.LG MSC 62D2062G0562G20
keywords causalmachinelearningheterogeneoustreatmenteffectsmissingoutcomedataatrandomoracleefficiencydoublyrobustestimationtargetedinverseprobabilityofcensoringweights
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tackles a common but underappreciated problem: when outcome data is missing at random (MAR), standard machine-learning estimators of the conditional average treatment effect (CATE) can give biased estimates for subgroups with high dropout. The authors propose two modified learners, the mDR-learner and the mEP-learner, that multiply the usual correction term by the inverse probability of censoring, so that complete cases are reweighted to represent the full population. They prove that the mDR-learner is oracle efficient under rate conditions, meaning its estimation error behaves as if the nuisance functions were known, and they argue the same holds for the mEP-learner. In simulations the new learners outperform available-case and imputation-based versions of standard learners, and in the GBSG2 breast cancer example they yield smaller treatment-effect estimates than available-case analyses, consistent with correction for selective dropout.

What carries the argument

The efficient influence function of the mean-squared-error risk for the CATE under missing outcomes, given in the paper as eq. (7), whose correction term is (A - pi(Z)) C / (pi(Z)(1 - pi(Z)) G(A,Z)) times (Y - mu_A(Z)). The mDR-learner builds its pseudo-outcomes directly from this EIF; the mEP-learner uses the same expression to define the weight H(A,C,Z) = C A/(G pi) + C(1-A)/(G(1-pi)) in a weighted linear regression onto a sieve basis $\varphi$(X), updating the outcome predictions so the sample average of the drift term converges to zero. The excess-risk bound of Theorem 1, obtained via the orthogonal statistical learning framework, carries the oracle-efficiency argument.

What would settle it

Run the mEP-learner on data where the true CATE is a step function that the trigonometric sieve basis cannot approximate, with fast nuisance estimators; if the sample average of the drift term after the targeting step is not zero, or if the excess risk fails to shrink at the oracle rate as n grows, the mEP oracle-efficiency claim is refuted. The same check applies to the mDR-learner, where the bound is proven and should hold.

Watch

Extended reading notes

Core claim

The paper's central claim is that incorporating inverse probability of censoring weights into the DR-learner and EP-learner pseudo-outcomes produces estimators whose excess risk is governed by products of nuisance-function errors, not by the errors themselves. Theorem 1 bounds the mDR-learner's excess risk by the oracle regression rate plus terms such as $Rate_pi^{4}$, $Rate_pi^{2}$ times $Rate_mu^{2}$, $Rate_pi^{2}$ times $Rate_G^{2}$, $Rate_mu^{2}$ times $Rate_G^{2}$, and $Rate_G^{4}$; when those product rates beat the oracle rate, the learner is oracle efficient. The same oracle-efficiency claim is made for the mEP-learner, whose sieve-based iTMLE update uses the weight H(A,C,Z) = C A/(G pi) + C(1-A)/(G(1-pi)) to zero out the drift term. The authors support the theoretical claim with simulations in which the mDR and mEP learners match or beat available-case and imputation-based versions of the standard learners, and with a GBSG2 analysis showing that correcting for selective dropout reduces the estimated benefit of hormonal therapy.

Load-bearing premise

For the mEP-learner, the claim of oracle efficiency depends on an unproved premise: that the sieve-based iTMLE update, reweighted by H(A,C,Z), actually drives the sample average of the drift term to zero and delivers the same guarantees as the EP-learner; if that targeting step fails, the mEP-learner may not be oracle efficient.

Editorial extensions

If this is right

  • Under MAR outcomes, mDR and mEP provide CATE estimates that do not require imputation and remain valid when subgroups with high dropout are under-represented in the complete cases.
  • When propensity and missingness models converge fast, slow outcome learners can still be used without breaking oracle efficiency, because nuisance errors enter only as products.
  • In the GBSG2 example, correcting for selective missingness lowers the estimated hormonal-therapy benefit at high progesterone-receptor levels relative to available-case analyses, showing the direction of the bias the new learners remove.
  • The new learners extend the comparison set for practitioners choosing among metalearners, giving a reason to prefer them over available-case or imputed versions of the DR and EP learners in MAR settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mEP target step is verified, the mEP-learner's stability advantage (fewer extreme pseudo-outcomes) would make it the safer default in finite samples; until then the mDR-learner rests on a proven bound.
  • The paper's Appendix N sketch suggests the mDR construction generalizes to longitudinal settings by multiplying the correction term by cumulative censoring weights, so the same EIF logic may transfer to post-baseline missingness.
  • The half-sample bootstrap intervals, as applied here, had poor conditional coverage for some individuals; users should treat the interval estimates as less mature than the point-estimate theory.
  • A natural testable extension is to replace the fixed trigonometric sieve in mEP with a data-adaptive basis chosen by cross-validation; if the targeting step still zeros the drift term, oracle efficiency may hold for a wider class of CATEs.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper addresses estimation of the conditional average treatment effect (CATE) when outcome data are missing at random. It proposes two estimators: the mDR-learner, which incorporates inverse probability of censoring weights into the DR-learner pseudo-outcomes, and the mEP-learner, which modifies the EP-learner's targeting step with a censoring-weighted clever covariate. The authors claim oracle efficiency for both estimators under reasonable nuisance convergence rates. The mDR-learner's excess-risk bound is derived in detail in Appendix E following Foster and Syrgkanis (2023), and simulation studies and a GBSG2 data application are used to illustrate performance. The mDR-learner derivation appears internally consistent, but the mEP-learner's oracle-efficiency claim is asserted without proof, and the targeting weight used in Algorithm 2 does not set the EIF drift term to zero as stated.

Significance. The mDR-learner is a solid contribution: its pseudo-outcomes follow from the efficient influence function of the MSE risk under MAR, and the excess-risk bound in Appendix E gives a concrete second-order robustness guarantee that includes the missingness model. The paper also provides reproducible code and a real-data illustration, which are strengths. If the mEP-learner's targeting step is corrected and its oracle-efficiency property is rigorously established, the paper would offer two practical debiased learners for a common missing-data problem. As it stands, the mDR half of the paper is well supported, while the mEP half lacks the theoretical grounding necessary for the paper's central claim.

major comments (3)
  1. [Section 3.2 and Algorithm 2] The targeting weight H(A,C,Z)=CA/(G pi)+C(1-A)/(G(1-pi)) is not the correct clever covariate for the EIF in eq. (7). The EIF drift term is -2 theta(X)(A-pi)C/[pi(1-pi)G](Y-mu_A), whose treatment-arm coefficient is C/(G pi) and control-arm coefficient is -C/(G(1-pi)): the correct weight is CA/(G pi) - C(1-A)/(G(1-pi)). With the weight as defined, the weighted least squares regression in Algorithm 2 has score equation sum H_i phi_i (Y_i - mu_{A_i} - eps^T phi_i)=0. For any theta in the sieve, this gives sum theta_i H_i (Y_i - mu_{A_i} - eps^T phi_i)=0. The required drift-zero condition after the update mu_{A}* = mu_A + (2A-1)eps^T phi is sum theta_i (2A_i-1) H_i (Y_i - mu_{A_i,i}*)=0, which reduces, using the score equation, to a generally nonzero residual sum over the untreated complete cases (sum_{A_i=0} theta_i H_i (Y_i - mu_{0,i}) up to an overall sign). Therefore Algorithm 2 does not set the sample average of the second term in eq. (7) to zero. The same sign issue appears in the complete-data EP-learner in Appendix C. The manuscript's assertion in Section 3.2 that this weight 'ensures that the sample average of the second term in eq. (7) converges to 0' is unsupported.
  2. [Section 3.2] The oracle-efficiency claim for the mEP-learner is stated without proof. Unlike the mDR-learner, which has the detailed excess-risk analysis in Appendix E and Theorem 1, the mEP-learner is only asserted to be oracle efficient on the basis of the targeting step. Even if the sign of the clever covariate were corrected, there is no theorem showing that the sieve-based iTMLE update makes the drift term vanish in the required metric, nor is there an excess-risk bound analogous to the mDR case. This missing proof is load-bearing because the abstract and introduction claim oracle efficiency for both estimators.
  3. [Section 4 and Appendix K] The simulation evaluation emphasizes the root mean square median error (RMSME) because the mean RMSE is skewed by rare extreme estimates, but this metric obscures the instability that the paper itself identifies as a weakness of the mDR/mEP learners. Appendix K shows that the mean RMSE of mEP is often larger than that of mDR (e.g., DGP 3 at n=1600 and n=3200), which is not visible in the RMSME plots. The paper should present both metrics in the main text or clearly justify the choice of RMSME as the primary metric, and the stability claims in Section 4.2 should be qualified accordingly.
minor comments (3)
  1. [Algorithm 1, step 1] The phrase 'Split the data randomly into K (e.g., 10) equal sized folds of n observations' is imprecise; each fold contains approximately n/K observations, not n observations.
  2. [Appendix C, Algorithm 2] In the complete-data EP-learner, the outcome models are written as mu_0(Z)=E[Y|A=0, C=1, Z] and mu_1(Z)=E[Y|A=1, C=1, Z], but the variable C is not defined in the no-missing-data setting; this appears to be a copy-paste error from the mEP-learner algorithm.
  3. [Section 4.2] The statement that the mEP-learner and EP-learner are more stable than the mDR-learner should be tied more explicitly to the RMSME metric, since the mean RMSE results in Appendix K show a different ordering in some settings.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the mDR-learner is checked against the independent Foster-Syrgkanis framework, and the mEP-learner claim, while unproved, is an omission rather than a reduction to its inputs.

full rationale

The paper's central derivation chain is not circular. The mDR-learner's pseudo-outcomes (eq. 8) are constructed from the EIF of the MSE risk (eq. 7), and the oracle-efficiency claim is verified in Appendix E by verifying the Foster-Syrgkanis orthogonality, strong convexity, and smoothness assumptions in Appendix F; the bound is an independent theoretical check rather than a restatement of the estimator's definition. The mEP-learner is an extension of the EP-learner with IPCW weights; its oracle-efficiency statement in Section 3.2 is asserted ('This weight ensures that the sample average of the second term in eq. (7) converges to 0, and hence the mEP-learner will also be oracle efficient') without a theorem or proof, and a skeptic could question whether Algorithm 2's weighted regression actually zeroes the EIF drift term. However, this is an omitted proof or a potential correctness gap, not a circular reduction: the claim does not define the target in terms of the estimator, fit a parameter and rename it a prediction, or rely on a self-citation to force the conclusion. The self-citations that appear (e.g., Vansteelandt and Morzywolek 2023 for iTMLE as background) are not load-bearing for the main claims, and the simulation study compares against external benchmarks rather than being used to construct the estimators. Therefore no circular step is exhibited.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The estimators introduce no new entities or fitted constants. The central theoretical claim rests on standard causal identification assumptions, the Foster-Syrgkanis framework, and a rate condition on nuisance models. The mEP-specific iTMLE targeting assumption is not proved in the paper.

free parameters (1)
  • mEP sieve basis dimension and interaction order = unspecified (chosen via Zhang and Simon 2023 guidance or cross-validation)
    The mEP-learner requires the user to choose the sieve basis phi(X); the finite-sample performance depends on this choice, and the paper gives only general guidance.
assumptions (4)
  • domain assumption A1-A5: consistency, no unmeasured confounding, treatment positivity, outcome MAR, positivity of censoring
    Identifies the CATE from observed data. Used in eq. (5)-(8) to justify the IPCW-weighted pseudo-outcomes.
  • standard math Foster-Syrgkanis conditions: Neyman orthogonality, strong convexity, higher-order smoothness of the MSE risk
    Proven for the mDR-learner in Appendix F; they underpin the excess risk bound of Theorem 1.
  • domain assumption Nuisance estimators (propensity, missingness, outcome models) converge faster than n^{-1/4}
    Required for oracle efficiency; stated in Section 3.1. If slow, the second-order terms in Theorem 1 dominate.
  • ad hoc to paper For mEP-learner, the sieve-based iTMLE targeting step correctly approximates the CATE and sets the drift term to zero under MAR
    Assumed without proof in Section 3.2; this is the key unverified step for the mEP oracle-efficiency claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Causal machine learning for heterogeneous treatment effects in the presence of missing outcome data." pith.science (2026). https://pith.science/paper/CQEUOEFO

@misc{pith2026241219711,
  author       = {Pith},
  title        = {Pith review of: Causal machine learning for heterogeneous treatment effects in the presence of missing outcome data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CQEUOEFO}},
  note         = {Machine review of arXiv:2412.19711}
}
read the original abstract

When estimating heterogeneous treatment effects, missing outcome data can complicate treatment effect estimation, causing certain subgroups of the population to be poorly represented. In this work, we discuss this commonly overlooked problem and consider the impact that missing at random (MAR) outcome data has on causal machine learning estimators for the conditional average treatment effect (CATE). We propose two de-biased machine learning estimators for the CATE, the mDR-learner and mEP-learner, which address the issue of under-representation by integrating inverse probability of censoring weights into the DR-learner and EP-learner respectively. We show that under reasonable conditions, these estimators are oracle efficient, and illustrate their favorable performance through simulated data settings, comparing them to existing CATE estimators, including comparison to estimators which use common missing data techniques. We present an example of their application using the GBSG2 trial, exploring treatment effect heterogeneity when comparing hormonal therapies to non-hormonal therapies among breast cancer patients post surgery, and offer guidance on the decisions a practitioner must make when implementing these estimators.

Figures

Figures reproduced from arXiv: 2412.19711 by the authors.

Figure 1
Figure 1. mDR-learner algorithm As the mDR-learner defines its pseudo-outcomes using the EIF of the MSE for the CATE (under outcome miss￾ingness), it not only minimizes the MSE risk function, but also experiences less sensitivity to errors in its nuisance functions (including the IPCWs). We demonstrate this by exploring the excess risk of the mDR-learner, defined as the 5 [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 3
Figure 3. Root mean square median error (RMSME) for mDR-learner, mEP-learner, DR-learner, EP-learner and [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Median CATE estimates plotted by progesterone receptor (fmol/l). [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figures from the paper (6 more)
Figure 5
Figure 5. Figure 5: CATE estimates from single cross-fitting seeds plotted by progesterone receptor (fmol/l). [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Simulated data example plotting the logit of the outcome probabilities in each treatment arm across [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Mean of CATE estimates across 500 simulations, plotted by [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Mean RMSE for mDR-learner, mEP-learner, DR-learner, EP-learner and T-learner in three DGPs plotted [PITH_FULL_IMAGE:figures/full_fig_p035_8.png]
Figure 9
Figure 9. Figure 9: Median CATE estimates plotted by progesterone receptor (fmol/l). [PITH_FULL_IMAGE:figures/full_fig_p040_9.png]
Figure 10
Figure 10. Figure 10: CATE estimates from single cross-fitting seeds plotted by progesterone receptor (fmol/l). [PITH_FULL_IMAGE:figures/full_fig_p041_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 16 canonical work pages

  1. [1]

    Causal inference using potential outcomes: Design, modeling, decisions

    Donald B Rubin. Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100 0 (469): 0 322--331, 2005. ISSN 0162-1459

    work page 2005

  2. [2]

    Metalearners for estimating heterogeneous treatment effects using machine learning

    Sören R Künzel, Jasjeet S Sekhon, Peter J Bickel, and Bin Yu. Metalearners for estimating heterogeneous treatment effects using machine learning. Proceedings of the national academy of sciences, 116 0 (10): 0 4156--4165, 2019. ISSN 0027-8424

    work page 2019

  3. [3]

    Towards optimal doubly robust estimation of heterogeneous causal effects

    Edward H Kennedy. Towards optimal doubly robust estimation of heterogeneous causal effects. Electronic Journal of Statistics, 17 0 (2): 0 3008--3049, 2023. ISSN 1935-7524

    work page 2023

  4. [4]

    Quasi-oracle estimation of heterogeneous treatment effects

    Xinkun Nie and Stefan Wager. Quasi-oracle estimation of heterogeneous treatment effects. Biometrika, 108 0 (2): 0 299--319, 2021. ISSN 0006-3444

    work page 2021

  5. [5]

    Combining t-learning and dr-learning: a framework for oracle-efficient estimation of causal contrasts

    Lars van der Laan, Marco Carone, and Alex Luedtke. Combining t-learning and dr-learning: a framework for oracle-efficient estimation of causal contrasts. arXiv preprint arXiv:2402.01972, 2024

    arXiv 2024

  6. [6]

    Randomized trials with missing outcome data: how to analyze and what to report

    Rolf HH Groenwold, Karel GM Moons, and Jan P Vandenbroucke. Randomized trials with missing outcome data: how to analyze and what to report. Cmaj, 186 0 (15): 0 1153--1157, 2014. ISSN 0820-3946

    work page 2014

  7. [7]

    To impute or not to impute? missing data in treatment effect estimation

    Jeroen Berrevoets, Fergus Imrie, Trent Kyono, James Jordon, and Mihaela van der Schaar. To impute or not to impute? missing data in treatment effect estimation. In International Conference on Artificial Intelligence and Statistics, pages 3568--3590. PMLR, 2023. ISBN 2640-3498

    work page 2023

  8. [8]

    Estimation of regression coefficients when some regressors are not always observed

    James M Robins, Andrea Rotnitzky, and Lue Ping Zhao. Estimation of regression coefficients when some regressors are not always observed. Journal of the American statistical Association, 89 0 (427): 0 846--866, 1994. ISSN 0162-1459

    work page 1994

Show all 23 references
  1. [9]

    Stacked inverse probability of censoring weighted bagging: A case study in the infcarehiv register

    Pablo Gonzalez Ginestet, Ales Kotalik, David M Vock, Julian Wolfson, and Erin E Gabriel. Stacked inverse probability of censoring weighted bagging: A case study in the infcarehiv register. Journal of the Royal Statistical Society Series C: Applied Statistics, 70 0 (1): 0 51--6...

    2021

  2. [10]

    Causal inference in statistics: A primer

    Judea Pearl, Madelyn Glymour, and Nicholas P Jewell. Causal inference in statistics: A primer. John Wiley & Sons, 2016. ISBN 1119186862

    2016

  3. [11]

    On a general class of orthogonal learners for the estimation of heterogeneous treatment effects

    Pawel Morzywolek, Johan Decruyenaere, and Stijn Vansteelandt. On a general class of orthogonal learners for the estimation of heterogeneous treatment effects. arXiv preprint arXiv:2303.12687, 2023

    2023 arXiv

  4. [12]

    Demystifying statistical learning based on efficient influence functions

    Oliver Hines, Oliver Dukes, Karla Diaz-Ordaz, and Stijn Vansteelandt. Demystifying statistical learning based on efficient influence functions. The American Statistician, pages 1--13, 2022. ISSN 0003-1305

    2022

  5. [13]

    Orthogonal statistical learning

    Dylan J Foster and Vasilis Syrgkanis. Orthogonal statistical learning. The Annals of Statistics, 51 0 (3): 0 879--908, 2023. ISSN 0090-5364

    2023

  6. [14]

    Sequential double robustness in right-censored longitudinal models

    Alexander R Luedtke, Oleg Sofrygin, Mark J van der Laan, and Marco Carone. Sequential double robustness in right-censored longitudinal models. arXiv preprint arXiv:1705.02459, 2017

    2017 arXiv

  7. [15]

    Orthogonal prediction of counterfactual outcomes

    Stijn Vansteelandt and Paweł Morzywołek. Orthogonal prediction of counterfactual outcomes. arXiv preprint arXiv:2311.09423, 2023

    2023 arXiv

  8. [16]

    Super learner

    Mark J Van der Laan, Eric C Polley, and Alan E Hubbard. Super learner. Statistical applications in genetics and molecular biology, 6 0 (1), 2007. ISSN 1544-6115

    2007

  9. [17]

    Cross-fitting and averaging for machine learning estimation of heterogeneous treatment effects

    Daniel Jacob. Cross-fitting and averaging for machine learning estimation of heterogeneous treatment effects. arXiv preprint arXiv:2007.02852, 2020

    2007 arXiv

  10. [18]

    Regression in tensor product spaces by the method of sieves

    Tianyu Zhang and Noah Simon. Regression in tensor product spaces by the method of sieves. Electronic Journal of Statistics, 17 0 (2): 0 3660--3727, 2023. ISSN 1935-7524

    2023

  11. [19]

    Debiased inference for a covariate-adjusted regression function

    Kenta Takatsu and Ted Westling. Debiased inference for a covariate-adjusted regression function. arXiv preprint arXiv:2210.06448, 2022

    2022 arXiv

  12. [20]

    Flexibly estimating and interpreting heterogeneous treatment effects of laparoscopic surgery for cholecystitis patients

    Matteo Bonvini, Zhenghao Zeng, Miaoqing Yu, Edward H Kennedy, and Luke Keele. Flexibly estimating and interpreting heterogeneous treatment effects of laparoscopic surgery for cholecystitis patients. arXiv preprint arXiv:2311.04359, 2023

    2023 arXiv

  13. [21]

    Uniform inference for subsampled moment regression

    David M Ritzwoller and Vasilis Syrgkanis. Uniform inference for subsampled moment regression. arXiv preprint arXiv:2405.07860, 2024

    2024 arXiv

  14. [22]

    Estimation of subsidiary performance metrics under optimal policies

    Zhaoqi Li, Houssam Nassif, and Alex Luedtke. Estimation of subsidiary performance metrics under optimal policies. arXiv preprint arXiv:2401.04265, 2024

    2024 arXiv

  15. [23]

    Recovering target causal effects from post-exposure selection induced by missing outcome data

    Johan de Aguas, Johan Pensar, Tomás Varnet Pérez, and Guido Biele. Recovering target causal effects from post-exposure selection induced by missing outcome data. arXiv preprint arXiv:2401.16990, 2024

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.