REVIEW 2 major objections 5 minor 1 cited by
Accuracy Limits of Causal Trees for Individualized Treatment Effects
T0 review · 2 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read This paper establishes lower bounds showing that standard recursive-partitioning causal tree estimators—difference-in-means, inverse-probability-weighted, or squared-error splitting, with or without honesty—cannot achieve polynomial-in-samp
desk verdict A serious, technically substantial negative result for DIM/IPW causal trees, but the main theorems overclaim for SSE splitting: the supplement only proves the SSE case under an extra equal-variance condition that the main text never states. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the first-split index i-hat, the number of observations in the left child at the root split. The proof's engine is an approximation of the split-criterion process (a partial-sum empirical process indexed by split location and coordinate) by the square of a time-transformed Ornstein-Uhlenbeck (stationary Gaussian) process, achieved through high-dimensional Gaussian coupling, Gaussian comparison inequalities, Gaussian correlation inequalities, and a corrected Darling-Erdos limit. This yields the imbalanced-split probability bound P(n^a <= i-hat <= n^b or n-n^b <= i-hat <= n-n^a) >= (b-a)/(2pe) for each coordinate, which is then applied recursively to deeper nodes. Th
What would settle it
Simulate the constant-effect randomized benchmark with p=2 independent uniform covariates and n=10^5, recording the root split fraction i-hat/n over many replications. Theorem 1 implies liminf of P(i-hat <= n^b or i-hat >= n-n^b) >= b/e for every b in (0,1); if these boundary-tail probabilities decay to zero as n grows, the paper's central lower bound is wrong.
Extended reading notes
Core claim
On the paper's own terms, the central claim is Theorem 1: in a randomized benchmark with constant treatment effect, independent continuous covariates, and sub-exponential errors, the DIM, IPW, and SSE causal tree estimators—with or without honesty—each have uniform error at least C n^{-b/2} sqrt(log log n) (NSS) or C n^{-b/2} (HON) with probability bounded away from zero, for every b in (0,1); no polynomial-in-n sup-norm rate is possible. The root cause is an imbalanced-split bound: at the first split, the probability that a child contains between n^a and n^b observations is at least (b-a)/(2pe). Greedy recursion propagates such small cells through the tree, and local averaging inside them p
Load-bearing premise
The proof's multivariate lower bound relies on Assumption 1(iv), that the covariate components are independent and continuously distributed: this independence is what lets the split-coordinate criteria be treated as asymptotically independent, producing the 1/p factor in the bound; with dependent covariates the quantitative bound is unproven, though the univariate end-cut mechanism may persist.
Editorial extensions
If this is right
- In the constant-effect benchmark, no DIM, IPW, or SSE causal tree, honest or not, can achieve uniform error o_p(n^{-b/2}) for all b>0; at least a b/e fraction of samples exhibit superslow local errors.
- Honesty only removes the sqrt(log log n) factor from the lower bound; the uniform rate remains slower than polynomial, so sample splitting is not a cure for this inaccuracy.
- X-adaptive trees—those sharing covariates across splits but refreshing outcomes—are uniformly inconsistent whenever depth K_n grows like log log n, meaning even very shallow trees can break worst-case accuracy at realistic sample sizes.
- Integrated squared error bounds remain near-parametric, so evaluations based on average error will not detect the local inaccuracy that Theorem 1 establishes.
- Balanced-cell assumptions such as alpha-regularity are incompatible with standard CART-type splitting in the constant-effect benchmark; existing polynomial-rate results for causal forests therefore do not apply to these implementations.
Reading between the lines
- Editorial inference: if the same end-cut mechanism operates in forests built from unregularized CART base learners, uniform confidence intervals for heterogeneous effects from those ensembles could inherit the slow convergence; the paper does not analyze forests, so this is an extrapolation.
- Editorial inference: a direct diagnostic for practitioners is the empirical distribution of the first-split fraction in a null-effect simulation; the theorem predicts probability mass accumulating at 0 and 1 with magnitude governed by b/e, and a flat histogram would contradict the mechanism.
- Editorial inference: the lower bound suggests that simple regularizations such as minimum leaf size or balance constraints can restore polynomial uniform rates, but only by introducing smoothing bias in high-curvature regions—a trade-off the paper notes but does not quantify.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies the uniform and integrated accuracy of causal tree estimators based on the Athey--Imbens recursive partitioning framework, covering DIM, IPW, and SSE splitting, with no sample splitting (NSS), honesty (HON), and X-adaptive versions. Under a constant-treatment-effect DGP with independent continuous covariates and sub-exponential errors, the authors prove lower bounds showing that these estimators cannot attain polynomial-in-n uniform convergence rates, with explicit non-vanishing probability bounds involving constants such as b/e. They also prove that X-adaptive trees of depth at least proportional to log log n are uniformly inconsistent, while integrated squared error can be small. The proofs, given in a long supplement, use empirical process approximations, high-dimensional CLTs, Gaussian comparison inequalities, and Ornstein-Uhlenbeck process calculations, and they correct an error in Eicker (1979). The paper includes a simulation study.
Significance. If the results hold as stated, this is an important negative result for a widely used methodology: it shows that canonical causal tree implementations need not satisfy the balanced-cell conditions behind existing consistency and inference guarantees, and that honesty alone does not cure the problem. The explicit lower-bound probabilities and the contrast between uniform and L2 behavior are valuable, as are the L2 upper bounds and the clarification of the role of α-regularity. The supplement is unusually detailed, with many named technical tools and explicit proof steps, and the correction to Eicker (1979) is a useful by-product. The main caveat is that the SSE component of the central theorems is only proved in the supplement under an equal-variance condition that is absent from the main theorem statements and is explicitly left as a conjecture for the general case.
major comments (2)
- [§SA-3.3, Theorem SA-29; main Theorems 1–2] The main text states Theorem 1 and Theorem 2 for l ∈ {DIM, IPW, SSE} under Assumption 1, which only requires E[ε_i(d)^2] > 0 for d = 0,1 and does not impose equality of variances. However, every SSE result in the supplement adds the condition V[ε_i(0)] = V[ε_i(1)]: Theorem SA-29, Corollaries SA-30/31/33/35, and the L2 bounds SA-32/34/36. Remark SA-2 explicitly states that this condition is needed to apply a two-dimensional Darling–Erdős theorem and leaves the unequal-variance case as a conjecture. Thus Theorems 1 and 2 assert SSE results in a regime for which no proof is supplied. This is load-bearing because SSE is one of the three splitting rules in the paper's central claim. The authors should either state the equal-variance condition in the main theorems for SSE (and adjust the abstract and discussion accordingly) or prove the missing case; DIM and IPW are not affected by this issue.
- [§SA-4.19–SA-4.20, Lemma SA-18] Lemma SA-18 is stated under the condition lim sup_n ρ_n log log n = 0, with s_n = exp((log n)^{ρ_n}). But the proof of Theorem SA-19 chooses ρ_n satisfying log log log log n / log log n ≪ ρ_n ≪ 1, which implies ρ_n log log n → ∞, not 0. The same proof relies on the resulting truncation bound to restrict the argmax to [s_n, n-s_n], so as written Lemma SA-18 does not apply. Lemma SA-28, the SSE analogue, has a different condition (lim sup ρ_n log log n = ∞), which is also not the 'ρ_n log log n → ∞' condition used in its proof. This is likely a typographical/condition error, but it must be corrected because Theorem SA-19 feeds directly into the proof of Theorem 1 for the DIM estimator.
minor comments (5)
- [§SA-4.18] The proof of Lemma SA-17 contains the sentence 'The assumption that lim inf_n ρ_n log log n = ∞ implies lim inf r_n = ∞', but Lemma SA-17 has no ρ_n and only assumes r_n → ∞. This looks like a leftover from Lemma SA-18 and should be removed or rewritten.
- [§SA-3.2.3, Theorem SA-25] Theorem SA-25 states the condition lim inf K_n / log log(K_n) > 0, while the main-text Theorem 3 uses lim inf K_n / log log n = κ > 0. The two statements should be aligned, or the difference should be explained.
- [§7] The text says 'The second is consistent with the X-results of Theorem 1', but X-adaptive results are presented in Theorem 3. Please correct the cross-reference.
- [References] Some references contain broken markup: 'Chernozhuokov' (should be 'Chernozhukov') in the main text and references, and 'Lata/suppress la' appears in place of 'Latała'. These should be fixed in the final version.
- [Assumption 1(iv)] The independence of covariate components in Assumption 1(iv) is used to obtain the factor 1/p in equation (6) and the asymptotic independence of split coordinates. It would be helpful to state explicitly that the quantitative lower bounds are not proven for dependent covariates, since the multivariate coupling argument in the supplement relies on this assumption.
Circularity Check
No significant circularity: lower bounds are proved on an explicit constant-effect DGP; the SSE variance-equality gap is a proof gap, not a circular step.
full rationale
The paper's central claims are lower bounds on the uniform estimation error of causal tree estimators. The derivation fixes an explicit constant-treatment-effect DGP satisfying Assumption 1, with τ(x)=c1-c0 and no fitted parameters; the lower bound conclusions are not obtained by renaming the inputs or by defining the target in terms of the estimators. The key split-imbalance probabilities (SA-6 and its causal analogues Corollary SA-9, Theorem SA-19, Theorem SA-29) are proved from strong approximations, high-dimensional CLTs, Darling-Erdős asymptotics, and the Gaussian correlation inequality, not assumed. The earlier Cattaneo-Klusowski-Tian (2022) manuscript is cited only to be superseded ('The present paper supersedes the unpublished manuscript...'), and its conjectures are explicitly proved here, so the self-citation is not load-bearing. No uniqueness theorem is imported from the authors' prior work, and no external result is adopted by ansatz from a self-citation. The one flagged weakness is an internal proof gap rather than circularity: the main-text Theorem 1 states SSE results under Assumption 1, but the supplement's SSE results all add 'with V[εi(0)] = V[εi(1)]' (e.g., Theorem SA-29, Corollaries SA-30/31/33/35), and Remark SA-2 says 'We conjecture that without V[εi(0)] = V[εi(1)], the conclusion still holds.' This is missing support for the SSE leg of Theorem 1 as stated, but the extra variance-equality condition is not the target conclusion, no estimator is fitted from data to produce the claim, and the DIM/IPW arguments are unaffected. Accordingly, the circularity score is low.
Assumptions & free parameters
assumptions (5)
- domain assumption Assumption 1(i)-(ii),(v): randomized treatment, additive potential outcomes, errors with mean zero independent of covariates and sub-exponential tails.
- domain assumption Assumption 1(iii): constant potential outcome means mu_d(x)=c_d, so tau(x) is constant.
- ad hoc to paper Assumption 1(iv): covariate components are mutually independent and continuously distributed.
- domain assumption Greedy CART splitting with variance-maximization or SSE criteria and at least one split (Definitions 2-3, Athey-Imbens).
- standard math Classical probability tools: high-dimensional CLTs (Chernozhukov et al. 2017), Darling-Erdos theorem for O-U processes, Gaussian correlation inequality, and Csorgo-Horvath approximations.
Cite this review
Pith. "Pith review of Accuracy Limits of Causal Trees for Individualized Treatment Effects." pith.science (2026). https://pith.science/paper/YWTRZQJD
@misc{pith2026250911381,
author = {Pith},
title = {Pith review of: Accuracy Limits of Causal Trees for Individualized Treatment Effects},
year = {2026},
howpublished = {\url{https://pith.science/paper/YWTRZQJD}},
note = {Machine review of arXiv:2509.11381}
}
read the original abstract
Recursive decision trees are widely used to estimate heterogeneous causal treatment effects in experimental and observational studies. These methods are typically implemented using CART-type recursive partitioning, with splitting criteria designed to identify variation in treatment effects across covariate-defined subgroups. We study causal tree estimators based on adaptive recursive partitioning and establish lower bounds on their estimation accuracy. The class we analyze includes versions with and without sample splitting, based on common treatment effect and squared-error splitting criteria. Even in a constant-effect benchmark with randomized treatment assignment, causal trees constructed via standard CART-type splitting rules can have uniform-norm errors that decrease more slowly than any power of the sample size. The underlying mechanism is that greedy recursive partitioning selects highly imbalanced splits with nonvanishing probability, producing terminal nodes containing very few observations and leading to large estimation variance. We further show that sample splitting, often called ``honesty,'' does not remove this limitation. As a consequence, causal tree estimators may converge arbitrarily slowly uniformly over the covariate space. At the same time, these estimators can have small integrated mean squared error, showing that average accuracy can mask local inaccuracy. Our results also clarify the role of balanced partition assumptions in existing theoretical guarantees for causal forests and related ensemble methods.
Figures
Forward citations
Cited by 1 Pith paper
-
Honesty in Causal Forests: When It Helps and When It Hurts
Honest estimation in causal forests can reduce CATE accuracy and require up to 27% more data, especially when effect heterogeneity is strong and detectable.
Reference graph
Works this paper leans on
-
[1]
Recursive partitioning for heterogeneous causal effects
Susan Athey and Guido Imbens. Recursive partitioning for heterogeneous causal effects. Proceedings of the National Academy of Sciences, 113 0 (27): 0 7353--7360, 2016
2016
-
[2]
McKeague
Moulinath Banerjee and Ian W. McKeague. Confidence sets for split points in decision trees. Annals of Statistics, 35 0 (2): 0 543 -- 574, 2007
2007
-
[3]
Provable boolean interaction recovery from tree ensemble obtained via random forests
Merle Behr, Yu Wang, Xiao Li, and Bin Yu. Provable boolean interaction recovery from tree ensemble obtained via random forests. Proceedings of the National Academy of Sciences, 119 0 (22): 0 e2118636119, 2022
2022
-
[4]
Statistical learning from a regression perspective
Richard A Berk. Statistical learning from a regression perspective. Springer Series in Statistics. Springer Nature, 2020
2020
-
[5]
Classification and Regression Trees
Leo Breiman, Jerome Friedman, RA Olshen, and Charles J Stone. Classification and Regression Trees. Chapman and Hall/CRC, 1984
1984
-
[6]
Analyzing bagging
Peter B \"u hlmann and Bin Yu. Analyzing bagging. Annals of Statistics, 30 0 (4): 0 927 -- 961, 2002
2002
-
[7]
Cattaneo, Max H
Matias D. Cattaneo, Max H. Farrell, and Yingjie Feng. Large sample properties of partitioning-based series estimators. Annals of Statistics, 48 0 (3): 0 1718--1741, 2020
2020
-
[8]
Matias D Cattaneo, Jason M Klusowski, and Peter M Tian. On the pointwise behavior of recursive partitioning and its implications for heterogeneous causal effect estimation. Technical report, arXiv preprint arXiv:2211.10805, 2022
arXiv 2022
Show all 37 references
-
[9]
Cattaneo, Rajita Chandak, and Jason M
Matias D. Cattaneo, Rajita Chandak, and Jason M. Klusowski. Convergence rates of oblique regression trees for flexible function libraries. Annals of Statistics, 52 0 (2): 0 466 -- 490, 2024
2024
-
[10]
Uniform estimation and inference for nonparametric partitioning-based m-estimators
Matias D Cattaneo, Yingjie Feng, and Boris Shigida. Uniform estimation and inference for nonparametric partitioning-based m-estimators. arXiv preprint arXiv:2409.05715, 2025
2025 arXiv
-
[11]
Central limit theorems and bootstrap in high dimensions
Victor Chernozhukov, Denis Chetverikov, and Kengo Kato. Central limit theorems and bootstrap in high dimensions. Annals of Probability, 45 0 (4): 0 2309 -- 2352, 2017
2017
-
[12]
Improved central limit theorem and bootstrap approximations in high dimensions
Victor Chernozhuokov, Denis Chetverikov, Kengo Kato, and Yuta Koike. Improved central limit theorem and bootstrap approximations in high dimensions. Annals of Statistics, 50 0 (5): 0 2562--2586, 2022
2022
-
[13]
Asymptotic properties of high-dimensional random forests
Chien-Ming Chi, Patrick Vossler, Yingying Fan, and Jinchi Lv. Asymptotic properties of high-dimensional random forests. The Annals of Statistics, 50 0 (6): 0 3415--3438, December 2022
2022
-
[14]
Chipman, Edward I
Hugh A. Chipman, Edward I. George, and Robert E. McCulloch. BART: Bayesian additive regression trees . Annals of Applied Statistics, 4 0 (1): 0 266 -- 298, 2010
2010
-
[15]
Cs \"o rg \"o and L
M. Cs \"o rg \"o and L. Horv \'a th. Limit Theorems in Change-Point Analysis. Wiley, 1997
1997
-
[16]
Cs \"o rg \"o and P
M. Cs \"o rg \"o and P. R \'e v \'e sz. Strong Approximations in Probability and Statistics. Probability and Mathematical Statistics : a series of monographs and textbooks. Academic Press, 1981
1981
-
[17]
A Probabilistic Theory of Pattern Recognition, volume 31
Luc Devroye, L \'a szl \'o Gy \"o rfi, and G \'a bor Lugosi. A Probabilistic Theory of Pattern Recognition, volume 31. Springer Science & Business Media, 2013
2013
-
[18]
F. Eicker. The asymptotic distribution of the suprema of the standardized empirical processes. Annals of Statistics, 7 0 (1): 0 116 -- 138, 1979
1979
-
[19]
Transductive rademacher complexity and its applications
Ran El-Yaniv and Dmitry Pechyony. Transductive rademacher complexity and its applications. Journal of Artificial Intelligence Research, 35: 0 193--234, 2009
2009
-
[20]
A survey and some generalizations of bessel processes
Anja G \"o ing-Jaeschke and Marc Yor. A survey and some generalizations of bessel processes. Bernoulli, 9 0 (2): 0 313 -- 349, 2003
2003
-
[21]
A Distribution-Free Theory of Nonparametric Regression
L \'a szl \'o Gy \"o rfi, Michael Kohler, Adam Krzy \.z ak, and Harro Walk. A Distribution-Free Theory of Nonparametric Regression. Springer-Verlag, 2002
2002
-
[22]
Hernán and James M
Miguel A. Hernán and James M. Robins. Causal Inference: What If. Boca Raton: Chapman & Hall/CRC, 2020
2020
-
[23]
The maximum likelihood method for testing changes in the parameters of normal observations
Lajos Horv \'a th. The maximum likelihood method for testing changes in the parameters of normal observations. Annals of statistics, 21 0 (2): 0 671--680, 1993
1993
-
[24]
The effect of splitting on random forests
Hemant Ishwaran. The effect of splitting on random forests. Machine Learning, 99 0 (1): 0 75--118, 2015
2015
-
[25]
Large scale prediction with decision trees
Jason M Klusowski and Peter M Tian. Large scale prediction with decision trees. Journal of the American Statistical Association, 119 0 (545): 0 525--537, 2024
2024
-
[26]
Royen's Proof of the Gaussian Correlation Inequality, pages 265--275
Rafa Lata a and Dariusz Matlak. Royen's Proof of the Gaussian Correlation Inequality, pages 265--275. Springer International Publishing, 2017
2017
-
[27]
On the convergence of CART under sufficient impurity decrease condition
Rahul Mazumder and Haoyue Wang. On the convergence of CART under sufficient impurity decrease condition. Advances in Neural Information Processing Systems, 36, 2024
2024
-
[28]
On the maximal perimeter of a convex set in R ^n with respect to a Gaussian measure
Fedor Nazarov. On the maximal perimeter of a convex set in R ^n with respect to a Gaussian measure. In Geometric Aspects of Functional Analysis: Israel Seminar, 2001--2002, pages 169--187. Springer, 2003
2001
-
[29]
Valentin V. Petrov. On lower bounds for tail probabilities. Journal of Statistical Planning and Inference, 137 0 (8): 0 2703--2705, 2007
2007
-
[30]
Consistency of random forests
Erwan Scornet, Gérard Biau, and Jean-Philippe Vert. Consistency of random forests . Annals of Statistics, 43 0 (4): 0 1716 -- 1741, 2015
2015
-
[31]
Inequalities for max| sk|/bk where k nr
Galen R Shorack and RT Smythe. Inequalities for max| sk|/bk where k nr. Proceedings of the American Mathematical Society, pages 331--336, 1976
1976
-
[32]
Bernstein-type bounds for beta distribution
Maciej Skorski. Bernstein-type bounds for beta distribution. Modern Stochastics: Theory and Applications, 10 0 (2): 0 211--228, 2023
2023
-
[33]
A cautionary tale on fitting decision trees to data from additive models: generalization lower bounds
Yan Shuo Tan, Abhineet Agarwal, and Bin Yu. A cautionary tale on fitting decision trees to data from additive models: generalization lower bounds. In International Conference on Artificial Intelligence and Statistics, pages 9663--9685. PMLR, 2022
2022
-
[34]
Statistical-computational trade-offs for recursive adaptive partitioning estimators
Yan Shuo Tan, Jason M Klusowski, and Krishnakumar Balasubramanian. Statistical-computational trade-offs for recursive adaptive partitioning estimators. arXiv preprint arXiv:2411.04394, 2024 a
2024 arXiv
-
[35]
The computational curse of big data for bayesian additive regression trees: A hitting time analysis
Yan Shuo Tan, Omer Ronen, Theo Saarinen, and Bin Yu. The computational curse of big data for bayesian additive regression trees: A hitting time analysis. arXiv preprint arXiv:2406.19958, 2024 b
2024
-
[36]
Estimation and inference of heterogeneous treatment effects using random forests
Stefan Wager and Susan Athey. Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 113 0 (523): 0 1228--1242, 2018
2018
-
[37]
Recursive Partitioning and Applications
Heping Zhang and Burton H Singer. Recursive Partitioning and Applications. Springer, 2010
2010
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.