REVIEW 4 major objections 3 minor 59 references
Generalized coarsened confounding for causal effects: a large-sample framework
T0 review · 4 major / 3 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read The paper claims any X-only confounder-partitioning method yields an asymptotically normal average causal effect estimator whenever stratum complexity grows slowly, and that the standard variance formula is valid for all such strata.
desk verdict The central asymptotic theorem is false: its variance omits outcome noise and the martingale argument fails, despite a useful reframing of coarsened confounding. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is a random-set estimating equation that rewrites the stratified estimator as the solution to $U(\tau)=0$, where $U(\tau)$ sums, over all observations and strata, the within-stratum treated-minus-control contrast weighted by stratum size. The asymptotic argument is carried by two tools: the shatter coefficient $\Delta_n(G(M))$ of the partition family, which counts the number of distinct subsets of $n$ points that the stratum cells can isolate and must grow subexponentially in $n$, and a martingale array formed by the normalized partial sums of the score contributions, a sequence of centered random sums whose conditional expectations at each step are zero, used to obtain the Gaussian limit of the leading term.
What would settle it
Under the paper's own definitions, for $n+1\le k\le 2n$ and a treated unit in stratum $j$, the conditional expectation of the increment $\xi_{n,k}$ given the filtration is $n^{-1/2}[(n_j/n_{1j}-1)\mu_1(X_k)+\mu_0(X_k)]$, which is nonzero whenever $n_j\ne n_{1j}$. A simulation with, say, two strata and imbalanced treatment counts inside one stratum can compute this quantity directly and check whether coverage of the proposed 95% intervals under the theorem's complexity condition matches nominal levels.
Extended reading notes
Core claim
The central result is Theorem 2: for an estimator $\hat{\tau}_S$ built from data-driven strata $S_1,\dots,S_J$, if the family of partitions has shatter coefficients $\Delta_n(G_n)$ satisfying $\lim_{n\to\infty} \log \Delta_n(G_n)/n = 0$, then $\sqrt{n}(\hat{\tau}_S-\tau)\to_d N(0,\sigma^2)$ with $\sigma^2 = E[(\mu_1(X)-\mu_0(X)-\tau)^2]$. The proof decomposes $\sqrt{n}(\hat{\tau}_S-\tau) = W_n + R_n$, where $W_n$ is handled by a martingale central limit theorem and the bias term $R_n$ is controlled by a uniform concentration inequality for partition estimates whose complexity is measured by $\Delta_n(G_n)$. The paper interprets this as a formal justification of the variance estimator $\hat{\sigma}^2_S = \sum_j (n_j/n)^2(\hat{\sigma}^2_{1j}/n_{1j} + \hat{\sigma}^2_{0j}/n_{0j})$ for coarsened exact matching, k-means strata, and random-forest strata alike.
Load-bearing premise
The main proof assumes the increments built from the stratum-randomized estimator have zero conditional expectation given the data already observed; this fails within any stratum whose treated and control counts are unequal.
Editorial extensions
If this is right
- Coarsened exact matching, k-means confounder clustering, and random-forest proximity clustering all produce asymptotically normal average causal effect estimators under the same subexponential-complexity condition.
- The variance estimator (3.9) is justified for the entire class, so users of these algorithms can report confidence intervals from within-stratum sample variances without additional modeling.
- For a fixed number of strata, the estimator carries a non-negligible bias equal to the error of approximating the outcome regression functions by piecewise constants, so consistency requires $J\to\infty$ with $n$.
- Subclassification on a known propensity score falls inside the framework because the propensity score is a function of $X$; subclassification on an estimated propensity score does not, as the strata would then depend on estimated parameters.
- The proposed bias correction fits a linear regression of estimates on $J^{-1}$ and extrapolates to $J^{-1}=0$, in analogy to SIMEX extrapolation.
Reading between the lines
- If the theorem is correct, the asymptotic variance does not depend on the number or shape of strata, only on the variability of the outcome regression contrast; this suggests the first-order efficiency of any such stratified estimator matches the efficiency of an estimator that knows $\mu_0,\mu_1$, a comparison the paper does not draw.
- The bias-correction step implicitly assumes bias decays linearly in $1/J$; a natural testable extension, not pursued in the paper, is to allow curvature or to choose $J$ to minimize an estimated bias-variance trade-off.
- The martingale condition can be checked numerically: for $k>n$, the conditional mean of the increment involves $(n_j/n_{1j}-1)\mu_1(X_k)$ for treated units, so a simulation with unbalanced strata would show whether the centering is correct or a bias term must be subtracted.
- The complexity condition suggests a practical diagnostic: estimate the shatter coefficient or the number of cells as a function of $n$ for the chosen clustering algorithm and verify the subexponential growth, something the paper leaves implicit.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces a class of 'generalized coarsened confounding' estimators that stratify confounders using coarsened exact matching, k-means clustering, or random-forest-based clustering, then estimate the average causal effect by weighted within-stratum outcome differences. The central theoretical contribution is Theorem 2 in Section 3.4, which claims asymptotic normality of the estimator under a combinatorial partition-complexity condition, with limiting variance E[(μ1(X)−μ0(X)−τ)^2], and which is presented as a formal justification of the variance estimator (3.2) of Iacus et al. (2011). The paper also proposes a SIMEX-style bias correction based on extrapolating estimates as a function of 1/J, and applies the methods to two observational datasets.
Significance. If Theorem 2 were correct, the paper would provide a valuable unified large-sample treatment of CEM and related data-adaptive stratification estimators, and it would fill a genuine gap in the literature. The paper's framing of coarsened matching as a quantization problem and its emphasis on partition complexity are attractive ideas, and the inclusion of reproducible R code and two real-data applications is a plus. However, the central asymptotic claim has a load-bearing flaw: the asserted martingale structure is not valid, and the stated limiting variance is contradicted by the simplest possible special case. Because the main theorem is the basis for the paper's claimed justification of the CEM variance estimator and for the subsequent methodology, the current claims are not supported.
major comments (4)
- [Section 3.4, Eq. (3.10) and the following paragraph] The claim that the partial sums of {ξ_{n,j}} form a martingale for each n is false. For k>n, conditioning on F_{n,k-1} fixes T_k and X_k but not Y_k, and the conditional mean of the increment for a treated unit in stratum j is n^{-1/2}(n_j/n_{1j}−1)μ1(X_k), with an analogous nonzero term for controls; n_j/n_{1j} and n_j/n_{0j} are not generally equal to one. Moreover, the displayed second line of (3.10) uses T_kY_k in both the treated and control brackets, whereas the control term should involve (1−T_k)Y_k. Consequently the martingale central limit theorem cannot be applied to W_n as claimed, and the proof of Theorem 2 does not establish (3.14).
- [Theorem 2, Section 3.4] The claimed limiting variance σ² = E[(μ1(X)−μ0(X)−τ)^2] is internally inconsistent with the estimator and with the variance estimator (3.2). Take J=1, which trivially satisfies (3.13). Then τ̂_S = Ȳ1−Ȳ0. If Y(1)=Y(0)=ε with ε∼N(0,1) and T independent Bernoulli(1/2), then μ1=μ0=0 and τ=0, so Theorem 2 predicts √n(τ̂_S−τ) ⇒ 0, while the actual limiting distribution is N(0,4). The estimator (3.2) also gives asymptotic variance 4 in this case, so the theorem contradicts the very formula it is meant to justify. The stated variance omits the outcome-noise contribution; this is not a minor regularity gap but a substantive error in the main theorem.
- [Section 3.6 and Section 4.1] The SIMEX-style bias correction is used for the headline applied results—for example, the k-means estimate of −0.041 with standard error 0.01 and p=0.003 in Section 4.1—but the paper explicitly states that formal justification of the extrapolation is beyond the current scope and under investigation. The method fits a line through estimates at a small set of J values and reads off the intercept at 1/J=0; no consistency or distributional result is supplied for this extrapolated estimator. The standard errors and p-values reported in Section 4 are therefore not backed by any theorem in the paper and should at minimum be labeled heuristic.
- [Remark 1, Section 3.4] Condition (3.13) is not verified for the random-forest procedure described in Section 3.3. The remark states a condition on the number of splits S_n for a single tree, but the proposed algorithm uses the random-forest proximity matrix followed by Ward's hierarchical clustering; the shatter coefficient of the resulting data-dependent partition family is not analyzed, and no argument shows that the proximity-based partitions inherit the single-tree condition. Thus the claim that Theorem 2 covers the random-forest estimator is unsupported even apart from the flaws in the theorem's proof.
minor comments (3)
- [Eq. (3.3)] The normalization of the τ term in the estimating function is unclear as printed; if the term is τ/(nJ), the equation does not solve to the estimator in (3.1), so the intended normalization should be stated explicitly.
- [Proof of Theorem 2] The proof is only a sketch: it does not state or verify the Lindeberg and conditional-variance conditions needed for the martingale central limit theorem, nor does it detail the argument that R_n→0 in probability from Theorem 1. A complete proof is essential given the centrality of the result.
- [Throughout] There are numerous typographical errors, including 'apporach', 'peforming', 'nonneglible', 'casual effect estimation', and 'consistency properity'; these should be corrected in a revision.
Circularity Check
No significant circularity: the asymptotic derivation is self-contained and its cited external results are independent.
full rationale
The paper's central derivation is not circular. Theorem 2 is obtained from the explicit decomposition (3.10), splitting sqrt(n)(tau_hat_S - tau) into W_n plus R_n; W_n is handled by a martingale central limit theorem applied to the constructed array, and R_n is controlled using the external concentration inequality of Lugosi and Nobel (1996) under condition (3.13). Neither ingredient is fitted from the data, nor is the target variance formula inserted as an assumption; it is computed from the potential-outcome means in W_n. The Iacus et al. (2011) variance estimator (3.2) is the object the paper seeks to justify, not an input to the theorem, so the claimed justification is an external bridge rather than a constructed equivalence. The SIMEX-style bias correction in Section 3.6 fits estimates against 1/J and extrapolates to J^{-1}=0, but the paper explicitly labels this as heuristic and states that formal justification is 'under investigation'; it is therefore not presented as a first-principles prediction. The only self-citations (Ghosh and Cruz Cortés 2019; Grover et al. 2024) appear in introductory motivation and are not load-bearing. The mathematical defect noted by the reader — that the increments xi_{n,k} have nonzero conditional means after stratum creation, and that the stated limiting variance omits outcome noise — is a correctness concern about the proof, not a circularity concern, since the derivation does not assume the conclusion it is trying to prove.
Assumptions & free parameters
free parameters (2)
- Bias-correction extrapolation intercept =
-0.041 (RHC k-means), 0.00387 (EFM k-means)
- Grid of stratum counts J (or K) =
RHC: kmeans K=2,5,7,10,20,50; RF K=5,7,10,20,30. EFM: kmeans K=2,5,10,20,45; RF K=5,7,10,20,40
assumptions (5)
- domain assumption Strong ignorability: T ⊥⊥ {Y(0), Y(1)} | X (equation 2.1)
- domain assumption Positivity: 0 < P(T=1|X) < 1
- domain assumption Consistency assumption
- ad hoc to paper Partition complexity condition (3.13): log Δ_n(G_n)/n → 0
- ad hoc to paper The increments ξ_{n,k} form a martingale difference array
Cite this review
Pith. "Pith review of Generalized coarsened confounding for causal effects: a large-sample framework." pith.science (2026). https://pith.science/paper/BVW65WLT
@misc{pith2026250103129,
author = {Pith},
title = {Pith review of: Generalized coarsened confounding for causal effects: a large-sample framework},
year = {2026},
howpublished = {\url{https://pith.science/paper/BVW65WLT}},
note = {Machine review of arXiv:2501.03129}
}
read the original abstract
There has been widespread use of causal inference methods for the rigorous analysis of observational studies and to identify policy evaluations. In this article, we consider a class of generalized coarsened procedures for confounding. At a high level, these procedures can be viewed as performing a clustering of confounding variables, followed by treatment effect and attendant variance estimation using the confounder strata. In addition, we propose two new algorithms for generalized coarsened confounding. While Iacus et al. (2011) developed some statistical properties for one special case in our class of procedures, we instead develop a general asymptotic framework. We provide asymptotic results for the average causal effect estimator as well as providing conditions for consistency. In addition, we provide an asymptotic justification for the variance formulae in Iacus et al. (2011). A bias correction technique is proposed, and we apply the proposed methodology to data from two well-known observational studies.
Reference graph
Works this paper leans on
-
[1]
Abadie, A. and G. W. Imbens (2006). Large sample properties of matching estimators for average treatment effects. Econometrica\/ 74\/ (1), 235--267
2006
-
[2]
Abadie, A. and G. W. Imbens (2011). Bias-corrected matching estimators for average treatment effects. Journal of Business and Economic Statistics\/ 29\/ (1), 1--11
work page 2011
-
[3]
Abadie, A. and G. W. Imbens (2012). A martingale representation for matching estimators. Journal of the American Statistical Association\/ 107\/ (498), 833--843
work page 2012
-
[4]
Abadie, A. and G. W. Imbens (2016). Matching on the estimated propensity score. Econometrica\/ 84\/ (2), 781--807
work page 2016
-
[5]
Bengio, Y., A. Courville, and P. Vincent (2013). Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence\/ 35\/ (8), 1798--1828
work page 2013
-
[6]
Berger, T. (2003). Rate-distortion theory. Wiley Encyclopedia of Telecommunications\/
work page 2003
-
[7]
Biau, G. (2012). Analysis of a random forests model. The Journal of Machine Learning Research\/ 13\/ (1), 1063--1095
work page 2012
-
[8]
Biau, G., L. Devroye, and G. Lugosi (2008). Consistency of random forests and other averaging classifiers. Journal of Machine Learning Research\/ 9\/ (9)
work page 2008
Show all 59 references
-
[9]
Biau, G. and E. Scornet (2016). A random forest guided tour. Test\/ 25 , 197--227
2016
-
[10]
Black, B. S., P. Lalkiya, and J. Y. Lerner (2020). The trouble with coarsened exact matching. Northwestern Law & Econ Research Paper Forthcoming\/
2020
-
[11]
Box, G. E., W. G. Hunter, and J. S. Hunter (1978). Statistics for experimenters , Volume 664. John Wiley and sons New York
1978
-
[12]
Breiman, L. (2001). Random forests. Machine learning\/ 45 , 5--32
2001
-
[13]
Carroll, R. J., H. K \"u chenhoff, F. Lombard, and L. A. Stefanski (1996). Asymptotics for the simex estimator in nonlinear measurement error models. Journal of the American Statistical Association\/ 91\/ (433), 242--250
1996
-
[14]
Chetverikov, M
Chernozhukov, V., D. Chetverikov, M. Demirer, E. Duflo, C. Hansen, W. Newey, and J. Robins (2018). Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal\/ 21\/ (1), C1--C68
2018
-
[15]
Kasahara, and P
Chernozhukov, V., H. Kasahara, and P. Schrimpf (2021). Causal impact of masks, policies, behavior on early covid-19 pandemic in the us. Journal of Econometrics\/ 220\/ (1), 23--62
2021
-
[16]
Cochran, W. G. (1968). The effectiveness of adjustment by subclassification in removing bias in observational studies. Biometrics\/ , 295--313
1968
-
[17]
Cochran, W. G. and D. B. Rubin (1973). Controlling bias in observational studies: A review. Sankhy \=a : The Indian Journal of Statistics, Series A\/ , 417--446
1973
-
[18]
Connors, A. F., T. Speroff, N. V. Dawson, C. Thomas, F. E. Harrell, D. Wagner, N. Desbiens, L. Goldman, A. W. Wu, R. M. Califf, et al. (1996). The effectiveness of right heart catheterization in the initial care of critically iii patients. JAMA\/ 276\/ (11), 889--897
1996
-
[19]
Gy \"o rfi, and G
Devroye, L., L. Gy \"o rfi, and G. Lugosi (2013). A probabilistic theory of pattern recognition , Volume 31. Springer Science & Business Media
2013
-
[20]
D’Amour, A., P. Ding, A. Feller, L. Lei, and J. Sekhon (2021). Overlap in observational studies with high-dimensional covariates. Journal of Econometrics\/ 221\/ (2), 644--654
2021
-
[21]
Fleming, T. R. and D. P. Harrington (2013). Counting processes and survival analysis , Volume 625. John Wiley & Sons
2013
-
[22]
Fraley, C. and A. E. Raftery (2002). Model-based clustering, discriminant analysis, and density estimation. Journal of the American statistical Association\/ 97\/ (458), 611--631
2002
-
[23]
Gersho, A. and R. M. Gray (2012). Vector quantization and signal compression , Volume 159. Springer Science & Business Media
2012
-
[24]
Ghosh, D. and E. Cruz Cort \'e s (2019). A gaussian process framework for overlap and causal effect estimation with high-dimensional covariates. Journal of Causal Inference\/ 7\/ (2), 20180024
2019
-
[25]
Graf, S. and H. Luschgy (2007). Foundations of quantization for probability distributions . Springer
2007
-
[26]
Gray, R. (1984). Vector quantization. IEEE Assp Magazine\/ 1\/ (2), 4--29
1984
-
[27]
Grover, E. N., A. G. Buchwald, D. Ghosh, and E. J. Carlton (2024). Does behavior mediate the effect of weather on sars-cov-2 transmission? evidence from cell-phone data. PLOS ONE\/ 19\/ (6), e0305323
2024
-
[28]
Ho, D. E., K. Imai, G. King, and E. A. Stuart (2007). Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis\/ 15\/ (3), 199--236
2007
-
[29]
Holland, P. W. (1986). Statistics and causal inference. Journal of the American statistical Association\/ 81\/ (396), 945--960
1986
-
[30]
Allen, S
Hsiang, S., D. Allen, S. Annan-Phan, K. Bell, I. Bolliger, T. Chong, H. Druckenmiller, L. Y. Huang, A. Hultgren, E. Krasovich, et al. (2020). The effect of large-scale anti-contagion policies on the covid-19 pandemic. Nature\/ 584\/ (7820), 262--267
2020
-
[31]
Hullsiek, K. H. and T. A. Louis (2002). Propensity score modeling strategies for the causal analysis of observational data. Biostatistics\/ 3\/ (2), 179--193
2002
-
[32]
King, and G
Iacus, S., G. King, and G. Porro (2009). Cem: Software for coarsened exact matching. Journal of Statistical Software\/ 30 , 1--27
2009
-
[33]
Iacus, S. M., G. King, and G. Porro (2011). Multivariate matching methods that are monotonic imbalance bounding. Journal of the American Statistical Association\/ 106\/ (493), 345--361
2011
-
[34]
Imbens, G. W. and D. B. Rubin (2015). Causal inference in statistics, social, and biomedical sciences . Cambridge University Press
2015
-
[35]
Kieffer, J. (1982). Exponential rate of convergence for lloyd's method i. IEEE Transactions on Information Theory\/ 28\/ (2), 205--210
1982
-
[36]
Chen, and W
Liu, Q., Z. Chen, and W. H. Wong (2024). An encoding generative modeling approach to dimension reduction and covariate adjustment in causal inference with observational studies. Proceedings of the National Academy of Sciences\/ 121\/ (23), e2322376121
2024
-
[37]
Lloyd, S. (1982). Least squares quantization in pcm. IEEE transactions on information theory\/ 28\/ (2), 129--137
1982
-
[38]
Lugosi, G. and A. Nobel (1996). Consistency of data-driven histogram methods for density estimation and classification. The Annals of Statistics\/ 24\/ (2), 687--706
1996
-
[39]
Macqueen, J. (1967). Some methods for classification and analysis of multivariate observations. In Proceedings of 5-th Berkeley Symposium on Mathematical Statistics and Probability/University of California Press
1967
-
[40]
Neutra, R. R., S. Greenland, and E. A. Friedman (1980). Effect of fetal monitoring on cesarean section rates. Obstetrics and Gynecology\/ 55\/ (2), 175--180
1980
-
[41]
Neyman, J. (1923). Sur les applications de la th \'e orie des probabilit \'e s aux experiences agricoles: Essai des principes. Roczniki Nauk Rolniczych\/ 10 , 1--51
1923
-
[42]
Pollard, D. (1981). Strong consistency of k-means clustering. The Annals of Statistics\/ , 135--140
1981
-
[43]
Rosenbaum, P. R. (1989). Optimal matching for observational studies. Journal of the American Statistical Association\/ 84\/ (408), 1024--1032
1989
-
[44]
Rosenbaum, P. R. and D. B. Rubin (1983). The central role of the propensity score in observational studies for causal effects. Biometrika\/ 70\/ (1), 41--55
1983
-
[45]
Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology\/ 66\/ (5), 688
1974
-
[46]
Rubin, D. B. (1976). Multivariate matching methods that are equal percent bias reducing, i: Some examples. Biometrics\/ , 109--120
1976
-
[47]
Rubin, D. B. and E. A. Stuart (2006). Affinely invariant matching methods with discriminant mixtures of proportional ellipsoidally symmetric distributions. The Annals of Statistics\/ 34\/ (4), 1814--1826
2006
-
[48]
Rubin, D. B. and N. Thomas (1992). Affinely invariant matching methods with ellipsoidal distributions. The Annals of Statistics\/ , 1079--1093
1992
-
[49]
Rubin, D. B. and N. Thomas (2000). Combining propensity score matching with additional adjustments for prognostic covariates. Journal of the American Statistical Association\/ 95\/ (450), 573--585
2000
-
[50]
Sabin, M. and R. Gray (1986). Global convergence and empirical consistency of the generalized lloyd algorithm. IEEE Transactions on information theory\/ 32\/ (2), 148--155
1986
-
[51]
Salimi, B. and D. Suciu (2016). Zaliql: A sql-based framework for drawing causal inference from big data. arXiv preprint arXiv:1609.03540\/
2016 arXiv
-
[52]
Shannon, C. E. (1948). A mathematical theory of communication. The Bell System Technical Journal\/ 27\/ (3), 379--423
1948
-
[53]
Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical science\/ 25\/ (1), 1
2010
-
[54]
Tsiatis, A. A. (2006). Semiparametric theory and missing data , Volume 4. Springer
2006
-
[55]
Van Der Vaart, A. W. and J. A. Wellner (1996). Weak convergence. In Weak convergence and empirical processes , pp.\ 16--28. Springer
1996
-
[56]
Morucci, M
Wang, T., M. Morucci, M. U. Awan, Y. Liu, S. Roy, C. Rudin, and A. Volfovsky (2017). Flame: A fast large-scale almost matching exactly approach to causal inference
2017
-
[57]
Ward, J. H. (1963). Hierarchical grouping to optimize an objective function. Journal of the American statistical association\/ 58\/ (301), 236--244
1963
-
[58]
Wolfowitz, J. (2012). Coding theorems of information theory , Volume 31. Springer Science & Business Media
2012
-
[59]
Zivich, P. N. and A. Breskin (2021). Machine learning for causal inference: on the use of cross-fit estimators. Epidemiology\/ 32\/ (3), 393--401
2021
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.