{"id":"7f21f202-1c22-4adb-9e7f-96932c5b4a7f","arxiv_id":"2501.03129","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A large-sample theory for strata-based causal effect estimators is proposed, but the central theorem's proof is flawed and the claimed limiting variance appears to miss the outcome noise within strata.","lead":"This statistics paper proposes a general framework for estimating causal effects by clustering people into similar strata and comparing treated versus untreated individuals inside each stratum, covering coarsened exact matching, k-means, and random forest groupings. Its main theorem, claiming a normal approximation and valid variance formula, appears to contain a flawed martingale step and a limiting variance that omits within-stratum outcome noise.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 2's claimed limiting variance omits outcome noise and is contradicted by the single-stratum difference-in-means case, so the central asymptotic claim is internally inconsistent.","rationale":"The paper's central contribution is Theorem 2, and the paper explicitly claims that this theorem provides a formal justification of the Iacus et al. (2011) variance estimator. That theorem is the load-bearing result: the k-means and random forest proposals and the data analyses all depend on it, while Section 3.6 itself concedes that the SIMEX-style bias correction lacks formal justification. The J = 1 counterexample is decisive because it uses only the paper's own definitions in Sections 3.1 and 3.4 and requires no extra assumptions: the claimed variance is zero while the actual estimator has positive sampling variance. This is not a disagreement with external consensus; it is an internal inconsistency. The reader's weakest assumption correctly identifies the broken martingale assertion after (3.10), but the variance contradiction is an even more basic failure. The appendix code and empirical examples cannot repair an invalid theorem, and the empirical Wald statistics inherit the unsupported normality claim. No formal verification is supplied. As submitted, the central asymptotic framework is not established, so rejection is appropriate.","tokens_in":19108,"tokens_out":13672,"duration_ms":127748,"concrete_test":"Settle the concern by checking the J = 1 special case of Theorem 2: set one stratum, Y(1) = Y(0) = ε ~ N(0,1), and T ~ Bernoulli(1/2) independent of ε, so μ1 = μ0 = 0 and τ = 0. The theorem's formula gives asymptotic variance E[(μ1 − μ0 − τ)²] = 0, whereas the estimator Ȳ1 − Ȳ0 has √n asymptotic variance 1/P(T=1) + 1/P(T=0) = 4. Recompute the proof in this case; the claimed martingale construction fails because the conditional mean of the second-block increment is nonzero unless the stratum contains only one treatment group.","verdict_should_be":"REJECT","load_bearing_attack":"The most load-bearing defect is the claimed limiting variance in Theorem 2. Even if the martingale proof were repaired, σ² = E[(μ1(X)−μ0(X)−τ)²] cannot be the asymptotic variance of τ̂_S because it contains no outcome noise. Take J = 1, a single stratum, for which condition (3.13) holds trivially. Then τ̂_S = Ȳ1 − Ȳ0. If Y(1) = Y(0) = ε with ε ~ N(0,1) and T is independent Bernoulli(1/2), then μ1 = μ0 = 0 and τ = 0, so Theorem 2 predicts that √n(τ̂_S − τ) converges to a point mass at 0. The actual limiting distribution is N(0,4). Thus the stated variance formula is false, and the theorem cannot justify the Iacus et al. variance estimator (3.2), whose single-stratum limit is 4. The proof difficulty is visible just after (3.10): the increments ξ_{n,k} for k > n have conditional mean involving (n_j/n_{1j} − 1)μ1(X_k) and (n_j/n_{0j} − 1)μ0(X_k), which are nonzero whenever a stratum contains both treatment groups. This is an internal inconsistency in the central theorem, not a missing regularity condition.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a class of 'generalized coarsened confounding' estimators that stratify confounders using coarsened exact matching, k-means clustering, or random-forest-based clustering, then estimate the average causal effect by weighted within-stratum outcome differences. The central theoretical contribution is Theorem 2 in Section 3.4, which claims asymptotic normality of the estimator under a combinatorial partition-complexity condition, with limiting variance E[(μ1(X)−μ0(X)−τ)^2], and which is presented as a formal justification of the variance estimator (3.2) of Iacus et al. (2011). The paper also proposes a SIMEX-style bias correction based on extrapolating estimates as a function of 1/J, and applies the methods to two observational datasets.","tokens_in":19351,"tokens_out":10230,"duration_ms":95789,"significance":"If Theorem 2 were correct, the paper would provide a valuable unified large-sample treatment of CEM and related data-adaptive stratification estimators, and it would fill a genuine gap in the literature. The paper's framing of coarsened matching as a quantization problem and its emphasis on partition complexity are attractive ideas, and the inclusion of reproducible R code and two real-data applications is a plus. However, the central asymptotic claim has a load-bearing flaw: the asserted martingale structure is not valid, and the stated limiting variance is contradicted by the simplest possible special case. Because the main theorem is the basis for the paper's claimed justification of the CEM variance estimator and for the subsequent methodology, the current claims are not supported.","major_comments":[{"comment":"The claim that the partial sums of {ξ_{n,j}} form a martingale for each n is false. For k>n, conditioning on F_{n,k-1} fixes T_k and X_k but not Y_k, and the conditional mean of the increment for a treated unit in stratum j is n^{-1/2}(n_j/n_{1j}−1)μ1(X_k), with an analogous nonzero term for controls; n_j/n_{1j} and n_j/n_{0j} are not generally equal to one. Moreover, the displayed second line of (3.10) uses T_kY_k in both the treated and control brackets, whereas the control term should involve (1−T_k)Y_k. Consequently the martingale central limit theorem cannot be applied to W_n as claimed, and the proof of Theorem 2 does not establish (3.14).","section":"Section 3.4, Eq. (3.10) and the following paragraph"},{"comment":"The claimed limiting variance σ² = E[(μ1(X)−μ0(X)−τ)^2] is internally inconsistent with the estimator and with the variance estimator (3.2). Take J=1, which trivially satisfies (3.13). Then τ̂_S = Ȳ1−Ȳ0. If Y(1)=Y(0)=ε with ε∼N(0,1) and T independent Bernoulli(1/2), then μ1=μ0=0 and τ=0, so Theorem 2 predicts √n(τ̂_S−τ) ⇒ 0, while the actual limiting distribution is N(0,4). The estimator (3.2) also gives asymptotic variance 4 in this case, so the theorem contradicts the very formula it is meant to justify. The stated variance omits the outcome-noise contribution; this is not a minor regularity gap but a substantive error in the main theorem.","section":"Theorem 2, Section 3.4"},{"comment":"The SIMEX-style bias correction is used for the headline applied results—for example, the k-means estimate of −0.041 with standard error 0.01 and p=0.003 in Section 4.1—but the paper explicitly states that formal justification of the extrapolation is beyond the current scope and under investigation. The method fits a line through estimates at a small set of J values and reads off the intercept at 1/J=0; no consistency or distributional result is supplied for this extrapolated estimator. The standard errors and p-values reported in Section 4 are therefore not backed by any theorem in the paper and should at minimum be labeled heuristic.","section":"Section 3.6 and Section 4.1"},{"comment":"Condition (3.13) is not verified for the random-forest procedure described in Section 3.3. The remark states a condition on the number of splits S_n for a single tree, but the proposed algorithm uses the random-forest proximity matrix followed by Ward's hierarchical clustering; the shatter coefficient of the resulting data-dependent partition family is not analyzed, and no argument shows that the proximity-based partitions inherit the single-tree condition. Thus the claim that Theorem 2 covers the random-forest estimator is unsupported even apart from the flaws in the theorem's proof.","section":"Remark 1, Section 3.4"}],"minor_comments":[{"comment":"The normalization of the τ term in the estimating function is unclear as printed; if the term is τ/(nJ), the equation does not solve to the estimator in (3.1), so the intended normalization should be stated explicitly.","section":"Eq. (3.3)"},{"comment":"The proof is only a sketch: it does not state or verify the Lindeberg and conditional-variance conditions needed for the martingale central limit theorem, nor does it detail the argument that R_n→0 in probability from Theorem 1. A complete proof is essential given the centrality of the result.","section":"Proof of Theorem 2"},{"comment":"There are numerous typographical errors, including 'apporach', 'peforming', 'nonneglible', 'casual effect estimation', and 'consistency properity'; these should be corrected in a revision.","section":"Throughout"}],"recommendation":"reject","confidential_remarks":"The central theorem is not merely missing a regularity condition; it is false in a simple special case, and the martingale argument on which it relies is invalid. Repairing the result would require redeveloping the asymptotic theory, including the outcome-noise and stratum-weight terms, which is well beyond a routine revision. The paper contains some useful ideas and reproducible code, but the main theoretical claim and the applied conclusions that depend on it are not supportable as they stand."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You asked about Ghosh and Wang's generalized coarsened confounding paper. The honest bottom line: the packaging is attractive but the central theorem is not correct, and the error is not a missing regularity condition but an internal inconsistency in the variance formula.\n\nWhat's actually new: they reframe CEM as a data-adaptive stratification procedure, define a general class using arbitrary clustering of covariates (k-means, random forests), and propose a SIMEX-style bias correction that extrapolates to J tending to infinity. That framing and the two algorithms are reasonable contributions, and the connection to Lugosi-Nobel partition complexity is a sensible tool. If the theory were right, the paper would be a useful unified treatment of stratum-based confounder adjustment.\n\nThe soft spots are load-bearing. Theorem 2 claims sqrt(n)(tau_hat_S - tau) converges to N(0, sigma^2) with sigma^2 = E[(mu1(X)-mu0(X)-tau)^2]. That variance has no outcome noise. The single-stratum case J=1 satisfies their complexity condition (3.13) trivially, and the estimator is just the difference in means. If Y(1)=Y(0)=epsilon ~ N(0,1) and T is Bernoulli(1/2), the theorem predicts a point mass at zero, but the actual limit is N(0,4). So the theorem contradicts its own estimator. The proof break is just after (3.10): the increments xi_{n,k} for k>n have conditional mean involving (n_j/n_1j - 1)mu1(X_k) and (n_j/n_0j - 1)mu0(X_k), which are not zero. The claimed martingale property fails. And the bias term R_n is asserted to vanish using Theorem 1, but condition (3.13) does not require J to grow, so the approximation error of the conditional means by stratum constants need not go to zero.\n\nThe paper acknowledges the bias-correction is heuristic and lacks asymptotic justification, which is honest. But the empirical section has no simulations, only two data examples with a handful of K values and a linear extrapolation; that cannot validate the normal-based inference.\n\nWho should read it: anyone working on stratification or matching methods might find the framing of CEM as quantized confounder adjustment worth thinking about, and the partition-complexity condition is a plausible route to a correct theory. But as submitted, the central theorem is demonstrably false. A revision would need to replace the martingale argument with a proper decomposition (e.g., Hajek projection or M-estimation with estimated weights), add the within-stratum variance to sigma^2, and prove R_n vanishes under conditions that actually force J to grow.\n\nMy recommendation: send it to referees only if you are willing to have them document the fatal flaw and send back for major overhaul; otherwise desk reject. It is not publishable in anything like its current form.","headline":"The central asymptotic theorem is false: its variance omits outcome noise and the martingale argument fails, despite a useful reframing of coarsened confounding.","tokens_in":19909,"tokens_out":5036,"would_cite":false,"duration_ms":45150,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G20","62D20"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims any X-only confounder-partitioning method yields an asymptotically normal average causal effect estimator whenever stratum complexity grows slowly, and that the standard variance formula is valid for all such strata.","keywords":["average treatment effect","coarsened exact matching","k-means clustering","random forests","confounder stratification","asymptotic normality","quantization","causal inference"],"falsifier":"Under the paper's own definitions, for $n+1\\le k\\le 2n$ and a treated unit in stratum $j$, the conditional expectation of the increment $\\xi_{n,k}$ given the filtration is $n^{-1/2}[(n_j/n_{1j}-1)\\mu_1(X_k)+\\mu_0(X_k)]$, which is nonzero whenever $n_j\\ne n_{1j}$. A simulation with, say, two strata and imbalanced treatment counts inside one stratum can compute this quantity directly and check whether coverage of the proposed 95% intervals under the theorem's complexity condition matches nominal levels.","tokens_in":18815,"feed_emoji":"📊","tokens_out":9974,"duration_ms":82447,"temperature":0.7,"pith_summary":"This paper tries to establish that a very broad family of confounding-adjustment methods—anything that clusters or partitions the confounders using only the covariates, not the treatment or the outcome—produces a valid large-sample estimate of the average causal effect. The family includes coarsened exact matching, k-means clustering of confounders, and random-forest proximity clustering. The key claim is that all of these estimators are consistent and, after scaling by the square root of the sample size, converge to a Gaussian distribution with the same variance formula, as long as the family of partitions does not grow too complex relative to the sample size. The paper also offers a bias-correction step for the finite-sample bias that stratification inevitably introduces, and it applies the methods to two observational datasets.","feed_headline":"Any X-only clustering gives valid causal effect estimates","feed_subtitle":"One condition on stratum complexity makes coarsened matching, k-means, and random-forest strata all valid.","key_machinery":"The central object is a random-set estimating equation that rewrites the stratified estimator as the solution to $U(\\tau)=0$, where $U(\\tau)$ sums, over all observations and strata, the within-stratum treated-minus-control contrast weighted by stratum size. The asymptotic argument is carried by two tools: the shatter coefficient $\\Delta_n(G(M))$ of the partition family, which counts the number of distinct subsets of $n$ points that the stratum cells can isolate and must grow subexponentially in $n$, and a martingale array formed by the normalized partial sums of the score contributions, a sequence of centered random sums whose conditional expectations at each step are zero, used to obtain the Gaussian limit of the leading term.","core_discovery":"The central result is Theorem 2: for an estimator $\\hat{\\tau}_S$ built from data-driven strata $S_1,\\dots,S_J$, if the family of partitions has shatter coefficients $\\Delta_n(G_n)$ satisfying $\\lim_{n\\to\\infty} \\log \\Delta_n(G_n)/n = 0$, then $\\sqrt{n}(\\hat{\\tau}_S-\\tau)\\to_d N(0,\\sigma^2)$ with $\\sigma^2 = E[(\\mu_1(X)-\\mu_0(X)-\\tau)^2]$. The proof decomposes $\\sqrt{n}(\\hat{\\tau}_S-\\tau) = W_n + R_n$, where $W_n$ is handled by a martingale central limit theorem and the bias term $R_n$ is controlled by a uniform concentration inequality for partition estimates whose complexity is measured by $\\Delta_n(G_n)$. The paper interprets this as a formal justification of the variance estimator $\\hat{\\sigma}^2_S = \\sum_j (n_j/n)^2(\\hat{\\sigma}^2_{1j}/n_{1j} + \\hat{\\sigma}^2_{0j}/n_{0j})$ for coarsened exact matching, k-means strata, and random-forest strata alike.","pith_inferences":["If the theorem is correct, the asymptotic variance does not depend on the number or shape of strata, only on the variability of the outcome regression contrast; this suggests the first-order efficiency of any such stratified estimator matches the efficiency of an estimator that knows $\\mu_0,\\mu_1$, a comparison the paper does not draw.","The bias-correction step implicitly assumes bias decays linearly in $1/J$; a natural testable extension, not pursued in the paper, is to allow curvature or to choose $J$ to minimize an estimated bias-variance trade-off.","The martingale condition can be checked numerically: for $k>n$, the conditional mean of the increment involves $(n_j/n_{1j}-1)\\mu_1(X_k)$ for treated units, so a simulation with unbalanced strata would show whether the centering is correct or a bias term must be subtracted.","The complexity condition suggests a practical diagnostic: estimate the shatter coefficient or the number of cells as a function of $n$ for the chosen clustering algorithm and verify the subexponential growth, something the paper leaves implicit."],"forward_implications":["Coarsened exact matching, k-means confounder clustering, and random-forest proximity clustering all produce asymptotically normal average causal effect estimators under the same subexponential-complexity condition.","The variance estimator (3.9) is justified for the entire class, so users of these algorithms can report confidence intervals from within-stratum sample variances without additional modeling.","For a fixed number of strata, the estimator carries a non-negligible bias equal to the error of approximating the outcome regression functions by piecewise constants, so consistency requires $J\\to\\infty$ with $n$.","Subclassification on a known propensity score falls inside the framework because the propensity score is a function of $X$; subclassification on an estimated propensity score does not, as the strata would then depend on estimated parameters.","The proposed bias correction fits a linear regression of estimates on $J^{-1}$ and extrapolates to $J^{-1}=0$, in analogy to SIMEX extrapolation."],"supporting_citations":[{"why":"Proposes coarsened exact matching and the estimator and variance formula (3.2) whose large-sample validity the paper seeks to establish.","marker":"Iacus et al. (2011)"},{"why":"Supplies Theorem 1, the uniform concentration inequality for partition estimates used to control the bias term $R_n$.","marker":"Lugosi and Nobel (1996)"},{"why":"Provides the martingale representation for matching estimators that the paper adapts to its strata-based estimator.","marker":"Abadie and Imbens (2012)"},{"why":"Cited as the martingale central limit theorem used to prove asymptotic normality of the leading term $W_n$.","marker":"Fleming and Harrington (2013)"},{"why":"Supplies the shatter-coefficient and partition-consistency machinery underlying the complexity condition and the coarsened exact matching bound.","marker":"Devroye et al. (2013)"},{"why":"Gives strong consistency of k-means clustering, used to justify k-means strata as a member of the generalized class.","marker":"Pollard (1981)"},{"why":"Introduces SIMEX, the measurement-error extrapolation procedure on which the paper's bias-correction algorithm is modeled.","marker":"Carroll et al. (1996)"},{"why":"Documents the non-negligible finite-sample bias in matching estimators and the need for bias correction, motivating Section 3.6.","marker":"Abadie and Imbens (2006)"}],"fun_headline_variants":["Any low-complexity strata yield valid causal estimates","Coarsened confounding: general asymptotic theory for causal effects","Any strata with small shatter coefficient give valid causal effects","Generalized coarsened confounding: asymptotic guarantees for any strata","Causal estimates valid for any bounded-complexity clustering"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The main proof assumes the increments built from the stratum-randomized estimator have zero conditional expectation given the data already observed; this fails within any stratum whose treated and control counts are unequal.","fun_headline_variants_meta":{"raw":{"variants":["Any low-complexity strata yield valid causal estimates","Coarsened confounding: general asymptotic theory for causal effects","Any strata with small shatter coefficient give valid causal effects","Generalized coarsened confounding: asymptotic guarantees for any strata","Causal estimates valid for any bounded-complexity clustering"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001553,"raw_usage":{"total_tokens":6209,"prompt_tokens":950,"completion_tokens":5259,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":566,"completion_tokens_details":{"reasoning_tokens":5178}},"tokens_in":566,"tokens_out":5259,"duration_ms":113125,"temperature":1.0,"reasoning_tokens":5178,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:54:12.732217+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Under the paper's own definitions, for $n+1\\le k\\le 2n$ and a treated unit in stratum $j$, the conditional expectation of the increment $\\xi_{n,k}$ given the filtration is $n^{-1/2}[(n_j/n_{1j}-1)\\mu_1(X_k)+\\mu_0(X_k)]$, which is nonzero whenever $n_j\\ne n_{1j}$. A simulation with, say, two strata and imbalanced treatment counts inside one stratum can compute this quantity directly and check whether coverage of the proposed 95% intervals under the theorem's complexity condition matches nominal levels.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Proposes coarsened exact matching and the estimator and variance formula (3.2) whose large-sample validity the paper seeks to establish."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies Theorem 1, the uniform concentration inequality for partition estimates used to control the bias term $R_n$."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the martingale representation for matching estimators that the paper adapts to its strata-based estimator."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Cited as the martingale central limit theorem used to prove asymptotic normality of the leading term $W_n$."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives strong consistency of k-means clustering, used to justify k-means strata as a member of the generalized class."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces SIMEX, the measurement-error extrapolation procedure on which the paper's bias-correction algorithm is modeled."}],"review_version":1}