Pith. sign in

REVIEW 4 major objections 4 minor 23 references

Finite sample-optimal adjustment sets in linear Gaussian causal models

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proves a finite-sample mean-squared-error identity for OLS covariate adjustment and shows the best adjustment set uses only non-suboptimal confounding and precision variables.

desk verdict Clean exact MSE criterion, promising but under-proven graph pruning—worth reviewing, but Theorem 2's proof needs a serious fix. read the letter →

arxiv 2506.12869 v1 pith:UWQT7BQG submitted 2025-06-15 math.ST stat.MEstat.TH

Nadja Rutsch , Sara Magliacane , Stéphanie van der Pas This is my paper · ORCID
classification math.STstat.MEstat.TH MSC 62D2062J0562F12
keywords adjustmentsetaveragetreatmenteffectcausalinferencelinearGaussianmodelmeansquarederrorordinaryleastsquaresgraphicalcriteriafinitesample
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks which covariates to adjust for when estimating a causal effect with ordinary least squares from a finite sample, and it argues that the right target is the adjustment set with smallest mean squared error, not the unbiased set with best asymptotic variance. For linear Gaussian causal models it derives an exact MSE decomposition, squared bias plus asymptotic variance divided by $n - |K| - 3$, and a necessary-and-sufficient sample-size condition for deciding when one adjustment set beats another. It then proves that a best set always exists among candidates containing only non-suboptimal confounding and precision variables, so the search over all covariate subsets can be pruned using the causal graph alone. The practical consequence is that at small sample sizes, deliberately using an invalid or smaller adjustment set can yield a lower mean squared error than the asymptotically optimal set, and the paper's simulations show such gains.

What carries the argument

The load-bearing object is the finite-sample variance formula $\mathrm{var}(\hat\tau_K) = \mathrm{aVar}(\hat\tau_K)/(n - |K| - 3)$, which follows from the ordinary least squares residual sum of squares having an inverse-chi-squared distribution under Gaussian errors. The second engine is the graph-theoretic notion of a suboptimal variable: a precision variable $S$ is suboptimal if another precision variable $P^*$ blocks every path from $S$ to $Y$ in the graph without the edge $A \to Y$, and a confounding variable is suboptimal if $S$ is separated from $Y$ given $W^*$ and $W^*$ from $A$ given $S$. The paper's lemmas show that such variables can be replaced by their blocking partner without increasing MSE, so adjustment-set selection becomes a finite, graph-pruned search.

What would settle it

Enumerate all small linear Gaussian DAGs with, say, five or six nodes that satisfy the pre-treatment assumption, and check for any pair $(S, W^*)$ meeting Definition 7 while $S$ remains connected in the graph to some other extended confounder $U$ given some conditioning set $Z$. If such a graph exists, numerically compute the OLS MSE of sets $K \cup S$ and $K \setminus S \cup W^*$ with known coefficients and Gaussian errors to see whether the inequality $\mathrm{MSE}(\hat\tau_{K\cup S}) \ge \mathrm{MSE}(\hat\tau_{K\setminus S\cup W^*})$ required by Theorem 2 is violated; that would settle Lemma 5(b) directly.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central result is the identity $\mathrm{MSE}(\hat\tau_K) = B^2(\hat\tau_K) + \mathrm{aVar}(\hat\tau_K)/(n - |K| - 3)$, where $B$ is the estimator's bias and $\mathrm{aVar}$ its asymptotic variance $\sigma_{yy\cdot A K}/\sigma_{AA\cdot K}$. Comparing two adjustment sets therefore reduces to comparing bias, asymptotic variance, and set size: if $B^2(\hat\tau_K) > B^2(\hat\tau_L)$, then $K$ has lower MSE than $L$ exactly when the sample size satisfies Inequality (3). The paper further proves (Theorem 2) that there exists an MSE-optimal adjustment set contained in the non-suboptimal precision and extended-confounding variables, so irrelevant variables, suboptimal precision variables, suboptimal confounding variables, and certain forbidden combinations can be removed from consideration using d-separation checks on the graph. In the simulated examples the resulting MSE-optimal set differs from the asymptotically optimal set at finite $n$ and outperforms it, while converging to it (or to another set) as $n$ grows.

Load-bearing premise

The reduction to non-suboptimal confounding variables rests on the claim that a suboptimal confounding variable is d-separated from every other extended confounder given any conditioning set (Lemma 5(b)); the paper's proof of this claim contains an incomplete sentence and a case analysis that is not fully convincing, so if the claim fails the search over the reduced candidate set could miss the true MSE-optimal adjustment set.

Editorial extensions

If this is right

  • For any two adjustment sets, the paper's criterion $n < \frac{\mathrm{aVar}(\hat\tau_L) - \frac{n-|L|-3}{n-|K|-3}\mathrm{aVar}(\hat\tau_K)}{B^2(\hat\tau_K)-B^2(\hat\tau_L)} + |L| + 3$ tells a researcher which set has lower expected MSE at a given sample size.
  • The MSE-optimal set can always be chosen from the non-suboptimal confounding and precision variables, so the search space is much smaller than the full power set of covariates.
  • In finite samples, invalid adjustment sets and even the empty set can beat the asymptotically optimal set $O$, making it unnecessary to measure expensive covariates when $n$ is small.
  • The paper's selection algorithm matches or beats the asymptotically optimal adjustment set in simulated linear Gaussian settings, with the largest gains at small sample sizes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Not tested in the paper: if the same finite-sample denominator arises for weighted least squares or for multivariate treatments, the bias-versus-variance tradeoff could extend beyond scalar OLS adjustment.
  • The forbidden-combination criterion resembles conditional-independence checks used in standard variable selection, so it may be possible to automate the graph pruning without enumerating all subsets.
  • The practical method requires estimating bias from data via bootstrap; a natural extension would be to use cross-validated MSE estimates when the causal graph is only partially known.
  • The contested d-separation claim behind Theorem 2 suggests that a brute-force search over small graphs could reveal whether the pruned candidate space always contains the true optimum or only a near-optimum.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper studies finite-sample optimal covariate adjustment for average treatment effect estimation in linear Gaussian causal models using ordinary least squares. It defines the MSE-optimal adjustment set, derives a necessary and sufficient sample-size criterion for comparing two adjustment sets (Theorem 1), and develops graphical rules to prune the search space (Theorems 2-4), claiming that an MSE-optimal set exists containing only non-suboptimal precision and confounding variables. Simulations on two toy models show that the MSE-optimal set can outperform the asymptotically optimal adjustment set at finite sample sizes.

Significance. If the main claims hold, the paper makes an original contribution by moving beyond asymptotic efficiency to finite-sample MSE-based adjustment set selection, and by providing a sample-size criterion with a closed-form comparison formula. The derivation of Theorem 1 is exact, the simulations match its predictions, and the code is made available; these are concrete strengths. However, the proof of the central search-space reduction (Theorem 2) rests on Lemma 9 and Lemma 5, whose proofs contain substantial gaps. The significance is therefore conditional on repairing these proofs.

major comments (4)
  1. [Supplementary Material, Appendix 2, proof of Lemma 9] In the proof of Lemma 9, the set U is introduced only by requiring that K ∪ S ∪ U be a K ∪ S-irreducible adjustment set, but Lemma 5 is stated under the stronger hypothesis that S ∪ W* ∪ Z ∪ U is S ∪ W* ∪ Z-irreducible. No argument shows that adding W* preserves irreducibility or even validity of the larger set. Consequently, the applications of Lemma 5(a)-(e) and the covariance equalities σ_ua.ks = σ_ua.k\sw* and Σ_uu.ks = Σ_uu.k\sw* (items (3) and (4) in the proof) are unsupported, and the key inequality MSE(τ̂K∪S) ≥ MSE(τ̂K\S∪W*) does not follow. This is a load-bearing gap for Theorem 2.
  2. [Supplementary Material, Appendix 2, Lemma 5(b)] The proof of Lemma 5(b) asserts 'By the definition of extended confounding variables, there exists a path φ = (S,...,W*)'. Definition 6 only guarantees that S and W* are each separately d-connected to A and to Y; it does not imply a direct d-connection between S and W*. The subsequent claim that A,Y ∉ φ is also not justified: a path through A can be blocked by conditioning on S, and a path through Y can be blocked by conditioning on W*, so the suboptimality d-separations in Definition 7 do not rule out such paths. The proof of S ⊥⊥G U | Z is therefore incomplete.
  3. [Supplementary Material, Appendix 2, Lemma 5(ii)-(iii)] The derivations of Lemma 5(ii) and (iii) do not correctly apply the contraction property. To obtain (ii) from (i), one would need W* ⊥⊥ A | Z ∪ S ∪ U, but Lemma 5(a) only provides W* ⊥⊥ A | Z ∪ S. To obtain (iii), the proof combines A ⊥⊥ Y | S ∪ W* ∪ Z\S ∪ U with S ⊥⊥ Y | W* ∪ Z\S ∪ U, whereas contraction requires a condition of the form A ⊥⊥ S | ... (or S ⊥⊥ A | ...), not S ⊥⊥ Y | ... . As written, the claimed validity of S ∪ Z ∪ U and W* ∪ Z\S ∪ U is not established, and Lemma 9's use of these validity statements is again unsupported.
  4. [Supplementary Material, Appendix 2, Lemma 5(b) case analysis] The case analysis in the proof of Lemma 5(b) is not fully verifiable. For example, the sentence 'all paths of the form πb ⊆ contain W* as a non-collider' is incomplete, and the transitions from concatenated paths π'1 and π'2 to simple paths with specified colliders are not rigorously justified. Because Lemma 5(b) is used for the covariance equalities in Lemma 9, this part of the proof needs to be completed or restructured.
minor comments (4)
  1. [Main text, Theorem 4] Theorem 4 has a typo: '|L| ≥O' should read '|L| ≥ |O|'.
  2. [Main text, Section 3.3] The main text refers to 'Lemma B10' in the Supplementary Material, but the supplementary lemmas are numbered from 1; please align the cross-references.
  3. [Main text, Theorem 1] The sample size criterion in Equation (3) is stated under the condition B^2(τ̂K) > B^2(τ̂L); the case of equal squared biases is not addressed and could be mentioned for completeness.
  4. [Supplementary Material, proof of Lemma 9] The notation in the proof, e.g., '|K ∪ S|', treats a single variable S as a set; this is understandable but should be made explicit, for instance by writing K ∪ {S}.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Theorem 1 rearranges an exact OLS variance decomposition, and Theorem 2's reduction rests on external graphical lemmas rather than on its own conclusion.

full rationale

The paper's core claims are derived from standard distributional facts and external results, not from their own conclusions. Theorem 1 follows by algebraically rearranging the exact decomposition MSE(τ̂_K) = B²(τ̂_K) + aVar(τ̂_K)/(n − |K| − 3), which is itself derived from the inverse-chi-squared distribution of RSS_{a.k}/σ_{aa.k}; no fitted parameter, estimated quantity, or MSE-optimal set is used as an input. The search-space reduction in Theorem 2 is supported by d-separation lemmas whose proofs use the causal Markov property, contraction/weak-union, and external lemmas from Henckel et al. and Peña. Even where the proof of Lemma 5(b) contains a gap, that gap concerns the validity of a d-separation claim and is not a circular step. The experiments simulate data from the stated models and compare estimators, confirming rather than constructing the theoretical predictions. There is no load-bearing self-citation chain, and the asymptotically optimal set O is taken from prior external work. Consequently, no derivation step reduces by construction to its own input.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central derivation uses only the structure of linear Gaussian SEMs, the standard OLS variance formula, and cited lemmas from Henckel et al. and Peña. No constants are fitted to make the examples work; the model coefficients in Figure 1 are illustrative simulation inputs. The graphical definitions (suboptimal precision/confounding variables) are new definitions, not empirical inputs. The bootstrap in Algorithm 1 introduces a tuning choice but does not affect the theory.

assumptions (5)
  • domain assumption Linear Gaussian causal model with independent noise, Equation (1).
    The entire MSE analysis assumes each variable is a linear function of its parents plus independent Gaussian noise.
  • domain assumption Causal Markov and faithfulness assumptions.
    Invoked in Section 2.1 to connect d-separation in the graph to conditional independence in the distribution.
  • domain assumption Pre-treatment variables: no mediators, Assumption 1.
    All covariates are assumed to be non-descendants of the treatment, so mediators are excluded from the graph.
  • domain assumption Known causal graph and known population bias/variance quantities for the criterion.
    Theorem 1 compares population MSEs; the algorithm estimates these quantities from data, but the theory assumes they are available.
  • standard math Distributional result: RSS/sigma^2 follows chi-square with n-|K|-1 degrees of freedom under Gaussianity.
    This standard result underlies the exact variance formula var(tau_K) = aVar/(n-|K|-3) used in Theorem 1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Finite sample-optimal adjustment sets in linear Gaussian causal models." pith.science (2026). https://pith.science/paper/UWQT7BQG

@misc{pith2026250612869,
  author       = {Pith},
  title        = {Pith review of: Finite sample-optimal adjustment sets in linear Gaussian causal models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UWQT7BQG}},
  note         = {Machine review of arXiv:2506.12869}
}
read the original abstract

Traditional covariate selection methods for causal inference focus on achieving unbiasedness and asymptotic efficiency. In many practical scenarios, researchers must estimate causal effects from observational data with limited sample sizes or in cases where covariates are difficult or costly to measure. Their needs might be better met by selecting adjustment sets that are finite sample-optimal in terms of mean squared error. In this paper, we aim to find the adjustment set that minimizes the mean squared error of the causal effect estimator, taking into account the joint distribution of the variables and the sample size. We call this finite sample-optimal set the MSE-optimal adjustment set and present examples in which the MSE-optimal adjustment set differs from the asymptotically optimal adjustment set. To identify the MSE-optimal adjustment set, we then introduce a sample size criterion for comparing adjustment sets in linear Gaussian models. We also develop graphical criteria to reduce the search space for this adjustment set based on the causal graph. In experiments with simulated data, we show that the MSE-optimal adjustment set can outperform the asymptotically optimal adjustment set in finite sample size settings, making causal inference more practical in such scenarios.

Figures

Figures reproduced from arXiv: 2506.12869 by the authors.

Figure 1
Figure 1. Two toy examples of causal models M1 and M2 and the root-mean squared error (RMSE) of the ordinary least squares estimator ˆτ of the causal effect τ of A on Y in M1 and M2, using different adjustment sets (10000 random seeds per set and sample size). The variables in M1 and M2 follow linear equations with the shown coefficients and additive Gaussian noise with a variance of 1 and a mean of 0. In both examples, O = {… view at source ↗
Figure 2
Figure 2. Example graph G3, G ′ 3 is the same without the edge be￾tween A and Y . For an example, see G3 in [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 19 canonical work pages

  1. [1]

    Jay Bhattacharya and William B. Vogt. Do Instrumental Variables Belong in Propensity Scores? NBER Technical Working Papers 0343, National Bureau of Economic Research, Inc, September 2007. URL https://ideas.repec.org/p/nbr/nberte/0343.html

    work page 2007

  2. [2]

    Variable selection for propensity score models

    M Alan Brookhart, Sebastian Schneeweiss, Kenneth J Rothman, Robert J Glynn, Jerry Avorn, and Til St \"u rmer. Variable selection for propensity score models. American journal of epidemiology, 163 0 (12): 0 1149--1156, 2006

    work page 2006

  3. [3]

    Long story short: Omitted variable bias in causal machine learning

    Victor Chernozhukov, Carlos Cinelli, Whitney Newey, Amit Sharma, and Vasilis Syrgkanis. Long story short: Omitted variable bias in causal machine learning. Working Paper 30302, National Bureau of Economic Research, July 2022. URL http://www.nber.org/papers/w30302

    work page 2022

  4. [4]

    Making Sense of Sensitivity: Extending Omitted Variable Bias

    Carlos Cinelli and Chad Hazlett. Making Sense of Sensitivity: Extending Omitted Variable Bias . Journal of the Royal Statistical Society Series B: Statistical Methodology, 82 0 (1): 0 39--67, 12 2019. ISSN 1369-7412. doi:10.1111/rssb.12348. URL https://doi.org/10.1111/rssb.12348

    doi:10.1111/rssb.12348 2019

  5. [5]

    A. P. Dawid. Conditional independence in statistical theory. Journal of the Royal Statistical Society. Series B (Methodological), 41 0 (1): 0 1--31, 1979. ISSN 00359246. URL http://www.jstor.org/stable/2984718

    work page arXiv 1979

  6. [6]

    Markov properties for graphical models with cycles and latent variables

    Patrick Forr \'e and Joris M Mooij. Markov properties for graphical models with cycles and latent variables. arXiv preprint arXiv:1710.08775, 2017

    arXiv 2017

  7. [7]

    William H. Greene. Econometric Analysis. Pearson Education, fifth edition, 2003. ISBN 0-13-066189-9. URL http://pages.stern.nyu.edu/ wgreene/Text/econometricanalysis.htm

    work page 2003

  8. [8]

    Richard Guo, Emilija Perkovi \'c , and Andrea Rotnitzky

    F. Richard Guo, Emilija Perkovi \'c , and Andrea Rotnitzky. Variable elimination, graph reduction and the efficient g-formula. Biometrika, 110 0 (3): 0 739--761, 2023

    work page 2023

Show all 23 references
  1. [9]

    Maathuis

    Leonard Henckel, Emilija Perkovi \'c , and Marloes H. Maathuis. Graphical Criteria for Efficient Total Effect Estimation Via Adjustment in Causal Linear Models . Journal of the Royal Statistical Society Series B: Statistical Methodology, 84 0 (2): 0 579--599, April 2022. ISSN ...

    2022 doi

  2. [10]

    Hernan and J.M

    M.A. Hernan and J.M. Robins. Causal Inference: What If . Chapman & Hall/ CRC Monographs on Statistics & Applied Probab. CRC Press, 2024. ISBN 978-1-4200-7616-5

    2024

  3. [11]

    Middleton, Marc A

    Joel A. Middleton, Marc A. Scott, Ronli Diakow, and Jennifer L. Hill. Bias amplification and bias unmasking. Political Analysis, 24 0 (3): 0 307–323, 2016. doi:10.1093/pan/mpw015

    2016 doi

  4. [12]

    Myers, Jeremy A

    Jessica A. Myers, Jeremy A. Rassen, Joshua J. Gagne, Krista F. Huybrechts, Sebastian Schneeweiss, Kenneth J. Rothman, Marshall M. Joffe, and Robert J. Glynn. Effects of Adjusting for Instrumental Variables on Bias and Precision of Effect Estimates . American Journal of Epidemi...

    2011 doi

  5. [13]

    Factorization of the partial covariance in singly-connected path diagrams

    Jose Pe\ na. Factorization of the partial covariance in singly-connected path diagrams. In Mihaela van der Schaar, Cheng Zhang, and Dominik Janzing, editors, Proceedings of the Second Conference on Causal Learning and Reasoning, volume 213 of Proceedings of Machine Learning Re...

    2023

  6. [14]

    Comment: Graphical models, causality and intervention

    Judea Pearl. Comment: Graphical models, causality and intervention. Statistical Science, 8 0 (3): 0 266--269, 1993. ISSN 08834237. URL http://www.jstor.org/stable/2245965

    1993

  7. [15]

    Causality: Models, Reasoning and Inference

    Judea Pearl. Causality: Models, Reasoning and Inference. Cambridge University Press, USA, 2nd edition, 2009. ISBN 052189560X

    2009

  8. [16]

    On a class of bias-amplifying variables that endanger effect estimates

    Judea Pearl. On a class of bias-amplifying variables that endanger effect estimates. In Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence, UAI'10, page 417–424, Arlington, Virginia, USA, 2010. AUAI Press. ISBN 9780974903965

    2010

  9. [17]

    Maathuis

    Emilija Perkovi\' c , Johannes Textor, Markus Kalisch, and Marloes H. Maathuis. A complete generalized adjustment criterion. In Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence, UAI'15, page 682–691, Arlington, Virginia, USA, 2015. AUAI Pres...

    2015

  10. [18]

    Complete graphical characterization and construction of adjustment sets in markov equivalence classes of ancestral graphs

    Emilija Perković, Johannes Textor, Markus Kalisch, Marloes H Maathuis, et al. Complete graphical characterization and construction of adjustment sets in markov equivalence classes of ancestral graphs. Journal of Machine Learning Research, 18 0 (220): 0 1--62, 2018

    2018

  11. [19]

    Efficient adjustment sets for population average treatment effect estimation in non-parametric causal graphical models, December 2019

    Andrea Rotnitzky and Ezequiel Smucler. Efficient adjustment sets for population average treatment effect estimation in non-parametric causal graphical models, December 2019

    2019

  12. [20]

    Ilya Shpitser, Tyler VanderWeele, and James M. Robins. On the validity of covariate adjustment for estimating causal effects. In Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence, UAI'10, page 527–536, Arlington, Virginia, USA, 2010. AUAI Pre...

    2010

  13. [21]

    The importance of covariate selection in controlling for selection bias in observational studies

    Peter M Steiner, Thomas D Cook, William R Shadish, and Margaret H Clark. The importance of covariate selection in controlling for selection bias in observational studies. Psychological methods, 15 0 (3): 0 250, 2010

    2010

  14. [22]

    Separators and adjustment sets in causal graphs: Complete criteria and an algorithmic framework

    Benito van der Zander , Maciej Liśkiewicz, and Johannes Textor. Separators and adjustment sets in causal graphs: Complete criteria and an algorithmic framework. Artificial Intelligence, 270: 0 1--40, 2019. ISSN 0004-3702. doi:https://doi.org/10.1016/j.artint.2018.12.006. URL h...

    2019 doi

  15. [23]

    Wooldridge

    Jeffrey M. Wooldridge. Should instrumental variables be used as matching variables? Research in Economics, 70 0 (2): 0 232--237, 2016. ISSN 1090-9443. doi:https://doi.org/10.1016/j.rie.2016.01.001. URL https://www.sciencedirect.com/science/article/pii/S1090944315301678

    2016 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.