{"id":"00dc4ff2-04fa-4ce1-9417-05f40634c6aa","arxiv_id":"2506.12869","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"In linear Gaussian causal models, the covariate adjustment set that minimizes finite-sample mean squared error can include invalid sets, and a new criterion plus graph-based pruning finds it.","lead":"The paper studies which covariates to control for when estimating a cause-effect relationship from a small dataset. It shows that the usual 'statistically optimal' set of controls can be worse than a deliberately biased set, and it gives rules for finding the set that minimizes average squared error.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Lemma 9 applies Lemma 5 under an irreducibility hypothesis that its chosen U does not satisfy, so the covariance equalities supporting Theorem 2's search-space reduction are unproven.","rationale":"The reader's weakest_assumption correctly targets Lemma 5(b) and its role in Lemma 9, and I agree that the proof is not convincing as written. My stress-test sharpens the concern: beyond the incomplete sentence and complex case analysis, there is a potential mismatch of hypotheses. Lemma 9 chooses U so that K ∪ S ∪ U is K ∪ S-irreducible, but Lemma 5 requires S ∪ W* ∪ Z ∪ U to be S ∪ W* ∪ Z-irreducible. No argument in the paper shows that the former implies the latter, or that the weaker condition suffices for the covariance equalities. The proof of Lemma 5(b) also assumes a path between S and W* without justification. If the d-separation S ⊥⊥ U | Z fails under the weaker hypothesis, then the replacement argument in Lemma 9 collapses, and with it the search-space reduction of Theorem 2. The rest of the paper is on firmer ground: Theorem 1's variance formula MSE = bias^2 + aVar/(n - |K| - 3) follows from the exact inverse-chi-squared distribution of RSS_a.k, and the simulations in Tables 1-3 are consistent with the stated formulas. The issue is confined to the graphical reduction, which is the main algorithmic contribution. Because the gap is potentially fixable by a complete proof or a machine-checked derivation, the appropriate verdict remains CONDITIONAL rather than REJECT; my read does not change the reader's verdict.","tokens_in":22815,"tokens_out":14845,"duration_ms":139574,"concrete_test":"Exhaustively enumerate all DAGs on six nodes (3^15 ≈ 14 million) with A,Y distinguished and no A→Y edge. For each graph and each assignment of S,W*,U ⊆ V \\ {A,Y} and Z, check whether Definition 7 holds and whether K ∪ S ∪ U is K ∪ S-irreducible for K = Z, the hypothesis actually available in Lemma 9. Then test whether S ⊥⊥_{G'} U | Z. If any instance satisfies the hypotheses but fails the d-separation, Lemma 9's equality (3) fails in that graph and Theorem 2's reduction is false or needs an additional condition. If no counterexample appears across the exhaustive space, the d-separation is empirically supported under the weaker hypothesis, and the missing proof is a presentational gap rather than a mathematical error.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Theorem 2's search-space reduction depends on Lemma 9, which replaces a suboptimal confounding variable S by W* and needs covariance equalities (1)-(5) in its proof. Those equalities are obtained by applying Lemma 5(a)-(e), but Lemma 5 is stated under the hypothesis that S ∪ W* ∪ Z ∪ U is S ∪ W* ∪ Z-irreducible. In Lemma 9, by contrast, U is introduced only as a set making K ∪ S ∪ U a K ∪ S-irreducible adjustment set (proof of Lemma 9, after Eq. 19). It is never shown that adding W* preserves irreducibility or even validity of S ∪ W* ∪ (K \\ S) ∪ U. Lemma 5(iii) is invoked to conclude that K \\ S ∪ W* ∪ U is valid, yet (iii) is derived from Lemma 5(i), whose proof simply cites the irreducibility assumption. If W* makes some element of U redundant, Lemma 5(b) need not hold for this U, and the equalities σ_ua.ks = σ_ua.k\\sw* and Σ_uu.ks = Σ_uu.k\\sw* (items (3) and (4) in Lemma 9's proof) lack support. Second, within the proof of Lemma 5(b), the line 'By the definition of extended confounding variables, there exists a path φ = (S,...,W*)' assumes a d-connection between S and W*; the definition only gives each variable separate d-connections to A and Y. The subsequent claim that A,Y ∉ φ because otherwise the suboptimality d-separations would fail is not justified: a path through A can be blocked by conditioning on S, and a path through Y by conditioning on W*, so the two defining d-separations do not rule out such paths. These issues are fixable with a complete proof, but as written the proof of Theorem 2 does not close.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies finite-sample optimal covariate adjustment for average treatment effect estimation in linear Gaussian causal models using ordinary least squares. It defines the MSE-optimal adjustment set, derives a necessary and sufficient sample-size criterion for comparing two adjustment sets (Theorem 1), and develops graphical rules to prune the search space (Theorems 2-4), claiming that an MSE-optimal set exists containing only non-suboptimal precision and confounding variables. Simulations on two toy models show that the MSE-optimal set can outperform the asymptotically optimal adjustment set at finite sample sizes.","tokens_in":23169,"tokens_out":7048,"duration_ms":70269,"significance":"If the main claims hold, the paper makes an original contribution by moving beyond asymptotic efficiency to finite-sample MSE-based adjustment set selection, and by providing a sample-size criterion with a closed-form comparison formula. The derivation of Theorem 1 is exact, the simulations match its predictions, and the code is made available; these are concrete strengths. However, the proof of the central search-space reduction (Theorem 2) rests on Lemma 9 and Lemma 5, whose proofs contain substantial gaps. The significance is therefore conditional on repairing these proofs.","major_comments":[{"comment":"In the proof of Lemma 9, the set U is introduced only by requiring that K ∪ S ∪ U be a K ∪ S-irreducible adjustment set, but Lemma 5 is stated under the stronger hypothesis that S ∪ W* ∪ Z ∪ U is S ∪ W* ∪ Z-irreducible. No argument shows that adding W* preserves irreducibility or even validity of the larger set. Consequently, the applications of Lemma 5(a)-(e) and the covariance equalities σ_ua.ks = σ_ua.k\\sw* and Σ_uu.ks = Σ_uu.k\\sw* (items (3) and (4) in the proof) are unsupported, and the key inequality MSE(τ̂K∪S) ≥ MSE(τ̂K\\S∪W*) does not follow. This is a load-bearing gap for Theorem 2.","section":"Supplementary Material, Appendix 2, proof of Lemma 9"},{"comment":"The proof of Lemma 5(b) asserts 'By the definition of extended confounding variables, there exists a path φ = (S,...,W*)'. Definition 6 only guarantees that S and W* are each separately d-connected to A and to Y; it does not imply a direct d-connection between S and W*. The subsequent claim that A,Y ∉ φ is also not justified: a path through A can be blocked by conditioning on S, and a path through Y can be blocked by conditioning on W*, so the suboptimality d-separations in Definition 7 do not rule out such paths. The proof of S ⊥⊥G U | Z is therefore incomplete.","section":"Supplementary Material, Appendix 2, Lemma 5(b)"},{"comment":"The derivations of Lemma 5(ii) and (iii) do not correctly apply the contraction property. To obtain (ii) from (i), one would need W* ⊥⊥ A | Z ∪ S ∪ U, but Lemma 5(a) only provides W* ⊥⊥ A | Z ∪ S. To obtain (iii), the proof combines A ⊥⊥ Y | S ∪ W* ∪ Z\\S ∪ U with S ⊥⊥ Y | W* ∪ Z\\S ∪ U, whereas contraction requires a condition of the form A ⊥⊥ S | ... (or S ⊥⊥ A | ...), not S ⊥⊥ Y | ... . As written, the claimed validity of S ∪ Z ∪ U and W* ∪ Z\\S ∪ U is not established, and Lemma 9's use of these validity statements is again unsupported.","section":"Supplementary Material, Appendix 2, Lemma 5(ii)-(iii)"},{"comment":"The case analysis in the proof of Lemma 5(b) is not fully verifiable. For example, the sentence 'all paths of the form πb ⊆ contain W* as a non-collider' is incomplete, and the transitions from concatenated paths π'1 and π'2 to simple paths with specified colliders are not rigorously justified. Because Lemma 5(b) is used for the covariance equalities in Lemma 9, this part of the proof needs to be completed or restructured.","section":"Supplementary Material, Appendix 2, Lemma 5(b) case analysis"}],"minor_comments":[{"comment":"Theorem 4 has a typo: '|L| ≥O' should read '|L| ≥ |O|'.","section":"Main text, Theorem 4"},{"comment":"The main text refers to 'Lemma B10' in the Supplementary Material, but the supplementary lemmas are numbered from 1; please align the cross-references.","section":"Main text, Section 3.3"},{"comment":"The sample size criterion in Equation (3) is stated under the condition B^2(τ̂K) > B^2(τ̂L); the case of equal squared biases is not addressed and could be mentioned for completeness.","section":"Main text, Theorem 1"},{"comment":"The notation in the proof, e.g., '|K ∪ S|', treats a single variable S as a set; this is understandable but should be made explicit, for instance by writing K ∪ {S}.","section":"Supplementary Material, proof of Lemma 9"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and practical problem, and Theorem 1 is a clean exact result. My main concern is that the proof of Theorem 2, which is the headline graphical reduction, depends on Lemma 9 and Lemma 5, and the current text does not provide a rigorous derivation of the required d-separation and covariance statements. These gaps are substantial but appear fixable with additional proof work rather than indicating a fundamentally flawed approach."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe headline: this paper has one clean exact result and one promising but currently under-supported graph result. Theorem 1—the exact finite-sample MSE decomposition for OLS adjustment sets—is correct and useful, and the simulations confirm it. The examples showing that a deliberately biased set can beat the asymptotically optimal set at small n make the point well. That part is solid.\n\nThe genuinely new stuff is the graphical search-space reduction (Theorems 2–4). The idea that you can prune suboptimal precision variables, suboptimal confounding variables, and certain combinations just from the graph is valuable and goes beyond the asymptotic literature. I also appreciate that they treat invalid sets systematically rather than treating validity as a binary.\n\nNow the soft spots. The proof of Theorem 2 runs through Lemma 9, which applies Lemma 5 to a set U that may not satisfy Lemma 5's hypotheses. Lemma 5 requires S∪W*∪Z∪U to be S∪W*∪Z-irreducible; Lemma 9 only tells you K∪S∪U is K∪S-irreducible. Nothing in the proof shows that adding W* preserves irreducibility or even validity, and without that, the covariance equalities (3) and (4) in Lemma 9 are unsupported. The reader's report flags exactly this, and I think it's a real gap, not a nitpick.\n\nThere's also a smaller problem inside Lemma 5(b). The proof says 'By the definition of extended confounding variables, there exists a path φ = (S,...,W*)'. That does not follow. Extended confounding only gives separate d-connections to A and Y, not a d-connection between two extended confounders. So that lemma needs rework too.\n\nThese are fixable, but as written the main graph-pruning theorem isn't fully proven. The experiments are simple enough that they don't rescue the proof gap. Also, the paper says code is on github but gives no link or commit hash; for a reproducibility-minded venue that's worth insisting on.\n\nWho should read this: anyone working on covariate selection for causal effect estimation in linear Gaussian models, especially at small sample sizes. It deserves a serious referee—the core idea is good and the exact result is worth publishing even if the pruning part needs heavy revision. I would not desk-reject.\n\nRecommendation: send to review, but make clear the authors must fix Lemma 9/Lemma 5 or the paper's main theoretical claim stays unproven.","headline":"Clean exact MSE criterion, promising but under-proven graph pruning—worth reviewing, but Theorem 2's proof needs a serious fix.","tokens_in":23723,"tokens_out":4107,"would_cite":false,"duration_ms":40934,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62D20","62J05","62F12"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proves a finite-sample mean-squared-error identity for OLS covariate adjustment and shows the best adjustment set uses only non-suboptimal confounding and precision variables.","keywords":["adjustment set","average treatment effect","causal inference","linear Gaussian model","mean squared error","ordinary least squares","graphical criteria","finite sample"],"falsifier":"Enumerate all small linear Gaussian DAGs with, say, five or six nodes that satisfy the pre-treatment assumption, and check for any pair $(S, W^*)$ meeting Definition 7 while $S$ remains connected in the graph to some other extended confounder $U$ given some conditioning set $Z$. If such a graph exists, numerically compute the OLS MSE of sets $K \\cup S$ and $K \\setminus S \\cup W^*$ with known coefficients and Gaussian errors to see whether the inequality $\\mathrm{MSE}(\\hat\\tau_{K\\cup S}) \\ge \\mathrm{MSE}(\\hat\\tau_{K\\setminus S\\cup W^*})$ required by Theorem 2 is violated; that would settle Lemma 5(b) directly.","tokens_in":22608,"feed_emoji":"🎯","tokens_out":9162,"duration_ms":91050,"temperature":0.7,"pith_summary":"This paper asks which covariates to adjust for when estimating a causal effect with ordinary least squares from a finite sample, and it argues that the right target is the adjustment set with smallest mean squared error, not the unbiased set with best asymptotic variance. For linear Gaussian causal models it derives an exact MSE decomposition, squared bias plus asymptotic variance divided by $n - |K| - 3$, and a necessary-and-sufficient sample-size condition for deciding when one adjustment set beats another. It then proves that a best set always exists among candidates containing only non-suboptimal confounding and precision variables, so the search over all covariate subsets can be pruned using the causal graph alone. The practical consequence is that at small sample sizes, deliberately using an invalid or smaller adjustment set can yield a lower mean squared error than the asymptotically optimal set, and the paper's simulations show such gains.","feed_headline":"Small samples favor different adjustment sets","feed_subtitle":"In small samples, a deliberately biased covariate set can beat the asymptotically best one.","key_machinery":"The load-bearing object is the finite-sample variance formula $\\mathrm{var}(\\hat\\tau_K) = \\mathrm{aVar}(\\hat\\tau_K)/(n - |K| - 3)$, which follows from the ordinary least squares residual sum of squares having an inverse-chi-squared distribution under Gaussian errors. The second engine is the graph-theoretic notion of a suboptimal variable: a precision variable $S$ is suboptimal if another precision variable $P^*$ blocks every path from $S$ to $Y$ in the graph without the edge $A \\to Y$, and a confounding variable is suboptimal if $S$ is separated from $Y$ given $W^*$ and $W^*$ from $A$ given $S$. The paper's lemmas show that such variables can be replaced by their blocking partner without increasing MSE, so adjustment-set selection becomes a finite, graph-pruned search.","core_discovery":"On the paper's own terms, the central result is the identity $\\mathrm{MSE}(\\hat\\tau_K) = B^2(\\hat\\tau_K) + \\mathrm{aVar}(\\hat\\tau_K)/(n - |K| - 3)$, where $B$ is the estimator's bias and $\\mathrm{aVar}$ its asymptotic variance $\\sigma_{yy\\cdot A K}/\\sigma_{AA\\cdot K}$. Comparing two adjustment sets therefore reduces to comparing bias, asymptotic variance, and set size: if $B^2(\\hat\\tau_K) > B^2(\\hat\\tau_L)$, then $K$ has lower MSE than $L$ exactly when the sample size satisfies Inequality (3). The paper further proves (Theorem 2) that there exists an MSE-optimal adjustment set contained in the non-suboptimal precision and extended-confounding variables, so irrelevant variables, suboptimal precision variables, suboptimal confounding variables, and certain forbidden combinations can be removed from consideration using d-separation checks on the graph. In the simulated examples the resulting MSE-optimal set differs from the asymptotically optimal set at finite $n$ and outperforms it, while converging to it (or to another set) as $n$ grows.","pith_inferences":["Not tested in the paper: if the same finite-sample denominator arises for weighted least squares or for multivariate treatments, the bias-versus-variance tradeoff could extend beyond scalar OLS adjustment.","The forbidden-combination criterion resembles conditional-independence checks used in standard variable selection, so it may be possible to automate the graph pruning without enumerating all subsets.","The practical method requires estimating bias from data via bootstrap; a natural extension would be to use cross-validated MSE estimates when the causal graph is only partially known.","The contested d-separation claim behind Theorem 2 suggests that a brute-force search over small graphs could reveal whether the pruned candidate space always contains the true optimum or only a near-optimum."],"forward_implications":["For any two adjustment sets, the paper's criterion $n < \\frac{\\mathrm{aVar}(\\hat\\tau_L) - \\frac{n-|L|-3}{n-|K|-3}\\mathrm{aVar}(\\hat\\tau_K)}{B^2(\\hat\\tau_K)-B^2(\\hat\\tau_L)} + |L| + 3$ tells a researcher which set has lower expected MSE at a given sample size.","The MSE-optimal set can always be chosen from the non-suboptimal confounding and precision variables, so the search space is much smaller than the full power set of covariates.","In finite samples, invalid adjustment sets and even the empty set can beat the asymptotically optimal set $O$, making it unnecessary to measure expensive covariates when $n$ is small.","The paper's selection algorithm matches or beats the asymptotically optimal adjustment set in simulated linear Gaussian settings, with the largest gains at small sample sizes."],"supporting_citations":[{"why":"supplies the asymptotic variance formula, the optimal adjustment set $O$, and the covariance lemmas used throughout the proofs","marker":"[Henckel et al., 2022]"},{"why":"supplies Lemma 16 used to equate partial covariances under d-separation in the replacement arguments","marker":"[Peña, 2023]"},{"why":"provides the do-operator and d-separation definitions that define the causal effect and the graphical criteria","marker":"[Pearl, 2009]"},{"why":"gives a necessary-and-sufficient adjustment criterion that distinguishes valid sets in the proofs of Theorems 3 and 4","marker":"[Shpitser et al., 2010]"},{"why":"completes the graphical characterization of valid adjustment sets used to identify unbiased candidates","marker":"[Perković et al., 2018]"},{"why":"defines efficient asymptotically optimal adjustment sets in the non-parametric setting that motivates the finite-sample comparison","marker":"[Rotnitzky and Smucler, 2019]"},{"why":"introduces precision variables, the class of covariates around which Definition 4 and Lemma 8 are built","marker":"[Brookhart et al., 2006]"},{"why":"provides the intuitive characterization of the optimal adjustment set $O$ used in Definition 2","marker":"[Guo et al., 2023]"},{"why":"supplies the contraction, weak union, and composition properties used to derive the d-separation lemmas","marker":"[Dawid, 1979]"}],"fun_headline_variants":["Finite samples change optimal covariate choice","Small data: biased covariate sets win","MSE-optimal adjustment sets shift at finite n","Optimal covariates depend on sample size"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reduction to non-suboptimal confounding variables rests on the claim that a suboptimal confounding variable is d-separated from every other extended confounder given any conditioning set (Lemma 5(b)); the paper's proof of this claim contains an incomplete sentence and a case analysis that is not fully convincing, so if the claim fails the search over the reduced candidate set could miss the true MSE-optimal adjustment set.","fun_headline_variants_meta":{"raw":{"variants":["Finite samples change optimal covariate choice","Small data: biased covariate sets win","MSE-optimal adjustment sets shift at finite n","Optimal covariates depend on sample size"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000173,"raw_usage":{"total_tokens":1289,"prompt_tokens":968,"completion_tokens":321,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":265}},"tokens_in":584,"tokens_out":321,"duration_ms":3801,"temperature":1.0,"reasoning_tokens":265,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:38:15.588744+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Enumerate all small linear Gaussian DAGs with, say, five or six nodes that satisfy the pre-treatment assumption, and check for any pair $(S, W^*)$ meeting Definition 7 while $S$ remains connected in the graph to some other extended confounder $U$ given some conditioning set $Z$. If such a graph exists, numerically compute the OLS MSE of sets $K \\cup S$ and $K \\setminus S \\cup W^*$ with known coefficients and Gaussian errors to see whether the inequality $\\mathrm{MSE}(\\hat\\tau_{K\\cup S}) \\ge \\mathrm{MSE}(\\hat\\tau_{K\\setminus S\\cup W^*})$ required by Theorem 2 is violated; that would settle Lemma 5(b) directly.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"gives a necessary-and-sufficient adjustment criterion that distinguishes valid sets in the proofs of Theorems 3 and 4"},{"cited_title":"Efficient adjustment sets for population average treatment effect estimation in non-parametric causal graphical models, December 2019","cited_arxiv_id":null,"evidence_quote":"defines efficient asymptotically optimal adjustment sets in the non-parametric setting that motivates the finite-sample comparison"},{"cited_title":"Variable selection for propensity score models","cited_arxiv_id":null,"evidence_quote":"introduces precision variables, the class of covariates around which Definition 4 and Lemma 8 are built"}],"review_version":1}