Pith. sign in

REVIEW 3 major objections 5 minor 32 references

Practically Effective Adjustment Variable Selection in Causal Inference

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proposes CAVS, a two-step algorithm that enumerates minimal back-door adjustment sets and selects the one with the smallest mutual information with the intervention variable, claiming this prevents accuracy degradation in causal…

desk verdict A practical two-step heuristic for choosing among back-door adjustment sets, with suggestive experiments but an unproven MI-minimization rule and a flawed CPDAG theorem. read the letter →

arxiv 2502.02701 v1 pith:HIO2UPEB submitted 2025-02-04 cs.LG cs.AIphysics.data-anstat.ME

Atsushi Noda , Takashi Isozaki This is my paper · ORCID
classification cs.LGcs.AIphysics.data-anstat.ME
keywords causalinferenceadjustmentvariablesback-doorcriterionmutualinformationCPDAGinterventioneffectscategoricaldatafinite-sampleestimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

When multiple variable sets satisfy the back-door criterion, causal effect estimates should all agree with infinite data, but with finite categorical data the choice matters. This paper proposes a two-step algorithm, CAVS: enumerate all minimal valid adjustment sets from the graph, then select the one with the smallest mutual information with the treatment X. The authors claim this choice reduces and stabilizes estimation error compared with using parents of X or other minimal sets, and demonstrates this on two Bayesian network datasets and random graphs. They also prove a theorem extending applicability to CPDAGs: if all edges adjacent to the treatment are directed, the intervention effect is computable; otherwise orienting at least those edges restores computability. If right, practitioners can get more reliable do-calculus estimates from small categorical datasets without extra assumptions.

What carries the argument

The load-bearing object is Algorithm 1, CAVS, a two-step selector. Step 1 converts the DAG by deleting nodes outside ancestors of X or Y and deleting edges out of X, enumerates all paths from X to Y, and finds minimal sets that d-separate them; the authors use the moral graph criterion to justify minimality. Step 2's identity is Cor(X,Si)=I(X,Si)=Σ P(x,s) log(P(x,s)/(P(x)P(s))) over all value combinations, and selects Si minimizing this. The supporting theorem for CPDAGs is Theorem 2: when all edges adjacent to X are directed, every Markov-equivalent DAG has the same adjustment set, so the do-calculus value is unique; if not, orienting edges adjacent to X recovers this. The mutual information criterion is what carries the finite-sample-accuracy claim.

What would settle it

Generate or take the Insurance DAG and choose conditional distributions so {RuggedAuto} has lower MI with Accident but a nearly empty cell in P(Y|Accident,RuggedAuto), while {VehicleYear,MakeModel} has balanced cells; estimate do-effects from 2,000 samples and compare errors. If the low-MI set is not the most accurate, the central criterion fails; if it is accurate, the claim survives.

Watch

Extended reading notes

Core claim

The paper's central claim is that the adjustment variable should be chosen by data as well as graph: among all minimal sets satisfying the back-door criterion for {X,Y}, the one with the least mutual information I(X,Si) with the intervention variable gives the most accurate finite-sample estimate of P(Y|do(X=x)) when variables are categorical. Algorithm 1 realizes this: Step 1 deletes irrelevant nodes and outgoing edges from X, enumerates paths from X to Y, and returns inclusion-minimal d-separating sets; Step 2 computes Cor(X,Si)=I(X,Si) and sets Z=argmin. The paper further claims Theorem 2: in a CPDAG the intervention effect is computable whenever all edges adjacent to X are directed, because the adjustment set is then common to all Markov-equivalent DAGs; if some adjacent edges are undirected, determining their direction narrows the equivalence class enough to make the effect computable and CAVS applicable.

Load-bearing premise

The method assumes that, among all equally valid adjustment sets, the one least statistically associated with the treatment produces the most reliable effect estimate from limited data, and this assumption is asserted rather than proved.

Editorial extensions

If this is right

  • In categorical datasets with limited samples, CAVS should yield smaller and less variable errors in P(Y|do(X=x)) than adjustment by pa(X) or by the smallest parent-based back-door set.
  • Variables semantically close to X but not needed to block back-door paths will be dropped, reducing the risk of Z-bias and instrument amplification.
  • For CPDAGs whose treatment-adjacent edges are all directed, the intervention effect is claimed computable from any Markov-equivalent DAG, making CAVS available without orienting the whole graph.
  • When a CPDAG has undirected edges adjacent to X, orienting only those edges, rather than the full equivalence class, is claimed sufficient to apply CAVS.
  • The same Step 1 can be extended to joint interventions by replacing the back-door criterion with the admissibility criterion, though the Step 2 criterion for multivariate X is left open.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit is that the MI-minimization step is a finite-sample heuristic rather than an optimality theorem: nothing in the paper ties I(X,Si) to estimator variance, so there may exist graphs where the lowest-MI set has sparser Y cells and performs worse; a formal variance bound would turn the empirical finding into a guarantee.
  • The proposed criterion is the categorical analogue of avoiding strong instruments in propensity-score models; one could test it against variance-optimal linear adjustment on continuous data by replacing mutual information with distance correlation or kernel measures.
  • Theorem 2's condition is graph-local: it suggests a practical diagnostic for whether estimated CPDAGs are ready for intervention analysis, and if the conclusion extends to locally oriented edges, it may reduce the cost of causal discovery in high-dimensional settings.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper proposes CAVS, a two-step method for selecting adjustment variables in categorical causal-effect estimation. Step 1 enumerates minimal sets satisfying the back-door criterion in a known DAG; Step 2 chooses among them the set with smallest mutual information with the intervention variable X. The authors also extend the procedure to CPDAGs, state a sufficient condition for adjustment-based identification in CPDAGs, and evaluate the method on the Insurance and Hailfinder benchmark Bayesian networks plus synthetic DAGs.

Significance. The problem addressed is practically important: multiple valid adjustment sets can exist, and finite-sample accuracy can differ substantially across them. The proposed algorithm is explicit and easy to implement, and the experiments use established discrete Bayesian-network benchmarks, which is a strength. If the mutual-information rule were shown to track finite-sample estimation risk, CAVS would be a useful default method. However, the central selection rule is asserted rather than derived from any risk or variance objective, and the paper's own Table 1 contains a direct counterexample to the monotonic claim. The CPDAG theorem also has a proof gap. The empirical evidence is suggestive but not yet sufficient to establish the main claim.

major comments (3)
  1. [3.2, Algorithm 1 Step 2, Eq. (4), Table 1] The selection rule 'choose the minimal back-door set with smallest mutual information I(X, S_i)' is not derived from any finite-sample risk, variance, or bias objective. The paper asserts a monotonic relationship between smaller MI and smaller estimation error, but Table 1 contradicts it: {VehicleYear, MakeModel} has larger MI (0.00354) than {RuggedAuto} (0.00016), yet its mean cosine distance is smaller (0.00175 versus 0.00189). Thus the arg-min-MI choice made by CAVS is strictly worse in that pairwise comparison. Since the entire practical benefit of CAVS over an arbitrary valid minimal adjustment set rests on this rule, the central claim requires either a theoretical justification or substantially more decisive empirical support.
  2. [3.3, proof of Theorem 2] The proof of Theorem 2 jumps from adjustment amenability to the conclusion that 'the adjustment variable is common in all Markov-equivalent DAGs.' Lemmas 1 and 2 are conditional statements about a fixed set Z satisfying conditions 2 and 3 of the generalized adjustment criterion; they do not establish that such a Z exists whenever all edges adjacent to X are directed. Condition 1 of GAC (amenability) alone does not imply conditions 2 and 3. The theorem is load-bearing for the CPDAG extension, so the existence of a GAC-satisfying set must be proved, or the theorem must be weakened to a conditional statement.
  3. [5.1 and 5.2] The experimental support for the MI rule is narrow: two benchmark datasets and one synthetic setup with 60 error values, with no confidence intervals or repeated-seed variability reported. The Hailfinder experiment reports mutual information for only three of the 14 candidate adjustment sets, so the relationship between MI and error is not systematically checked there. Given the Table 1 reversal, the current evidence does not establish that CAVS prevents accuracy degradation relative to other minimal valid sets.
minor comments (5)
  1. [Table 1 caption] The caption contains a typo, 'T able 1.', which should be corrected to 'Table 1.'
  2. [Throughout] The name 'CAVS' appears with an inserted space as 'CA VS' in several places, including the abstract and Algorithm 1; this should be made consistent.
  3. [5.2] The baseline described as 'the smallest set of variables among pa(X) that satisfy the back-door criterion' is ambiguous: if there are multiple minimal subsets of pa(X), the reader needs to know which one is used and how it is chosen.
  4. [3.3] Theorem 2 uses 'computable' without specifying the sense of identification; the statement should say 'identifiable via covariate adjustment' to avoid confusion with other identification strategies such as IDA.
  5. [Eq. (5)] The notation E_Dk [E_X [...]] is nonstandard because D_k is a random subset and X is already treated as a discrete variable in the inner sum; clarifying the probability spaces would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: CAVS's MI-minimization rule and Theorem 2 rest on external criteria and independent evaluation, not on self-referential reduction.

full rationale

The paper's derivation chain is not circular. Algorithm 1 has two steps: enumerate minimal back-door sets using standard d-separation on the DAG, then choose the set Si minimizing the mutual information I(X,Si). This MI criterion is not defined in terms of the paper's later cosine-distance error metric; I(X,Si) is computed from P(X,Si), while the evaluation compares full-sample and subset estimates of P(Y|do(X)). No parameter is fitted to the evaluation outcome, and the empirical benchmarks are independent of the method's construction. The CPDAG claim (Theorem 2) is derived from externally cited results on the generalized adjustment criterion [18] and from the paper's own proof; [18] is by other authors and is not the paper's premise restated. The cited lemmas and theorem are used as external mathematical support, not as a self-citation chain. The main weakness is that the MI-minimization rule is asserted as a heuristic rather than derived from a finite-sample variance or risk objective, and Table 1 even shows a mild non-monotonic pattern (e.g., {RuggedAuto} has smaller MI but larger error than {VehicleYear, MakeModel}). That is a correctness/support concern, not circularity: the predicted quantity is not equivalent to the selection input by construction, and no self-referential reduction can be exhibited.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No free parameters are fitted and no invented entities are introduced. The method rests on standard causal inference assumptions plus one ad hoc heuristic: that the valid adjustment set with the smallest mutual information with X gives the most accurate finite-sample estimates.

assumptions (5)
  • domain assumption Causal Markov condition and all confounders observed
    Section 2.1: X is independent of V \ {X, des(X)} given pa(X), and all confounding variables necessary for intervention calculations are observed.
  • domain assumption All variables are categorical and fully observed
    Section 2.1: 'Assume that the values of all nodes D are observed and all values are categorical.'
  • domain assumption The DAG or CPDAG correctly represents the causal structure
    Section 2.1: the DAG may be given by human knowledge or estimated from data; the method inherits the standard no-hidden-confounding and correct-structure assumptions.
  • standard math Validity of the generalized adjustment criterion theorem of Perkovic et al.
    Section 3.3 relies on Theorem 1 and Lemmas 1-2 from Perkovic et al. 2015 as external results; the CPDAG extension is built directly on this framework.
  • ad hoc to paper Minimizing mutual information with X minimizes finite-sample estimation error
    Algorithm 1 Step 2 selects argmin_{Si} I(X,Si). No proof is given that this minimizes any estimation loss; it is the paper's central unproved heuristic.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Practically Effective Adjustment Variable Selection in Causal Inference." pith.science (2026). https://pith.science/paper/HIO2UPEB

@misc{pith2026250202701,
  author       = {Pith},
  title        = {Pith review of: Practically Effective Adjustment Variable Selection in Causal Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HIO2UPEB}},
  note         = {Machine review of arXiv:2502.02701}
}
read the original abstract

In the estimation of causal effects, one common method for removing the influence of confounders is to adjust the variables that satisfy the back-door criterion. However, it is not always possible to uniquely determine sets of such variables. Moreover, real-world data is almost always limited, which means it may be insufficient for statistical estimation. Therefore, we propose criteria for selecting variables from a list of candidate adjustment variables along with an algorithm to prevent accuracy degradation in causal effect estimation. We initially focus on directed acyclic graphs (DAGs) and then outlines specific steps for applying this method to completed partially directed acyclic graphs (CPDAGs). We also present and prove a theorem on causal effect computation possibility in CPDAGs. Finally, we demonstrate the practical utility of our method using both existing and artificial data.

Figures

Figures reproduced from arXiv: 2502.02701 by the authors.

Figure 1
Figure 1. (Left) There is a strong correlation between X and P1, indicated by the bold arrow. P1 definitely closes the back-door path, but so do V 1 and V 2. (Right) X has three parents {P1, P2, P3}, but only V 1 is sufficient to close the back-door path. know which one should be used for adjustment. In any case, when a causal relationship between variables can be constructed based on expert knowledge, or when the DAG is esti… view at source ↗
Figure 2
Figure 2. Specific DAG and CPT examples that reduce accuracy. In the conditional probability table for Y , the numbers in parentheses indicate the number of data samples, and the numbers above them indicate the probability value of Y obtained by aggregating the data. In this example, the computational accuracy of do(X = 0) calculus will be unstable due to the small number of data samples available for the cases X = 0 and Z = … view at source ↗
Figure 3
Figure 3. An example explaining each step of CAVS algorithm. (a) shows the DAG, (b) shows the graph after Step 1-1 and (c) shows the graph after Step 1-2. (i) No element in Z is a descendant in G of any W ∈ V \ X that lies on a causal path from X to Y . (ii) All non-causal paths in G from X to Y are blocked by Z. In extending this criterion to CPDAG, we introduce the following concept called amenability [18]. Definition 3 (Am… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: (a) An example of CPDAG. (b) (c) (d) Markov-equivalent DAGs, but with different adjustment variables for each graph. (b) and (c) have the same direction of edges adjacent to X and the same adjustment variables {Z1, Z2}. Otherwise, the direction of the edges adjacent to…
Figure 5
Figure 5. Figure 5: DAG structure of Insurance data. The intervention variable Accident locates one level above of the outcome variable OtherCarCost. And it can be seen that Accident has three parent variables {Mileage, Antilock, DrivQuality}. Adjustment variable candidates MI with interv…
Figure 6
Figure 6. Figure 6: DAG structure of Hailfinder data. The intervention variable N34StarFcst locates one level above of the outcome variable R5Fcst. There are the parent set of the intervention variable {ScenRel3 4, PlainsFcst} and the last node in the back-door path from the intervention …
Figure 7
Figure 7. Figure 7: Relationship between sample size and error for each adjustment variable candidate in the Hailfinder data. There is an inversely proportional relationship between sample size and error. Particularly, the error increases sharply when the sample size is small. When {Mount…
Figure 8
Figure 8. Figure 8: Box plots of errors for CAVS and two baselines (the smallest parents of X that satisfy the back-door criterion, parents of X) on artificial data. Of these, CAVS has the smallest error and the smallest error variance, indicating that it is the best and most stable resul…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 32 canonical work pages

  1. [1]

    Wright S 1934 The method of path coefficients The Annals of Mathematical Statistics 5 161–215

    work page 1934

  2. [2]

    Pearl J 1995 Causal diagrams for empirical research Biometrika 82 669–688

    work page 1995

  3. [3]

    Spirtes P, Glymour C N and Scheines R 2000 Causation, prediction, and search (MIT Press)

    work page 2000

  4. [4]

    Box plots of errors for CA VS and two baselines (the smallest parents of X that satisfy the back-door criterion, parents of X) on artificial data

    Pearl J 2009 Causality (Cambridge University Press) Practically Effective Adjustment Variable Selection in Causal Inference 19 Figure 8. Box plots of errors for CA VS and two baselines (the smallest parents of X that satisfy the back-door criterion, parents of X) on artificial data. Of these, CA VS has the smallest error and the smallest error variance, i...

    work page 2009

  5. [5]

    Shpitser I, VanderWeele T and Robins J M 2010 On the validity of covariate adjustment for estimating causal effects Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence UAI’10 (AUAI Press) p 527–536

    work page 2010

  6. [6]

    Shrier I and Platt R W 2008 Reducing bias through directed acyclic graphs BMC Medical Research Methodology 8 1–15

    work page 2008

  7. [7]

    Kuroki M and Miyakawa M 2003 Covariate selection for estimating the causal effect of control plans by using causal diagrams Journal of the Royal Statistical Society Series B: Statistical Methodology 65 209–222

    work page 2003

  8. [8]

    Kuroki M and Cai Z 2004 Selection of identifiability criteria for total effects by using path diagrams Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence UAI ’04 (AUAI Press) p 333–340

    work page 2004

Show all 32 references
  1. [9]

    Henckel L, Perkovi´ c E and Maathuis M H 2022 Graphical criteria for efficient total effect estimation via adjustment in causal linear models Journal of the Royal Statistical Society Series B: Statistical Methodology 84 579–599

    2022

  2. [10]

    Witte J, Henckel L, Maathuis M H and Didelez V 2020 On efficient adjustment in causal graphs The Journal of Machine Learning Research 21 9956–10000

    2020

  3. [11]

    Cheng D, Li J, Liu L, Le T D, Liu J and Yu K 2022 Sufficient dimension reduction for average causal effect estimation Data Mining and Knowledge Discovery 36 1174–1196

    2022

  4. [12]

    Meek C 1995 Causal inference and causal explanation with background knowledge Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence UAI’95 (Morgan Kaufmann Publishers Inc.) p 403–410

    1995

  5. [13]

    1997 A characterization of markov equivalence classes for acyclic digraphs The Annals of Statistics 25 505–541

    Andersson S A, Madigan D, Perlman M D et al. 1997 A characterization of markov equivalence classes for acyclic digraphs The Annals of Statistics 25 505–541

    1997

  6. [14]

    Chickering D M 2002 Learning equivalence classes of Bayesian-network structures Journal of Machine Learning Research 2 445–498

    2002

  7. [15]

    Angrist J D, Imbens G W and Rubin D B 1996 Identification of causal effects using instrumental variables Journal of the American statistical Association 91 444–455 Practically Effective Adjustment Variable Selection in Causal Inference 20

    1996

  8. [16]

    Textor J and Li´ skiewicz M 2011 Adjustment criteria in causal diagrams: an algorithmic perspective Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence UAI’11 (AUAI Press) p 681–688

    2011

  9. [17]

    Pearl J and Robins J M 1995 Probabilistic evaluation of sequential plans from causal models with hidden variables UAI ’95: Proceedings of the Eleventh Annual Conference on Uncertainty in Artificial Intelligence, Montreal, Quebec, Canada, August 18-20, 1995 (Morgan Kaufmann) pp 444–453

    1995

  10. [18]

    Perkovi´ c E, Textor J, Kalisch M and Maathuis M H 2015 A complete generalized adjustment criterion Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence UAI’15 (AUAI Press) p 682–691

    2015

  11. [19]

    Ding P, VanderWeele T and Robins J M 2017 Instrumental variables as bias amplifiers with general outcome and confounding Biometrika 104 291–302

    2017

  12. [20]

    Hernan M and Robins J 2024 Causal Inference: What If (CRC Press)

    2024

  13. [21]

    VanderWeele T J and Shpitser I 2011 A new criterion for confounder selection Biometrics 67 1406–1413

    2011

  14. [22]

    VanderWeele T J 2019 Principles of confounder selection European Journal of Epidemiology 34 211–219

    2019

  15. [23]

    Rosenbaum P R and Rubin D B 1983 The central role of the propensity score in observational studies for causal effects Biometrika 70 41–55

    1983

  16. [24]

    Brookhart M A, Schneeweiss S, Rothman K J, Glynn R J, Avorn J and St¨ urmer T 2006 Variable selection for propensity score models American Journal of Epidemiology 163 1149–1156

    2006

  17. [25]

    Patrick A R, Schneeweiss S, Brookhart M A, Glynn R J, Rothman K J, Avorn J and St¨ urmer T 2011 The implications of propensity score variable selection strategies in pharmacoepidemiology: an empirical illustration Pharmacoepidemiology and Drug Safety 20 551–559

    2011

  18. [26]

    Rosenbaum P R 1984 The consequences of adjustment for a concomitant variable that has been affected by the treatment Journal of the Royal Statistical Society: Series A (General) 147 656–666

    1984

  19. [27]

    Cole S R, Platt R W, Schisterman E F, Chu H, Westreich D, Richardson D and Poole C 2010 Illustrating bias due to conditioning on a collider International Journal of Epidemiology 39 417–420

    2010

  20. [28]

    Greenland S 2003 Quantifying biases in causal models: Classical confounding vs collider- stratification bias Epidemiology 14 300–306

    2003

  21. [29]

    Maathuis M H and Colombo D 2015 A generalized back-door criterion The Annals of Statistics 43 1060–1088

    2015

  22. [30]

    Maathuis M H, Kalisch M and B¨ uhlmann P 2009 Estimating high-dimensional intervention effects from observational data The Annals of Statistics 37 3133–3164

    2009

  23. [31]

    Binder J, Koller D, Russell S and Kanazawa K 1997 Adaptive probabilistic networks with hidden variables Machine Learning 29 213–244

    1997

  24. [32]

    Abramson B, Brown J, Edwards W, Murphy A and Winkler R L 1996 Hailfinder: A bayesian system for forecasting severe weather International Journal of Forecasting 12 57–71

    1996

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.