{"id":"5811178a-e2f5-4619-b315-f6e08a8b1051","arxiv_id":"2502.02701","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"CAVS selects, among all valid back-door adjustment sets, the one with the smallest mutual information with the intervention variable, and reports better finite-sample causal effect estimates.","lead":"The paper proposes an algorithm, CAVS, that chooses which confounding variables to adjust for in causal effect calculations by picking the valid set with the weakest statistical association with the treatment. It matters because data is often scarce, and the choice of adjustment variables can make causal estimates unstable when sample sizes are small.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The MI-minimization rule in Step 2 is not derived from any finite-sample risk and is contradicted by the paper's own Table 1, so the central claim that CAVS prevents accuracy degradation lacks theoretical support.","rationale":"The reader's weakest_assumption identifies exactly the load-bearing premise: that minimizing I(X, Si) among minimal valid back-door sets yields the most accurate finite-sample estimates. This is the foundation of Algorithm 1 and is not derived from variance or risk considerations. My reading of the manuscript confirms this: Section 3.2 introduces the rule heuristically, and the experiments only show an average tendency rather than a consistent ordering. Table 1 actually contains a counterexample to strict monotonicity within the paper's own data, though the differences are small and may be noise. The concern is not that the heuristic never works, but that it has no theoretical guarantee and can fail in plausible settings, so the paper's claim of 'preventing accuracy degradation' is overstated. I also note the Theorem 2 proof issue identified by the reader: the proof's second case argues about orienting undirected edges, which is outside the theorem's premise. However, the MI-selection concern is more load-bearing because it affects the primary DAG algorithm and the practical recommendation. The appropriate verdict remains CONDITIONAL: the method is plausible and empirically suggestive, but the central selection rule needs a derivation or a falsification test. I agree with the reader's assessment, so no verdict change is needed.","tokens_in":13155,"tokens_out":2477,"duration_ms":25147,"concrete_test":"Construct a synthetic categorical DAG where two minimal valid back-door sets S1 and S2 satisfy I(X, S1) < I(X, S2) but Var(Y|X, S1) is much larger than Var(Y|X, S2). Sample finite training sets (e.g., n = 500), compute the back-door adjustment estimate using each set, and compare to the true interventional distribution over many replicates. If the set with larger MI (S2) consistently yields lower mean squared error, the Step 2 rule is false.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on Algorithm 1 Step 2 and Eq. (4): among all minimal back-door sets, select the one with smallest mutual information I(X, Si) to minimize finite-sample estimation error. This rule is asserted, not derived. The finite-sample accuracy of the adjustment estimator depends on the variance of the conditional outcome distribution P(Y | X, Si) and on cell counts P(X, Si), not directly on I(X, Si). A set with small MI with X may still have high conditional outcome variance (e.g., if it is a weak predictor of Y), leading to larger estimation error than another valid set with slightly larger MI but much stronger outcome association. The paper's own Table 1 shows non-monotonicity: {RuggedAuto} has the smallest MI (0.00016) but error 0.00189, while {VehicleYear, MakeModel} has larger MI (0.00354) but smaller error (0.00175). Thus the algorithm's hard rule 'minimize MI' is not even strictly supported by the reported data. Because the entire practical benefit of CAVS over arbitrary valid adjustment sets depends on this heuristic, the central claim is insecure without either a theoretical bound or more decisive empirical validation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes CAVS, a two-step method for selecting adjustment variables in categorical causal-effect estimation. Step 1 enumerates minimal sets satisfying the back-door criterion in a known DAG; Step 2 chooses among them the set with smallest mutual information with the intervention variable X. The authors also extend the procedure to CPDAGs, state a sufficient condition for adjustment-based identification in CPDAGs, and evaluate the method on the Insurance and Hailfinder benchmark Bayesian networks plus synthetic DAGs.","tokens_in":13349,"tokens_out":8015,"duration_ms":79684,"significance":"The problem addressed is practically important: multiple valid adjustment sets can exist, and finite-sample accuracy can differ substantially across them. The proposed algorithm is explicit and easy to implement, and the experiments use established discrete Bayesian-network benchmarks, which is a strength. If the mutual-information rule were shown to track finite-sample estimation risk, CAVS would be a useful default method. However, the central selection rule is asserted rather than derived from any risk or variance objective, and the paper's own Table 1 contains a direct counterexample to the monotonic claim. The CPDAG theorem also has a proof gap. The empirical evidence is suggestive but not yet sufficient to establish the main claim.","major_comments":[{"comment":"The selection rule 'choose the minimal back-door set with smallest mutual information I(X, S_i)' is not derived from any finite-sample risk, variance, or bias objective. The paper asserts a monotonic relationship between smaller MI and smaller estimation error, but Table 1 contradicts it: {VehicleYear, MakeModel} has larger MI (0.00354) than {RuggedAuto} (0.00016), yet its mean cosine distance is smaller (0.00175 versus 0.00189). Thus the arg-min-MI choice made by CAVS is strictly worse in that pairwise comparison. Since the entire practical benefit of CAVS over an arbitrary valid minimal adjustment set rests on this rule, the central claim requires either a theoretical justification or substantially more decisive empirical support.","section":"3.2, Algorithm 1 Step 2, Eq. (4), Table 1"},{"comment":"The proof of Theorem 2 jumps from adjustment amenability to the conclusion that 'the adjustment variable is common in all Markov-equivalent DAGs.' Lemmas 1 and 2 are conditional statements about a fixed set Z satisfying conditions 2 and 3 of the generalized adjustment criterion; they do not establish that such a Z exists whenever all edges adjacent to X are directed. Condition 1 of GAC (amenability) alone does not imply conditions 2 and 3. The theorem is load-bearing for the CPDAG extension, so the existence of a GAC-satisfying set must be proved, or the theorem must be weakened to a conditional statement.","section":"3.3, proof of Theorem 2"},{"comment":"The experimental support for the MI rule is narrow: two benchmark datasets and one synthetic setup with 60 error values, with no confidence intervals or repeated-seed variability reported. The Hailfinder experiment reports mutual information for only three of the 14 candidate adjustment sets, so the relationship between MI and error is not systematically checked there. Given the Table 1 reversal, the current evidence does not establish that CAVS prevents accuracy degradation relative to other minimal valid sets.","section":"5.1 and 5.2"}],"minor_comments":[{"comment":"The caption contains a typo, 'T able 1.', which should be corrected to 'Table 1.'","section":"Table 1 caption"},{"comment":"The name 'CAVS' appears with an inserted space as 'CA VS' in several places, including the abstract and Algorithm 1; this should be made consistent.","section":"Throughout"},{"comment":"The baseline described as 'the smallest set of variables among pa(X) that satisfy the back-door criterion' is ambiguous: if there are multiple minimal subsets of pa(X), the reader needs to know which one is used and how it is chosen.","section":"5.2"},{"comment":"Theorem 2 uses 'computable' without specifying the sense of identification; the statement should say 'identifiable via covariate adjustment' to avoid confusion with other identification strategies such as IDA.","section":"3.3"},{"comment":"The notation E_Dk [E_X [...]] is nonstandard because D_k is a random subset and X is already treated as a discrete variable in the inner sum; clarifying the probability spaces would improve reproducibility.","section":"Eq. (5)"}],"recommendation":"major_revision","confidential_remarks":"The topic fits the journal's scope and the paper is readable. The main obstacle is the unproven and partially contradicted MI-minimization rule; if the authors can supply a theoretical bound or a convincing targeted experiment that directly compares oracle-optimal adjustment sets with the MI rule across many graphs, a revision could be salvageable. The CPDAG theorem also needs a rigorous existence proof before the extension claim can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper proposes CAVS: enumerate minimal back-door adjustment sets, then pick the set with smallest mutual information with the intervention variable X. That combination is new as far as I can tell, and it targets a real gap: most efficiency results for adjustment sets assume linear-Gaussian models, while this paper works with categorical data and small samples. The experiments on Insurance, Hailfinder, and random DAGs are suggestive. The artificial-data result (roughly 30% lower error than the smallest parent-based back-door set, 67% lower than full pa(X)) is the strongest evidence, and the use of the Brookhart/Patrick finding about not selecting variables strongly associated with X is sensible. So there is a usable idea here.\n\nThe soft spots are real, but not necessarily fatal. First, the MI-minimization rule is asserted, not derived. The paper does not connect I(X,S) to the variance or finite-sample risk of the adjustment estimator. Table 1 actually undercuts the strict rule: {VehicleYear, MakeModel} has higher MI (0.00354) than {RuggedAuto} (0.00016) yet lower error (0.00175 vs 0.00189). That could be noise, but the paper presents the rule as deterministic and gives no error bars. This needs either a theoretical justification, a softened claim with uncertainty, or a more decisive experiment.\n\nSecond, the proof of Theorem 2 has a structural problem. The theorem states that the intervention effect is computable if all edges adjacent to X are directed. The first case is fine. The second case, however, argues that if not all edges are directed, determining the direction of those edges makes the effect computable. That is a different claim and is not what the theorem states. The proof also slides from \"adjustment amenable\" to \"adjustment variable is common across all Markov-equivalent DAGs\" without the careful conditions that Lemmas 1 and 2 require. The theorem may be salvageable, but the proof as written is not.\n\nMinor issues: no code or detailed enumeration procedure is provided, and the benchmark comparisons lack error bars or significance tests. The citation pattern is fine; the paper builds on Perkovic, Textor-Liskiewicz, and the propensity-score literature appropriately.\n\nMy overall take: the central heuristic is plausible and practically motivated, but it is not established. The paper deserves a serious referee, because the problem matters and the experiments give enough signal to warrant scrutiny. I would not cite it as a reliable method yet, but I would send it to review with a request for major revision, focusing the referee on the MI rule and Theorem 2.","headline":"A practical two-step heuristic for choosing among back-door adjustment sets, with suggestive experiments but an unproven MI-minimization rule and a flawed CPDAG theorem.","tokens_in":13887,"tokens_out":1993,"would_cite":false,"duration_ms":21152,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes CAVS, a two-step algorithm that enumerates minimal back-door adjustment sets and selects the one with the smallest mutual information with the intervention variable, claiming this prevents accuracy degradation in causal…","keywords":["causal inference","adjustment variables","back-door criterion","mutual information","CPDAG","intervention effects","categorical data","finite-sample estimation"],"falsifier":"Generate or take the Insurance DAG and choose conditional distributions so {RuggedAuto} has lower MI with Accident but a nearly empty cell in P(Y|Accident,RuggedAuto), while {VehicleYear,MakeModel} has balanced cells; estimate do-effects from 2,000 samples and compare errors. If the low-MI set is not the most accurate, the central criterion fails; if it is accurate, the claim survives.","tokens_in":108,"feed_emoji":"🎯","tokens_out":5839,"duration_ms":113219,"temperature":0.7,"pith_summary":"When multiple variable sets satisfy the back-door criterion, causal effect estimates should all agree with infinite data, but with finite categorical data the choice matters. This paper proposes a two-step algorithm, CAVS: enumerate all minimal valid adjustment sets from the graph, then select the one with the smallest mutual information with the treatment X. The authors claim this choice reduces and stabilizes estimation error compared with using parents of X or other minimal sets, and demonstrates this on two Bayesian network datasets and random graphs. They also prove a theorem extending applicability to CPDAGs: if all edges adjacent to the treatment are directed, the intervention effect is computable; otherwise orienting at least those edges restores computability. If right, practitioners can get more reliable do-calculus estimates from small categorical datasets without extra assumptions.","feed_headline":"Cut error by adjusting for variables least tied to treatment","feed_subtitle":"CAVS picks the valid back-door set with smallest mutual information with X, improving finite-sample effect estimates.","key_machinery":"The load-bearing object is Algorithm 1, CAVS, a two-step selector. Step 1 converts the DAG by deleting nodes outside ancestors of X or Y and deleting edges out of X, enumerates all paths from X to Y, and finds minimal sets that d-separate them; the authors use the moral graph criterion to justify minimality. Step 2's identity is Cor(X,Si)=I(X,Si)=Σ P(x,s) log(P(x,s)/(P(x)P(s))) over all value combinations, and selects Si minimizing this. The supporting theorem for CPDAGs is Theorem 2: when all edges adjacent to X are directed, every Markov-equivalent DAG has the same adjustment set, so the do-calculus value is unique; if not, orienting edges adjacent to X recovers this. The mutual information criterion is what carries the finite-sample-accuracy claim.","core_discovery":"The paper's central claim is that the adjustment variable should be chosen by data as well as graph: among all minimal sets satisfying the back-door criterion for {X,Y}, the one with the least mutual information I(X,Si) with the intervention variable gives the most accurate finite-sample estimate of P(Y|do(X=x)) when variables are categorical. Algorithm 1 realizes this: Step 1 deletes irrelevant nodes and outgoing edges from X, enumerates paths from X to Y, and returns inclusion-minimal d-separating sets; Step 2 computes Cor(X,Si)=I(X,Si) and sets Z=argmin. The paper further claims Theorem 2: in a CPDAG the intervention effect is computable whenever all edges adjacent to X are directed, because the adjustment set is then common to all Markov-equivalent DAGs; if some adjacent edges are undirected, determining their direction narrows the equivalence class enough to make the effect computable and CAVS applicable.","pith_inferences":["An implication the paper leaves implicit is that the MI-minimization step is a finite-sample heuristic rather than an optimality theorem: nothing in the paper ties I(X,Si) to estimator variance, so there may exist graphs where the lowest-MI set has sparser Y cells and performs worse; a formal variance bound would turn the empirical finding into a guarantee.","The proposed criterion is the categorical analogue of avoiding strong instruments in propensity-score models; one could test it against variance-optimal linear adjustment on continuous data by replacing mutual information with distance correlation or kernel measures.","Theorem 2's condition is graph-local: it suggests a practical diagnostic for whether estimated CPDAGs are ready for intervention analysis, and if the conclusion extends to locally oriented edges, it may reduce the cost of causal discovery in high-dimensional settings."],"forward_implications":["In categorical datasets with limited samples, CAVS should yield smaller and less variable errors in P(Y|do(X=x)) than adjustment by pa(X) or by the smallest parent-based back-door set.","Variables semantically close to X but not needed to block back-door paths will be dropped, reducing the risk of Z-bias and instrument amplification.","For CPDAGs whose treatment-adjacent edges are all directed, the intervention effect is claimed computable from any Markov-equivalent DAG, making CAVS available without orienting the whole graph.","When a CPDAG has undirected edges adjacent to X, orienting only those edges, rather than the full equivalence class, is claimed sufficient to apply CAVS.","The same Step 1 can be extended to joint interventions by replacing the back-door criterion with the admissibility criterion, though the Step 2 criterion for multivariate X is left open."],"supporting_citations":[{"why":"Defines the back-door criterion and the do-calculus adjustment formula that CAVS builds on.","marker":"[4]"},{"why":"Introduces the adjustment criterion; the paper notes that the minimal back-door set is also a minimal adjustment set.","marker":"[5]"},{"why":"Supplies the moral graph criterion and minimal covariate adjustment theorem used in Step 1 to enumerate minimal valid sets.","marker":"[16]"},{"why":"Provides the generalized adjustment criterion and the lemmas that Theorem 2 uses to extend CAVS to CPDAGs.","marker":"[18]"},{"why":"Simulation evidence that variables strongly related to the intervention harm effect estimation, motivating the Step 2 criterion.","marker":"[24]"},{"why":"Empirical comparison of adjustment variable selection strategies supporting the avoidance of strong treatment associations.","marker":"[25]"},{"why":"The Insurance Bayesian network dataset used to demonstrate CAVS's selection and error behavior.","marker":"[31]"},{"why":"The Hailfinder Bayesian network dataset used to test CAVS across decreasing sample sizes.","marker":"[32]"}],"fun_headline_variants":["CAVS: Less info with X, more accurate causal estimates","For causal effects, adjust for variables least tied to X","Pick minimal-MI back-door sets for sharper effect estimates","CAVS: selecting adjustment vars by least mutual information"],"cache_read_input_tokens":16000,"weakest_assumption_plain":"The method assumes that, among all equally valid adjustment sets, the one least statistically associated with the treatment produces the most reliable effect estimate from limited data, and this assumption is asserted rather than proved.","fun_headline_variants_meta":{"raw":{"variants":["CAVS: Less info with X, more accurate causal estimates","For causal effects, adjust for variables least tied to X","Pick minimal-MI back-door sets for sharper effect estimates","CAVS: selecting adjustment vars by least mutual information"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000742,"raw_usage":{"total_tokens":3274,"prompt_tokens":869,"completion_tokens":2405,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":485,"completion_tokens_details":{"reasoning_tokens":2338}},"tokens_in":485,"tokens_out":2405,"duration_ms":17907,"temperature":1.0,"reasoning_tokens":2338,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T11:25:09.821579+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate or take the Insurance DAG and choose conditional distributions so {RuggedAuto} has lower MI with Accident but a nearly empty cell in P(Y|Accident,RuggedAuto), while {VehicleYear,MakeModel} has balanced cells; estimate do-effects from 2,000 samples and compare errors. If the low-MI set is not the most accurate, the central criterion fails; if it is accurate, the claim survives.","supporting_citations":[{"cited_title":"Box plots of errors for CA VS and two baselines (the smallest parents of X that satisfy the back-door criterion, parents of X) on artificial data","cited_arxiv_id":null,"evidence_quote":"Defines the back-door criterion and the do-calculus adjustment formula that CAVS builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the adjustment criterion; the paper notes that the minimal back-door set is also a minimal adjustment set."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the moral graph criterion and minimal covariate adjustment theorem used in Step 1 to enumerate minimal valid sets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the generalized adjustment criterion and the lemmas that Theorem 2 uses to extend CAVS to CPDAGs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Simulation evidence that variables strongly related to the intervention harm effect estimation, motivating the Step 2 criterion."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Empirical comparison of adjustment variable selection strategies supporting the avoidance of strong treatment associations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The Insurance Bayesian network dataset used to demonstrate CAVS's selection and error behavior."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The Hailfinder Bayesian network dataset used to test CAVS across decreasing sample sizes."}],"review_version":1}