Environment-Adaptive Covariate Selection: Learning When to Use Spurious Correlations for Out-of-Distribution Prediction

Shuozhi Zuo; Yixin Wang

REVIEW 3 major objections 5 minor 1 cited by

The paper shows that when some causes are unobserved, the optimal covariate set depends on the shift type, and that shift type can be read from unlabeled covariates, so selection should adapt per environment.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 12:31 UTC pith:BD7LPOGV

load-bearing objection EACS is a useful, honest paper that deserves serious refereeing; the empirical idea is solid, but the theory rests on an assumption that is close to the conclusion. the 3 major comments →

arxiv 2601.02322 v2 pith:BD7LPOGV submitted 2026-01-05 stat.ME cs.LG

Environment-Adaptive Covariate Selection: Learning When to Use Spurious Correlations for Out-of-Distribution Prediction

Shuozhi Zuo , Yixin Wang This is my paper

classification stat.ME cs.LG

keywords out-of-distribution predictioncovariate selectionspurious correlationsunobserved causesproxy covariatesenvironment-adaptive algorithminvariancedistribution shift

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper is trying to establish that, when some true causes of an outcome are unobserved, the best covariate set for out-of-distribution prediction is not fixed or invariant: non-causal proxy covariates can be informative when the shift leaves their link to the outcome intact, and harmful when it breaks that link. It argues that different shifts leave distinct observable signatures in the covariate distribution, so a predictor can infer the shift type from unlabeled test covariates and choose covariates accordingly. To this end it introduces EACS, an algorithm that learns a mapping from environment-level covariate summaries to the risk-minimizing covariate subset for each environment. If the central claim is right, robust prediction is not about avoiding spurious correlations altogether but about learning when to trust them, and test-time adaptation with unlabeled data can beat any fixed rule.

Core claim

The paper's central claim is that the optimal covariate set for prediction under distribution shift is environment-dependent, and that the relevant shift information is available from the unlabeled covariate distribution. It demonstrates this in a linear Gaussian example with an observed cause C2, an unobserved cause C1, and a proxy X=C1-C2+epsilon: the proxy helps unless its own noise is inflated, and the crossover is governed by an analytic risk difference depending on variances and correlations. The paper then proposes EACS, which treats covariate selection as risk minimization over environments: each environment is summarized by statistics or learned embeddings of its covariates, a selec

What carries the argument

The central object is the EACS selector: a function g mapping an environment representation u_e, computed from unlabeled covariates only, to a covariate subset z from a finite library Z. Baseline predictors f_z are fit once on pooled training data; for each training environment, empirical risks are used to label the best subset, and a classifier or learned encoder trains g on those pairs. At test time, u_e is computed from the target environment's covariates and g(u_e) chooses which predictor to use. The load-bearing theoretical assumption is that u_e is sufficient: z*(e)=g*(u_e), so the summary carries all distributional information relevant to the subset choice. A continuous soft-gating va

Load-bearing premise

The method inherits its success from a sufficiency assumption: the environment summary u_e must contain all information about the covariate distribution that is relevant to deciding the risk-minimizing subset; if two shifts with different optimal subsets produce the same summary, the selector will choose incorrectly.

What would settle it

Construct two environments in the paper's linear Gaussian model, e.g., a mean shift in the unobserved cause C1 and an augmented-noise shift in the proxy X, chosen so that the summary statistics (correlation and standard deviations) are equal while the risk ordering of subsets {C2} and {C2,X} differs. If such a matched pair exists or is found in real data, then Assumption 1 fails and EACS must underperform a per-environment oracle on those environments.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

If EACS is correct, a practitioner can adapt covariate selection at test time using only unlabeled covariates, with no outcome labels from the target environment.
Causal or invariant covariate restriction is not always the right robustness strategy: when proxy relationships stay stable, keeping non-causal covariates yields lower OOD risk than causal-only selection.
With enough per-environment samples and enough diverse training environments, EACS approaches the oracle that knows the best subset for each shift (Theorems 1 and 2).
Enforcing known causal parents as constraints shrinks the search space and stabilizes selection, helping most when per-environment samples are small or outcome noise is high.
In the paper's empirical evaluations, EACS attains lower average prediction error than fixed causal, invariant, and ERM baselines on both the bike-sharing and census-income applications.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The sufficiency assumption is testable: on held-out environments, one can compare the adaptive rule's risk with that of the best fixed subset; whenever summaries are weak, the paper's own fallback suggests reverting to the fixed rule, which is a practical diagnostic for when adaptation has value.
The argument implies a shift taxonomy: distribution shifts can be classified by which proxy relationships they preserve, and this may be more actionable than the binary spurious-versus-causal view for choosing a prediction strategy.
An obvious extension is to let the environment summary determine not just the subset but the predictor coefficients; the paper deliberately avoids this because of variance, but fully flexible environment-specific predictors remain a boundary case.
For evaluation practice, the paper implies that average OOD performance across mixed shifts masks whether a method adapts; reporting per-shift-type accuracies would make adaptation gains visible.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

EACS is a useful, honest paper that deserves serious refereeing; the empirical idea is solid, but the theory rests on an assumption that is close to the conclusion.

read the letter

Quick take: the EACS idea is real and probably useful. Learning a mapping from unlabeled covariate summaries to covariate subsets, rather than to full predictors, is a nice twist on in-context risk minimization and on algorithm selection. The running example is clean, the analytic condition for when the proxy helps is correctly derived for the shifts they consider, and the simulations show the adaptive rule approaching oracle performance. Code and data are promised, which helps.

What's genuinely new: the formulation itself. Prior work selects among learning algorithms or adapts predictors via context; EACS selects covariate subsets per environment with causal-constrained hard/soft gating. That is a different tool, and the paper gives it a reasonable first pass.

Soft spots, in order of importance. The theoretical results are proof sketches. The |E_train|^{-1/2} generalization term is asserted under unspecified 'mild stability conditions,' and selector complexity is swept into constants. That is not a theorem, and the paper should either supply the proofs or clearly downgrade these to heuristic statements. Assumption 1 is the load-bearing one: it says the environment summary determines the optimal subset, which is basically the property the method needs to work. The paper admits observational equivalence can break it but gives no diagnostic for detecting that on a real dataset. I checked the stress-test's specific variance-shift example (Var(C1)=2 vs noise on X=2 giving identical summaries). Within the paper's own running example the pooled beta3 cannot exceed 1, and for beta3 in [0,1] both environments actually prefer the same subset (or tie at beta3=1), so that particular counterexample doesn't land. The broader concern remains: the summary is not generally sufficient for the joint distribution, and the assumption needs more support or a practical check. Also, the soft-gating variant used in the high-dimensional ACS application is explicitly a heuristic, so the theoretical guarantees do not cover the flagship application. The applied results are consistent with the claims, but the ACS gain is modest, and the bike-sharing comparison may be flattered by ICP collapsing to intercept-only.

Overall: honest, well-written, and the core idea is worth having. The paper is not ready as-is — the theory should be fixed or relabeled, and the sufficiency assumption needs a more careful discussion or a fallback diagnostic — but it deserves a serious referee, and I would expect a conditional accept after revision.

Referee Report

3 major / 5 minor

Summary. The paper proposes environment-adaptive covariate selection (EACS), which learns a mapping from environment-level summaries of unlabeled covariates (e.g., means, SDs, correlations) to environment-specific covariate subsets. The motivating claim is that when some causes are unobserved, non-causal proxies can be useful under some shifts and harmful under others, and that the type of shift is identifiable from the covariate distribution. The paper develops a discrete selector with a claimed oracle inequality and a continuous soft-gating variant, allows causal constraints, and reports simulations plus two applied studies (bike-sharing and ACS Income).

Significance. The paper is clearly written, provides code, and the analytic running example in Section 2.4 and Supplement S1 is correct. If the assumptions held, EACS would be a useful addition to the OOD toolbox, and the idea of adapting covariate subsets using unlabeled target data is worth developing. However, the central premise is not established: the sufficiency assumption (Assumption 1) fails in natural settings, including an extension of the paper's own running example, and the theoretical guarantee is asserted rather than proved. The contribution is therefore conditional and the claims are currently too strong.

major comments (3)

[Section 3.6 (Assumption 1) and Section 2.4/Supplement S1] Assumption 1 is not a mild regularity condition; it fails within the paper's own running example under a natural perturbation class. Consider two environments with Var(C1)=2, Var(epsX)=1 and Var(C1)=1, Var(epsX)=2, with C1, C2, epsX, epsY standard Gaussian and Y=C1+C2+epsY, X=C1-C2+epsX. In both environments, C2~N(0,1) and X|C2~N(-C2,3), so the full observed covariate distribution (C2,X) is identical; hence any summary u_e (including the paper's (r_e, s2,e, s3,e)) is identical. But the risk difference in S1 becomes Delta_e = R_e({C2}) - R_e({C2,X}) = beta3(4-3beta3) in the first environment and beta3(2-3beta3) in the second. For pooled beta3 in (2/3, 4/3), the first prefers {C2,X} and the second prefers {C2}. Thus z*(e) is not a function of u_e, and no covariate-only summary can repair this because the covariate distribution itself is the same. This contradicts the abstract's claim that
[Section 3.6, Theorem 1] The oracle inequality is not a theorem as stated. The term C2 |Etrain|^{-1/2} is asserted to follow from 'mild stability conditions' on the selector, and 'any mild dependence on the selector model... is absorbed into the constant C2.' The proof sketch's component (iii) simply asserts the rate. But the selector is trained on estimated labels bz*_e (which depend on n), and its generalization depends on the function class of f_sel, the number of environments, and the relationship between u_e and z*(e). Without explicit conditions and a derivation, Theorems 1–2 cannot support the paper's claim of 'theoretical guarantees.' Either provide a formal learning bound with stated regularity conditions and a proof, or explicitly label this as a heuristic rate.
[Sections 5 and 7 (experiments)] The reported experiments do not cover the shift class that violates Assumption 1, although the paper itself identifies observational equivalence as a failure mode. In particular, no simulation varies Var(C1), which produces identical observed covariate distributions but different optimal subsets, as shown in the first major comment. Adding such a simulation would demonstrate that EACS degrades exactly when Assumption 1 fails, or that a modified method can recover. Additionally, the applied evaluations use only five outer folds with contiguous block splits; the error bars in Figures 8 and 9 overlap substantially, so the abstract's claim that EACS 'consistently outperforms' static rules is stronger than the evidence supports. The empirical claims should be reworded and the missing stress test should be added.

minor comments (5)

[Contributions, Section 1] Typo: 'The contributions of this paper is as follows' should be 'are as follows.'
[Section 2.4] The notation s2,3,e - s2,2,e in the text is confusing; it mixes subscripts and a squared symbol. Use a consistent notation such as s_{3,e}^2 - s_{2,e}^2 throughout.
[Figure 1 caption] The 'hammer symbol (ὒ8)' appears to be a rendering artifact; use a standard intervention symbol or describe the perturbations in words.
[Section 3.6, Assumption 2] Assumption 2 assumes environments are i.i.d. draws, but the empirical evaluations use contiguous block splits with temporal ordering. This mismatch should be discussed, and the assumption should be stated as applying only to the theoretical analysis.
[Section 3.5] The soft-gating variant is described as a heuristic without inherited guarantees. The paper is appropriately transparent here, but the discussion would be strengthened by a small experiment showing how the temperature tau affects the gap to the discrete selector.

Circularity Check

0 steps flagged

No circular derivation: EACS's guarantees are conditional on an explicitly stated sufficiency assumption, and its empirical claims are evaluated on held-out environments.

full rationale

The derivation chain is not circular. Section 2.4 derives the analytic risk difference Δ_e = 2β3 − β3^2(s3,e^2 − s2,e^2) directly from the linear Gaussian structural equations; this is an independent analytical result, not a restatement of the desired conclusion. The EACS algorithm (Algorithm 1) is a standard meta-learning construction: baseline predictors f_z are fit on pooled training data, environment summaries u_e are computed from unlabeled covariates, per-environment risk estimates produce labels ẑ*_e, and a selector is fit from u_e to those labels. Test-time selection is a genuine prediction on held-out environments/blocks, so the reported gains over static baselines are not fitted to the test labels. Theorems 1–2 are explicitly conditional on Assumption 1 (there exists g* with z*(e) = g*(u_e)) and Assumption 2 (iid environment draws); the paper does not claim to prove Assumption 1, and it expressly acknowledges the two ways it can fail (observational equivalence; overly coarse summaries) in Section 3.6. An assumption that is stated and then used as a premise is a limitation or robustness risk if unverified, but it is not a circular reduction of the theorem's conclusion to its input. The natural failure mode raised by the skeptic — variance shifts of C1 versus additive noise on X producing identical summaries — is a concrete violation of Assumption 1, but the paper already flags observational equivalence as a possible failure and provides no diagnostic; this belongs under correctness risk, not circularity. The only self-citation (Wu et al., 2025, coauthored by Y. Wang) appears as an example source of prior causal knowledge, not as load-bearing support for the central claim. Hence no circular step meets the evidentiary bar.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The central guarantee rests on Assumption 1 (summaries are sufficient for optimal subset choice) and Assumption 2 (environments i.i.d.), both of which the paper flags but does not establish. The only fitted parameters in the analytic example are the pooled regression coefficients; the applied soft-gating temperature and covariance shrinkage are hand-set.

free parameters (3)

Pooled proxy coefficient beta_3 (and beta_2 = 1 + beta_3) in running example = 0.5 in population; estimated by OLS from pooled training data
The analytic condition for when including proxy X helps is s_{3,e}^2 - s_{2,e}^2 > 2/beta_3; beta_3 is a fitted regression coefficient, not a universal constant.
EACS soft-gating temperature tau = 0.20
Set by hand for the ACS experiments; controls how close the continuous mask is to a binary selection.
Covariance shrinkage alpha_max for ACS partial correlations = 0.3
Hand-chosen to stabilize precision-matrix estimates before computing partial correlations.

axioms (5)

ad hoc to paper Assumption 1: the environment summary u_e is sufficient for the optimal subset: there exists g* with z*(e) = g*(u_e).
This is the key premise of Theorem 1 and of the method's success; it is close to the paper's central claim. The paper notes observational equivalence and coarse summaries as failure modes.
domain assumption Assumption 2: training and test environments are i.i.d. draws from a common environment distribution P_U.
Needed for the |E_train|^{-1/2} generalization term; it is at odds with some OOD settings where test shifts are outside the training support. The paper acknowledges this in Supplement S2.
domain assumption Uniform concentration of empirical risks: sup_{z,e} |Rhat_e(z) - R_e(z)| <= C sqrt((log|Z| + log(1/delta))/n).
Stated as a mild assumption in Theorem 1; requires sub-Gaussian squared loss and a finite library Z.
ad hoc to paper The selector model ghat converges to g* at rate |E_train|^{-1/2} under 'mild stability conditions.'
No concrete conditions are given; selector complexity is absorbed into the constant C_2, so the theorem is not fully specified.
domain assumption Running example model: Y = C1 + C2 + eps_Y, X = C1 - C2 + eps_X with independent standard normal terms; perturbations shift C1 mean or add noise to C2 or X.
Used for the running example and the analytic derivation in Section S1; not a general data assumption.

pith-pipeline@v1.3.0-alltime-deepseek · 23649 in / 15119 out tokens · 137464 ms · 2026-08-03T12:31:42.105578+00:00 · methodology

0 comments

read the original abstract

A common approach to out-of-distribution prediction restricts models to causal or invariant covariates to avoid spurious associations that may change across environments. Despite its theoretical appeal, this strategy can underperform empirical risk minimization when only a subset of the causal parents of the outcome is observed. In such settings, non-causal covariates can serve as proxies for unobserved causal parents and improve prediction when the proxy relationship is stable, but they can hurt when shifts disrupt that relationship. Thus, the optimal covariate set can depend on the specific shift encountered. Because different shifts leave signatures in the unlabeled covariate distribution, we propose an environment-adaptive covariate selection algorithm that maps environment-level summaries to environment-specific covariate sets. These summaries may be hand-crafted or learned from multi-environment data, and prior causal knowledge can be incorporated as constraints. Across simulations and applied datasets, the proposed method improves over static causal, invariant, and other non-adaptive rules under diverse shifts.

Figures

Figures reproduced from arXiv: 2601.02322 by Shuozhi Zuo, Yixin Wang.

**Figure 2.** Figure 2: MSE of predictive models using only the observed causal covariate [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Figure 3: Summary statistics of the covariate distribution across environments corresponding to [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

**Figure 4.** Figure 4: OOD prediction MSE of the EACS algorithm under varying numbers of training en [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

**Figure 5.** Figure 5: OOD prediction MSE of the EACS algorithm under varying sample sizes per environ [PITH_FULL_IMAGE:figures/full_fig_p019_5.png] view at source ↗

**Figure 6.** Figure 6: OOD prediction MSE for EACS with causal constraints under condition (i) for different [PITH_FULL_IMAGE:figures/full_fig_p024_6.png] view at source ↗

**Figure 7.** Figure 7: OOD prediction MSE for EACS with causal constraints under condition (ii) for differ [PITH_FULL_IMAGE:figures/full_fig_p025_7.png] view at source ↗

**Figure 8.** Figure 8: The results for the oracle, anchor regression, lasso, EACS, ICP, and fixed-subset base [PITH_FULL_IMAGE:figures/full_fig_p028_8.png] view at source ↗

**Figure 9.** Figure 9: The results for the anchor regression, lasso, EACS, and linear model using all covariates [PITH_FULL_IMAGE:figures/full_fig_p029_9.png] view at source ↗

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Chaos Is a LADDER: Domain Generalization Beyond Invariance via Reweighting
stat.ML 2026-07 conditional novelty 5.0

Reweighting frozen source classifiers by target style-distribution distance (Sinkhorn-KNN) beats pooled and invariant predictors on some rule-varying DG benchmarks.

Reference graph

Works this paper leans on

5 extracted references · 1 linked inside Pith · cited by 1 Pith paper

[1]

Ding, F., Hardt, M., Miller, J., and Schmidt, L. (2021). Retiring adult: new datasets for fair machine learning. InAdvances in Neural Information Processing Systems, volume 34, pages 6478–6490

2021
[2]

and Graff, C

Dua, D. and Graff, C. (2017). Uci machine learning repository

2017
[3]

and Gama, J

Fanaee-T, H. and Gama, J. (2013). Event labeling combining ensemble detectors and background knowledge.Progress in Artificial Intelligence, 2(2–3):113–127

2013
[4]

and Rothenhäusler, D

Jeong, Y. and Rothenhäusler, D. (2025). Out-of-distribution generalization under random, dense distributional shifts.arXiv preprint arXiv:2404.18370

Pith/arXiv arXiv 2025
[5]

Zaheer, M., Kottur, S., Ravanbakhsh, S., Póczos, B., Salakhutdinov, R., and Smola, A. J. (2017). Deep sets. InAdvances in Neural Information Processing Systems, volume 30. 43

2017

[1] [1]

Ding, F., Hardt, M., Miller, J., and Schmidt, L. (2021). Retiring adult: new datasets for fair machine learning. InAdvances in Neural Information Processing Systems, volume 34, pages 6478–6490

2021

[2] [2]

and Graff, C

Dua, D. and Graff, C. (2017). Uci machine learning repository

2017

[3] [3]

and Gama, J

Fanaee-T, H. and Gama, J. (2013). Event labeling combining ensemble detectors and background knowledge.Progress in Artificial Intelligence, 2(2–3):113–127

2013

[4] [4]

and Rothenhäusler, D

Jeong, Y. and Rothenhäusler, D. (2025). Out-of-distribution generalization under random, dense distributional shifts.arXiv preprint arXiv:2404.18370

Pith/arXiv arXiv 2025

[5] [5]

Zaheer, M., Kottur, S., Ravanbakhsh, S., Póczos, B., Salakhutdinov, R., and Smola, A. J. (2017). Deep sets. InAdvances in Neural Information Processing Systems, volume 30. 43

2017