From Training to Deployment: Post-Hoc Causal Feature Identification via Sensitivity Ratios

Athanasios Vlontzos; Bernhard Kainz; Giorgos Papanastasiou; Sotirios Tsaftaris

REVIEW 3 major objections 4 minor 18 references

From Training to Deployment: Post-Hoc Causal Feature Identification via Sensitivity Ratios

T0 review · 3 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a single statistic—the squared coefficient of variation of per-environment sensitivity—exactly separates causal from spurious features in an already-trained linear model, provided environments shift only spurious featu

desk verdict NSR is a genuinely new post-hoc causal/spurious feature diagnostic with a clean linear-SCM identification theorem, but the exactness claim hides a strong fixed-causal-marginal assumption that the paper's own bike-sharing experiment appears to violate. read the letter →

arxiv 2607.25546 v1 pith:UQJ4RJ2K submitted 2026-07-28 cs.AI

Athanasios Vlontzos , Giorgos Papanastasiou , Bernhard Kainz , Sotirios Tsaftaris This is my paper · ORCID

classification cs.AI

keywords CausalfeatureidentificationPost-hocmodelauditingSpuriouscorrelationNormalisedSensitivityRatioStructureddistributionshiftStructuralanalysisattribution

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to solve a practical problem: once a model is deployed, how do you tell which features it relies on for genuinely causal reasons and which only because of environment-specific correlations? Its answer is the Normalised Sensitivity Ratio (NSR), computed by measuring, for each feature, how much the model's output changes when the feature is replaced by a draw from another environment's distribution, and then taking the squared coefficient of variation of that change across environments. The paper proves that, for any linear model with nonzero weights, NSR is exactly zero for causal features and strictly positive for spurious features, as long as there are at least three environments that shift only spurious feature means and at unequal distances from a reference. That would matter because it means deployed models can be audited post-hoc—without retraining, gradient access, or a causal graph—in the multi-site and multi-batch settings where spurious correlations silently break. The paper also gives exact rates and conditions for the method's failure, so practitioners know when to trust it.

What carries the argument

The central object is the Normalised Sensitivity Ratio: for each feature j, sample the model's mean absolute output change when values of j are replaced by draws from a reference environment's marginal, do this for each environment, and take the squared coefficient of variation (variance over environments divided by squared mean). For a linear predictor the per-environment sensitivity factorises as |w_j|·δ_j, where δ_j depends only on the feature's marginal across environments; the |w_j| factor and its square cancel in the CV² ratio, so the statistic isolates δ_j. Because δ_j is constant across environments for causal features and strictly increasing in the absolute environment shift for spu

What would settle it

A direct test of the central claim is to simulate the paper's linear SCM with at least three environments whose absolute shifts from the chosen reference are unequal, train any linear model with nonzero weights, and compute NSR for every feature; the theorem predicts exactly zero for causal features and strictly positive for spurious ones. Observing a nonzero NSR for a causal feature or a zero NSR for a spurious feature under these conditions would refute the identification claim. A complementary test is to let causal-feature means vary across environments while keeping P(Y|X_C) fixed; the pap

Watch

Extended reading notes

Core claim

The central claim is Theorem 4.1: under the paper's linear structural causal model (Definition 1), if the causal mechanism and causal-feature marginals are identical across environments while spurious features shift by a scalar per environment, then for any linear predictor with nonzero weights the Normalised Sensitivity Ratio of a feature j equals zero if and only if j is causal, and is strictly positive if and only if j is spurious. Crucially, the ratio does not depend on the learned weight w_j, because the squared weight cancels when the variance across environments is normalised by the square of the mean sensitivity. The identification signal therefore lives in the geometry of the enviro

Load-bearing premise

The load-bearing premise is that causal features' own distributions are identical across environments—not just the causal law—so if demographic or case-mix variation shifts causal covariates, NSR will label those causal features as spurious.

Editorial extensions

If this is right

A practitioner with only environment labels, held-out data, and forward passes of a trained linear model can identify the causal/spurious partition exactly, without retraining or knowing the model class.
Features with small coefficients are not hidden: because the ratio normalises out w_j², a spurious feature with negligible weight is still detected.
Before applying NSR, a practitioner can check the three quantitative conditions—at least three environments, non-constant absolute shifts from the reference, and shift-to-noise ratio not near zero—and know in advance whether the diagnostic will work or collapse as O(ε⁴).
NSR and training-time invariant-learning approaches are complements, not competitors: each detects spurious features under shift schedules the other misses, so the right tool depends on the environment geometry.
The O_p(n^{-1}) null rate versus O_p(n^{-1/2}) alternative rate gives a distribution-free permutation test with growing separation as the sample size increases.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An extension the paper leaves implicit: NSR could be used in reverse as a check on the structured-shift regime itself—if no feature scores near zero, the practitioner has evidence either that the regime is violated or that the model ignores causal features.
The weight-cancellation property suggests the method may keep working when only attribution estimates (e.g., permutation-based or additive attribution values) are available instead of model weights; the paper experiments this way, but the exact identification proof is linear, so the nonlinear guarantee is an extrapolation.
The scalar-uniform-shift assumption is the least realistic part of the model; replacing per-feature replacement with subspace-level interventions along environment-centroid principal components would be a natural way to handle feature-specific batch effects, but the paper does not prove guarantees for that relaxation.
A concrete testable extension: run NSR on the same model under two different reference environments; if rankings flip, the choice of reference is doing work, and the paper's recommended maximum-over-references variant would be the safer default.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know. First, the core idea is new and the theory is mostly solid: NSR, the squared coefficient of variation of per-environment sensitivity, is weight-invariant, and the proof that it is exactly zero for causal features and positive for spurious features under the linear SCM is clean. The failure-mode analysis—weak-shift collapse at O(ε^4), degenerate geometry, proxy attenuation at O((1−α)^4)—is a real contribution, and the incomparability result with IRM/ICP is genuinely useful. Synthetic validation matches the theory, including the phase transition in K and ε, and the model-agnostic ranking across five families is a nice check.

Second, the paper overclaims. Exact identification requires equation (1): causal feature marginals P(X_C) are fixed across environments. That is stronger than the standard invariance of P(Y|X_C), and the paper admits it. But it never quantifies how much shift in P(X_C) breaks the result. That would be acceptable if the real-world test stayed in the regime. On bike-sharing, the environments are month-quintiles; the paper labels temp causal, yet reports temp's environment means shifting by 6.2–9.4 °C, and the stability check only covers hr and workingday. So the headline real-world result may violate the paper's own central assumption. This is the main soft spot.

Other issues are minor but real. The nonlinear extension is empirical only; the theorem is for linear predictors. Real-world AUROC is modest (around 0.66) and depends on tuning a regularization target toward within-environment R²≈0.75, a free parameter whose choice is not justified. No code or data are provided, despite a checklist statement that code will come with the camera-ready—for a methods paper aimed at practitioners, that hurts reproducibility. There are also internal labeling errors (repeated RQ1, a broken cross-reference to Theorem 4.3–??), a sign of a rushed final pass.

Who should read it: researchers in invariant learning, model auditing, and multi-site/batch-effect settings. The paper gives a clear map of when this post-hoc sensitivity-variance test works and when it fails. It deserves peer review because the question is important and the theory is coherent; but a good referee will demand a revision that aligns the abstract with the fixed-marginal assumption, tests that assumption's boundary directly (including on the bike-sharing data), and ships code. I would cite the theorem if I wrote on this topic, though I would not rely on the real-world numbers.

Recommendation: send to peer review with an expectation of major revision.

Referee Report

3 major / 4 minor

Summary. The paper introduces the Normalised Sensitivity Ratio (NSR), a post-hoc diagnostic that uses multi-environment data to identify which features of an already-trained model are causally vs. spuriously relied upon. Under a linear SCM in which causal feature marginals are fixed across environments while spurious feature means shift by a scalar β_e (Definition 1, eqs. (1)–(3)), NSR is defined as the squared coefficient of variation of per-environment model sensitivity (eq. (5)). Theorem 4.1 claims exact identification: NSR_j = 0 iff j is causal and NSR_j > 0 iff j is spurious, independent of the learned weight ŵ_j. The paper also provides finite-sample rates (Theorem 4.2), a failure-mode characterisation for weak shifts, degenerate geometry, and proxy features (Theorem 4.3), and a formal comparison with IRM/ICP showing incomparable detection conditions (Proposition 1). Synthetic experiments confirm the theoretical operating regime, and real-world experiments on bike-sharing and wine quality demonstrate the method's practical use. The paper is explicit that the regime requires stable causal marginals and that violations can cause false positives.

Significance. If the identification claim holds, NSR is a valuable addition to the post-hoc interpretability toolbox: it requires no retraining, no gradient access, and no causal graph, and it provably separates causal from spurious features under a structured-shift regime. The proof of Theorem 4.1 is clean and self-contained: the cancellation of ŵ_j in the CV² ratio is explicit (Appendix A.2, eq. (7)), and the folded-normal monotonicity argument for spurious features is correct. The synthetic validation is thorough, with AUROC = 1.000 under the stated regime and sharp confirmation of the predicted failure modes. The explicit comparison with IRM and ICP is a useful conceptual contribution. The main weaknesses are that the exact guarantee is narrower than the abstract suggests (only linear predictors, fixed causal marginals, scalar shifts), and the real-data validation does not cleanly satisfy the stated assumptions. Nevertheless, the core theorem and its proof are sound, and the paper is honest about the principal limitation.

major comments (3)

[Section 5.5 / Table 2] The bike-sharing experiment is presented as satisfying marginal causal stability, but the text states that the causal feature `temp` shifts by 6.2–9.4°C across environments while `hr` and `workingday` are stable. Under eq. (1), X_C is fixed across environments; a causal feature with a shifting mean directly violates the assumption needed for Theorem 4.1(i). The 'stability check' only verifies `hr` and `workingday`, not `temp`, which is listed as a ground-truth causal feature. Since the experiment is the paper's main real-world evidence, the authors should either (a) correct the assertion that the checks are satisfied, (b) report the NSR values for the shifting causal feature and discuss how the partial recovery nonetheless occurs, or (c) reframe the result as a robustness test under assumption violation rather than a clean validation.
[Abstract / Theorem 4.1 / Section 5.3] The abstract claims a 'post-hoc, model-agnostic diagnostic' with exact identification, but Theorem 4.1 is stated and proved only for linear predictors. The extension to nonlinear models via SHAP/permutation importance is empirical only and lacks a theoretical justification that the CV² of the attribution equals the CV² of the marginal-replacement sensitivity signal. Section 5.3 reports consistent rankings across model families, but this does not substitute for a formal statement of the conditions under which the identification guarantee carries over. The paper should either soften the model-agnostic claim in the abstract or provide a theorem for a class of nonlinear models.
[Theorem 4.2 and Appendix A.5] The finite-sample result Theorem 4.2(ii) states that under H0, empirical NSR = O_p(n^{-1}). This rate is used to justify a permutation test with a growing signal-to-noise ratio. However, the appendix does not provide a calibration study demonstrating that the permutation null indeed controls Type I error at finite n, or a power analysis under H1. The real-data FDR claim in Section 5.5 ('correctly classify 10 of 12 features (4 false negatives, 0 false positives)') is arithmetically inconsistent: 10 correct with 0 false positives implies 2 false negatives, not 4. This undermines the interpretability of the reported real-world performance and should be checked.

minor comments (4)

[Section 5, intro paragraph] The research-question labels are inconsistent: 'RQ1' is used for three different questions (operating conditions, IRM comparison, real data), and the appendix refers to 'RQ1' for model-agnostic and proxy experiments that are distinct from the main RQ1. Please renumber to RQ1–RQ5 consistently.
[Introduction, contribution (ii)] The sentence '(Theorems 4.3–??)' contains a missing cross-reference; the proxy-attenuation result is Theorem 4.3(iii), not a separate numbered theorem.
[Section 3, 'Scope and assumptions'] The check (2) about leading PCs of environment centroids is not formally connected to the scalar-shift assumption. Clarify whether this is a heuristic or a condition whose violation provably breaks Theorem 4.1.
[Table 2] Table 2 is labeled 'RQ1' but is the RQ5 real-world validation. Correct the caption.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: NSR's identification is derived algebraically from explicitly stated SCM assumptions; the only self-citation is contextual and not load-bearing.

full rationale

Theorem 4.1 is obtained by a genuine algebraic derivation, not by renaming inputs or fitting then predicting. CSR factorizes as |ŵ_j|·δ_j (Proposition 2), and the CV^2 normalization cancels ŵ_j^2 (eq. 7). Equation (1) makes δ_j constant for causal features; equation (2) plus Assumption A2 makes δ_j non-constant for spurious features through the strict monotonicity of the folded-normal mean (Proposition 3(ii)). Thus the identification signal is shown to live in the environment geometry and in the stated SCM assumptions, rather than in fitted parameters or in the authors' earlier results. The synthetic validation uses independently generated ground truth, and the claimed rates (O_p(n^{-1}) under the null, O_p(n^{-1/2}) under the alternative, O(ε^4) weak-shift collapse, O((1-α)^4) proxy attenuation) follow from Taylor/delta-method arguments and are checked empirically. The only self-citation, Corcoll Andreu et al. 2024 with co-author A. Vlontzos, is cited in Related Work as a complementary intra-variable contrastive representation setting and is not load-bearing for the NSR derivation. The paper is explicit that marginal causal stability—P(X_C) fixed across environments—is stronger than standard P(Y|X_C) invariance and admits that NSR will misclassify causal features if causal marginals shift; this is an assumption boundary and an empirical validity concern (including the bike-sharing temp shift), not an equation-level circularity. The practical recommendation to check stability of 'putative causal features' is a scope-verification caveat for deployment, not a step in the derivation. No circular reduction of the enumerated kinds was found.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

No new physical or ontological entities are introduced; the structured-shift regime is an assumption on environment distributions, not an invented entity. Free parameters are experimental hyperparameters rather than derivation inputs. The highest-cost assumptions are fixed causal marginals and uniform scalar spurious shift, both explicit in Definition 1.

free parameters (2)

replacement draws m = 50
Chosen by hand for all experiments (Section 3, Practical computation). Affects Monte Carlo noise of CSR estimates but not the population identification.
SHAP-NSR target in-environment R2 = 0.75
Selected adaptively in Section 5.5 to make SHAP-NSR work on bike-sharing; without it Precision@7 drops from 0.75 to 0.66 and AUROC from 0.66 to 0.58. This is a data-dependent hyperparameter.

assumptions (6)

domain assumption Causal features have identical marginals across environments: X_C ~ N(mu_C, Sigma_C) for all e (Eq. 1).
Load-bearing: if P(X_C) shifts, delta_j is not constant and NSR misclassifies causal features as spurious (stated in Section 3, Scope and assumptions).
domain assumption Causal mechanism is stable: P(Y|X_C) is fixed across environments (Assumption A1).
Standard causal invariance shared with IRM/ICP; needed so the outcome mechanism does not vary.
domain assumption Non-constant absolute shifts from reference: {|beta_e - beta_ref| : e != ref} is not constant (Assumption A2).
Necessary for spurious delta_j to have positive variance; if violated, NSR=0 for all features (Theorem 4.3(ii)).
domain assumption Faithfulness and nondegeneracy: ŷ_j != 0 and Sigma_{C,jj} > 0 for j in C (Assumption A3).
Zero-weight causal features are undiagnosable by any sensitivity method; degenerate causal marginals make delta_j vanish.
domain assumption Uniform scalar spurious shift: X_S = Gamma X_C + beta_e 1 + xi (Eq. 2).
All spurious features share the same beta_e; feature-specific shifts are not covered by Theorem 4.1 and are deferred to a subspace extension.
ad hoc to paper Nonlinear plug-ins (SHAP/permutation importance) estimate the same cross-environment sensitivity signal.
Used for nonlinear models and real data (Sections 5.3–5.5) without a theorem; only empirical evidence is provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of From Training to Deployment: Post-Hoc Causal Feature Identification via Sensitivity Ratios." pith.science (2026). https://pith.science/paper/UQJ4RJ2K

@misc{pith2026260725546,
  author       = {Pith},
  title        = {Pith review of: From Training to Deployment: Post-Hoc Causal Feature Identification via Sensitivity Ratios},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UQJ4RJ2K}},
  note         = {Machine review of arXiv:2607.25546}
}

abstract

Given a model that is already trained, which features does it rely on causally versus spuriously? Existing methods require access to the training procedure and cannot answer this post-hoc. We introduce the \textbf{Normalised Sensitivity Ratio~(NSR)}, a post-hoc, model-agnostic diagnostic for this question under a structured-shift regime: environments differ primarily in the mean of spurious features while the causal mechanism and causal marginals remain stable, as in multi-site clinical data or multi-batch genomics. Within this regime, causal features induce constant model sensitivity across environments while spurious features track shift. NSR formalises this as the squared coefficient of variation of per-environment sensitivity. Under a linear structural causal model (SCM) with $K\ge3$ non-degenerate environments, NSR achieves exact identification (Theorem~1). We fully characterise failure: weak shifts ($O(\varepsilon^4)$ collapse), degenerate geometry, and proxy attenuation ($O((1-\alpha)^4)$), giving practitioners quantitative criteria for assessing whether the regime holds. Finite-sample rates are $O_p(n^{-1})$ under the null and $O_p(n^{-1/2})$ under the alternative. Experiments confirm all theoretical predictions on synthetic data (area under the ROC curve [AUROC] $= 1.000$ under conditions satisfying the regime), show consistent rankings across five model families (Kendall $\tau\ge0.529$), and recover six of eight causal features on bike-sharing data (Precision@7 $= 0.75$) without modifying any trained model.

Figures

Figures reproduced from arXiv: 2607.25546 by the authors.

**Figure 1.** RQ1: NSR AUROC as a function of K and ε (30 seeds per cell). K = 2: identically zero (Theorem 4.3(ii)). K ≥ 5, ε ≥ 1: AUROC = 1.000 with zero variance. K acts as a binary threshold, not a continuous dial. The theory makes four precise predictions: NSR is identically zero for K = 2 regardless of shift magnitude; it achieves reliable separation for K ≥ 3 with sufficient shift; it collapses as O(ε 4 ) near the noise f… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references

[1]

Advances in Neural Information Processing Systems (

Garg, Shivam and Tsipras, Dimitris and Liang, Percy and Valiant, Gregory , title =. Advances in Neural Information Processing Systems (. 2022 , url =

2022
[2]

International Conference on Learning Representations (

Xie, Sang Michael and Raghunathan, Aditi and Liang, Percy and Ma, Tengyu , title =. International Conference on Learning Representations (. 2022 , url =

2022
[3]

Transformers Can Do

M. Transformers Can Do. International Conference on Learning Representations (. 2022 , url =

2022
[4]

What Learning Algorithm Is In-Context Learning?

Aky. What Learning Algorithm Is In-Context Learning?. The Eleventh International Conference on Learning Representations (. 2023 , url =

2023
[5]

Transformers Learn In-Context by Gradient Descent , booktitle =

Von Oswald, Johannes and Niklasson, Eyvind and Randazzo, Ettore and Sacramento, Jo. Transformers Learn In-Context by Gradient Descent , booktitle =. 2023 , url =

2023
[6]

The Eleventh International Conference on Learning Representations (

Hollmann, Noah and M. The Eleventh International Conference on Learning Representations (. 2023 , url =

2023
[7]

Invariant Risk Minimization , journal =

Arjovsky, Mart. Invariant Risk Minimization , journal =. 2019 , url =

2019
[8]

Causal Inference by Using Invariant Prediction: Identification and Confidence Intervals , journal =

Peters, Jonas and B. Causal Inference by Using Invariant Prediction: Identification and Confidence Intervals , journal =. 2016 , doi =

2016

Show all 18 references

[9]

and Liang, Percy , title =

Sagawa, Shiori and Koh, Pang Wei and Hashimoto, Tatsunori B. and Liang, Percy , title =. International Conference on Learning Representations (. 2020 , note =

2020
[10]

Out-of-Distribution Generalization via Risk Extrapolation (

Krueger, David and Caballero, Ethan and Jacobsen, J. Out-of-Distribution Generalization via Risk Extrapolation (. Proceedings of the 38th International Conference on Machine Learning (. 2021 , note =

2021
[11]

arXiv preprint , year =

Yan, Shuhan and Song, Huan and Li, Nan and Zou, Linchao and Ying, Yi , title =. arXiv preprint , year =
[12]

, title =

Kaushik, Divyansh and Hovy, Eduard and Lipton, Zachary C. , title =. International Conference on Learning Representations (. 2020 , url =

2020
[13]

Logic, Language, Information, and Computation , pages =

Lu, Kaiji and Mardziel, Piotr and Wu, Fan and Agrawal, Preetam and Datta, Anupam , title =. Logic, Language, Information, and Computation , pages =. 2020 , publisher =

2020
[14]

Luecken, Malte D. and B. Benchmarking atlas-level data integration in single-cell genomics , journal =. 2022 , doi =

2022
[15]

and Horlbeck, Max A

Norman, Thomas M. and Horlbeck, Max A. and Replogle, Joseph M. and Ge, Alex Y. and Xu, Albert and Jost, Marco and Gilbert, Luke A. and Weissman, Jonathan S. , title =. Science , volume =. 2019 , doi =

2019
[16]

, title =

Corcoll Andreu, Oriol and Vlontzos, Athanasios and O'Riordan, Michael and Gilligan-Lee, Ciaran M. , title =. arXiv preprint , volume =. 2024 , url =

2024
[17]

Event labeling combining ensemble detectors and background knowledge , howpublished =

Fanaee-T, Hadi and Gama, Jo\. Event labeling combining ensemble detectors and background knowledge , howpublished =. 2013 , note =

2013
[18]

Modeling wine preferences by data mining from physicochemical properties , howpublished =

Cortez, Paulo and Cerdeira, Ant\'. Modeling wine preferences by data mining from physicochemical properties , howpublished =. 2009 , note =

2009

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.

[1] [1]

Advances in Neural Information Processing Systems (

Garg, Shivam and Tsipras, Dimitris and Liang, Percy and Valiant, Gregory , title =. Advances in Neural Information Processing Systems (. 2022 , url =

2022

[2] [2]

International Conference on Learning Representations (

Xie, Sang Michael and Raghunathan, Aditi and Liang, Percy and Ma, Tengyu , title =. International Conference on Learning Representations (. 2022 , url =

2022

[3] [3]

Transformers Can Do

M. Transformers Can Do. International Conference on Learning Representations (. 2022 , url =

2022

[4] [4]

What Learning Algorithm Is In-Context Learning?

Aky. What Learning Algorithm Is In-Context Learning?. The Eleventh International Conference on Learning Representations (. 2023 , url =

2023

[5] [5]

Transformers Learn In-Context by Gradient Descent , booktitle =

Von Oswald, Johannes and Niklasson, Eyvind and Randazzo, Ettore and Sacramento, Jo. Transformers Learn In-Context by Gradient Descent , booktitle =. 2023 , url =

2023

[6] [6]

The Eleventh International Conference on Learning Representations (

Hollmann, Noah and M. The Eleventh International Conference on Learning Representations (. 2023 , url =

2023

[7] [7]

Invariant Risk Minimization , journal =

Arjovsky, Mart. Invariant Risk Minimization , journal =. 2019 , url =

2019

[8] [8]

Causal Inference by Using Invariant Prediction: Identification and Confidence Intervals , journal =

Peters, Jonas and B. Causal Inference by Using Invariant Prediction: Identification and Confidence Intervals , journal =. 2016 , doi =

2016