IMEX Interaction-Based Model Explanation

Emiliano Massi

REVIEW 3 major objections 5 minor 18 references

IMEX Interaction-Based Model Explanation

T0 review · 3 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read IMEX's PCS score re-discovers the true explanatory features in three synthetic datasets and behaves more consistently than INVASE under non-linear, conditional, and multicollinear structures.

desk verdict The core operation is never defined, and the main metric is standard leave-one-out importance; the current experiments cannot support the paper's claim. read the letter →

arxiv 2607.14096 v1 pith:JFFCKK6X submitted 2026-04-16 cs.AI

Emiliano Massi This is my paper · ORCID

classification cs.AI

keywords IMEXPCSPCIfeatureimportancepost-hocexplainabilityINVASEinteractioneffectssyntheticgroundtruth

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes IMEX, a post-hoc explanation framework built on two metrics: PCS, which ranks each feature by the normalized absolute change in prediction when that feature is removed, and PCI, which quantifies non-additive interaction effects. Its central empirical claim is that PCS recovers the known explanatory structure in three synthetic datasets whose dependency rules are explicitly controlled, and that it does so more consistently than INVASE in settings with non-linear, conditional, and multicollinear relationships. The paper sees this as evidence that removal-based importance signals can serve as a reliable, model-agnostic basis for feature-level interpretability. PCI is introduced with a higher-order generalization, but the paper explicitly leaves its empirical validation for future work. The reader should care because a cheap, well-behaved feature-importance measure would strengthen trust in black-box model predictions.

What carries the argument

The load-bearing object is the PCS score defined in Equation (2): for a target k and feature i, Δy_i,k is the absolute difference between the model's prediction using all features and its prediction after feature i is removed, and PCS_i is that difference normalized by the sum over all features. The companion PCI score for a pair (i,j) is the absolute difference between the joint-removal variation and the sum of the two individual variations, normalized by the total interaction mass in the neighborhood of i and j. The evaluation pipeline converts continuous scores into binary importance tables by thresholding at 1.2 times the uniform baseline (1/N for PCS), then computes precision, recall, a

What would settle it

On the same three synthetic datasets, refit the XGBoost model once for each feature removal (i.e., train a model without feature i) and recompute PCS from those retrained predictions; if the feature rankings shift materially or no longer match the ground truth, then PCS is not a single, well-defined quantity. Alternatively, feed PCS a model trained on pure noise features: if the binarized importance table still selects features, the score is responding to artifacts rather than real dependence.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that PCS — defined as the ratio of the absolute prediction shift from removing feature i to the total shift from removing all features — produces binary importance tables that align with the ground-truth structural maps of the three synthetic datasets. Across the tested targets, PCS identifies the strong linear drivers and also flags weaker but genuine connections that INVASE misses, particularly under conditional and multicollinear dependencies. Because PCS and INVASE both output one score per feature, the paper argues the comparison is methodologically fair, whereas SHAP values are not directly comparable since they decompose a prediction

Load-bearing premise

The method assumes that 'removing a feature from the predictive process' is a well-defined operation, but the paper never specifies whether removal means retraining the model, masking the feature value, or imputing a baseline, and every PCS score depends on that unspecified choice.

Editorial extensions

If this is right

If PCS is as reliable as the experiments suggest, any trained model can be audited for feature relevance with a simple removal-and-measure procedure, no retraining or gradient access required.
PCS's apparent sensitivity on conditional and multicollinear targets implies that uniform-baseline normalization plus a modest threshold can extract genuine signals that more complex instance-wise selectors miss.
The PCI construction offers a path to interaction discovery: pairs (or larger subsets) whose joint removal effect exceeds the sum of individual effects are flagged as non-additive drivers.
The higher-order PCI generalization gives a principled, if computationally heavy, way to search for collective feature effects consistent with latent mechanisms.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The unspecified removal operation leaves PCS underdetermined: retraining the model without the feature, masking it with a baseline value, or imputing a marginal mean will generally yield different Δy values, so the paper's results are conditioned on an unstated implementation choice.
PCI's neighborhood normalization depends on the set of subsets sharing at least one feature with S; as N grows, the denominator aggregates many weak terms, which will dilute the interaction signal — a property the paper does not investigate.
A direct comparison between PCS under a mean-imputation removal and SHAP values on the same synthetic datasets could reveal that the two are numerically close, undermining the paper's claim that they are not comparable objects.
The 20 percent threshold is an ad hoc but tunable knob; checking PCS performance across a range of thresholds on the same synthetic data would show whether the reported advantage over INVASE is robust or an artifact of that specific cut.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

This one is a modest XAI paper that I don't think is ready for prime time, though the author is not sloppy in intent. The central empirical claim — that PCS, a normalized leave-one-feature-out predictive variation, recovers ground-truth structure on synthetic data — is plausible, but the paper as written does not let you check it.

What's actually new? Very little. Equation (2) defines PCS as the absolute change in prediction when a feature is removed, normalized by the sum over features. That's essentially drop-column / leave-one-out importance, a standard approach with a substantial literature (e.g., LOCO, permutation-based measures). The paper doesn't cite or compare against any of those baselines, only against INVASE. The interaction metric PCI is new, but the paper explicitly leaves it unvalidated, so it cannot carry any weight.

What the paper does well: it is unusually honest about what it does and doesn't validate. Section 4 says the data-generation functions are omitted 'for brevity,' and Section 3.2 says PCI validation is future work. The comparison protocol — binarize scores using a threshold, compare to ground truth with precision/recall/F1 — is clearly described, and the threshold is justified on the uniform-importance baseline. I give credit for that clarity.

The problems, in order of severity:

1. The feature removal operation is never defined. Section 3.1 says y_{−i,k} is the prediction 'after removing feature i,' but does not say whether that means retraining XGBoost without the feature, imputing a baseline value, or marginalizing over the feature. These give different Δy, different PCS, and different F1. This is a load-bearing ambiguity: the claim 'PCS aligns with ground truth' is not falsifiable until the operation is specified.

2. The evaluation is underpowered. 100 observations, no error bars, no confidence intervals. The 1.2×baseline threshold is arbitrary; the author says it is 'not derived from optimization' and sensitivity analysis is future work. Fine, but that means the reported F1 numbers are not robust evidence.

3. No code or data released, and the data-generation functions are omitted. The ground truth cannot be audited. That alone would stop me from relying on the results.

4. The novelty gap. Even if the removal ambiguity were fixed, the result would be 'a normalized LOCO importance matches known structure on three synthetic datasets.' That's not a contribution unless it is benchmarked against existing feature importance methods and shows an advantage.

So my bottom line: the author is thinking coherently, and the paper is readable, but the current evidence is too incomplete to warrant peer review as a substantive contribution. I would desk-reject with an invitation to resubmit after specifying the removal mechanism, adding full experimental details and baselines, and validating PCI. I wouldn't cite it yet, but I might point someone to it as an example of why removal-based explanations need precise operational definitions.

Hope that helps.

Referee Report

3 major / 5 minor

Summary. The paper introduces IMEX, a post-hoc explanation framework with two metrics: PCS (per-feature normalized absolute predictive variation after feature removal) and PCI (a normalized pairwise non-additive effect, with a higher-order generalization). The manuscript validates only PCS, comparing it against INVASE on three synthetic datasets with known ground-truth dependency structures. Importance scores are binarized with a threshold of 1.20 times the uniform baseline and evaluated by precision, recall, and F1 against the ground truth. The main claim, stated in §5.3, is that PCS provides a feature-importance signal aligned with the known ground truth and shows more consistent behavior than INVASE under non-linear, conditional, and multicollinear structures. PCI is introduced as a methodological extension but is not empirically benchmarked.

Significance. If the empirical claims are made reproducible, PCS would be a simple perturbation-based feature-importance baseline with a clear comparison protocol. The synthetic-ground-truth design is a strength: PCS is defined independently of the ground truth, no parameters are fitted to the F1 metrics, and the comparison with INVASE operates in the same feature space. However, the current manuscript does not provide code, data-generation functions, or a complete specification of the feature-removal operation, so the numerical results cannot be verified. The binarization threshold is an uncalibrated free parameter, and no confidence intervals or aggregate statistics are reported. The honest labeling of PCI as future work is appropriate, but it means the paper's actual contribution is limited to the PCS experiments.

major comments (3)

[§3.1, Eq. (1)] The definition of y_{−i,k} as 'the k-th target feature prediction obtained after removing feature i' never specifies the removal operation. It is not stated whether the model is retrained without the feature, whether the feature value is replaced by a baseline (zero, mean, or median), or whether the feature is marginalized out. Because Eq. (1) and Eq. (2) are built on Δy_{i,k}, every PCS score and every binarized importance table in Tables 8–19 depends on this choice. For a tree ensemble such as XGBoost, 'removing' a feature is not a native operation; different imputation choices can change PCS rankings and therefore the F1 scores against the ground truth. As written, the central claim in §5.3 that PCS is 'aligned with the known ground truth' and 'shows more consistent behavior than INVASE' is not falsifiable. The authors should state the exact removal mechanism, justify it, and report s
[§4 and §5.1.2] The synthetic data-generation functions are 'omitted for brevity,' no code or data is released, and the evaluation is performed on a representative subset of 100 observations. The ground truth is claimed to be deterministic and structural, but without the explicit generation rules or a complete connection map (Tables 5 and 17 are empty as rendered), the reader cannot reproduce the target values or the ground-truth tables. Reproducibility of Tables 8–19 therefore depends entirely on unstated details. At minimum, the full generation rules, dataset release or code, and results on the full datasets (or a fixed public seed) are needed.
[§5.1.3, Tables 8–19] The binarization threshold is set at 1.20 × baseline and is acknowledged to be tunable, yet no sensitivity analysis is reported. Since all downstream precision/recall/F1 values depend on this threshold, the claim that PCS is 'consistent' with ground truth is not robustly supported. Additionally, Tables 8–19 report per-feature F1 values without aggregation or confidence intervals; the summary in §5.3 is a qualitative synthesis. I request a threshold sweep (e.g., 1.0×–1.5× baseline) that reports aggregate P/R/F1 and a quantitative comparison with INVASE (e.g., mean F1 across targets or paired tests).

minor comments (5)

[§3.2] Notation is inconsistent: Eq. (3) uses Δy_{i,j,k} with commas, Eq. (4) uses Δy_{i,k}, and Eq. (6) uses Δy_{S,k}. Please unify the notation.
[Tables 5 and 17] The connection-map tables render only column headers; the arrows or graphical elements showing dependencies are missing. Without them, the ground-truth structure for each dataset is not fully specified.
[Table 1 and table headers] Typos include 'A verage spending' in Table 1 and 'FEA TURES' / 'INV ASE' in several tables.
[§5.1.2] The ground-truth excerpt repeats the same row ten times, which does not illustrate any variation across observations. A compact aggregated ground-truth vector per target would be clearer and more informative.
[§3.2, Eq. (4)] Taking absolute values before forming the interaction term makes PCI sensitive to sign cancellations. If this is intended, one sentence of justification would be helpful; otherwise, the interaction definition may need revision.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PCS is defined independently of the ground truth and validated against synthetic structural targets; no fitted parameter is relabeled as a prediction.

full rationale

The derivation chain is self-contained relative to its validation target. Equations (1)–(2) define PCS as the normalized absolute predictive change under feature removal; this quantity is not defined in terms of the ground-truth connection map, and no parameter is fitted to the F1 metrics. The ground truth is constructed from the synthetic data-generation rules (Section 5.1.2), and the comparison with INVASE uses a fixed 1.20·baseline binarization threshold that the paper explicitly states is not derived from an optimization procedure (Section 5.1.3). PCI is introduced but explicitly not benchmarked (Sections 3.2, 4, 6), so it cannot feed back into the empirical claim. The paper's acknowledged limitations — omitted data-generation functional forms (Section 4), the unspecified feature-removal mechanism in Equation (1), and deferred threshold-sensitivity analysis (Section 5.1.3) — are reproducibility and validity concerns, not circular reductions. There are no self-citations, no imported uniqueness theorems, and no ansatz smuggled through citation. Therefore no significant circularity is present.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The method introduces no new physical or latent entities. The main assumptions are the unspecified feature-removal mechanism and the correctness of the omitted synthetic data-generation rules. The threshold and subset size are free design parameters. The interaction definitions are ad hoc but not circular with respect to the ground truth.

free parameters (2)

binarization threshold multiplier = 1.20
Equation (10) defines threshold = 1.20 × (baseline = 1/N). Chosen by a consistency argument, not optimized, and no sensitivity analysis is provided.
evaluation subset size = 100 observations
Section 5.1.2 states the comparison was performed on a representative subset of 100 observations, claimed to preserve structural properties without quantitative justification.

assumptions (5)

domain assumption Removing a feature from the predictive process produces a meaningful counterfactual importance measure.
PCS and PCI depend on y_{-i,k} (Section 3.1), but the paper never specifies how feature removal is implemented, which determines all Δ values.
domain assumption XGBoost predictions adequately capture the true data-generating relationships.
Section 4 states explanations are meaningful only if the model is predictive, yet test R² ranges from ~0.56 to 0.999, including weak targets, and no per-target checks are used in the explanation comparison.
domain assumption The binary structural ground truth correctly encodes feature relevance for the target.
Section 5.1.2 builds ground truth from data-generation rules whose explicit functional forms are omitted, so a reader cannot verify that the ground truth matches the intended dependency structure.
ad hoc to paper Absolute predictive variation is an appropriate measure of feature contribution.
Equation (1) defines contribution as absolute change in prediction; this ignores the direction and scale of effects, and is a modeling choice not derived from an axiomatic framework.
ad hoc to paper Interaction strength is the absolute difference between joint variation and the sum of individual absolute variations.
Equations (4) and (7) define I_ij and I_S using absolute values in a way that is not connected to standard ANOVA/SHAP interaction definitions; this is an ad hoc construction.

how reviews work

0 comments

Cite this review

Pith. "Pith review of IMEX Interaction-Based Model Explanation." pith.science (2026). https://pith.science/paper/JFFCKK6X

@misc{pith2026260714096,
  author       = {Pith},
  title        = {Pith review of: IMEX Interaction-Based Model Explanation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JFFCKK6X}},
  note         = {Machine review of arXiv:2607.14096}
}

read the original abstract

In predictive modeling, the ability to explain why a model produces a given target prediction has become increasingly important [5, 10]. Black-box models do not provide a transparent description of the internal mechanisms that generate the prediction, making even accurate predictions difficult to interpret and validate. In critical contexts, predictive accuracy alone is not a sufficient validation metric if the reasons underlying model decisions remain unexplained. The IMEX (Interaction-Based Model Explanation) approach represents a methodological direction within explainable predictive modeling. IMEX is designed to identify which variables contribute most to the target prediction and which interactions among variables are significant in determining the target. The method does not impose limitations on higher-order interaction analysis, allowing the investigation of feature subsets with cardinality greater than two. Beyond the identification of feature importance, IMEX enables the exploration of interaction patterns that may be consistent with latent mechanisms influencing the outcome. Through the application of the IMEX algorithm, it is possible to construct an interpretability map of the predictions. The IMEX framework is built on two complementary metrics: Static Correlation Power (PCS), which quantifies the contribution of individual features, and Interaction Correlation Power (PCI), which captures non-additive effects among features. In the present work, the PCS component is experimentally validated through a comparison with INVASE [18] on three synthetic datasets with known structures. The results indicate that IMEX can recover relevant feature-level structures in the presence of non-linear, conditional, and multicollinear relationships between input features and prediction targets.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 2 linked inside Pith

[1]

Baehrens, D., Schroeter, T., Harmeling, S., Kawanabe, M., Hansen, K., and Müller, K.-R. (2010). How to explain individual classification decisions.Journal of Machine Learning Research

2010
[2]

Chen, T., and Guestrin, C. (2016). XGBoost: A scalable tree boosting system. InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data 19 IMEX Interaction-Based Model Explanation Mining

2016
[3]

Covert, I., Lundberg, S., and Lee, S.-I. (2020). Understanding global feature contributions with additive importance measures. InAdvances in Neural Information Processing Systems

2020
[4]

Datta, A., Sen, S., and Zick, Y. (2016). Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. InIEEE Symposium on Security and Privacy

2016
[5]

Doshi-Velez, F., and Kim, B. (2017). Towards a rigorous science of interpretable machine learning.arXiv preprint arXiv:1702.08608

arXiv 2017
[6]

Fisher, A., Rudin, C., and Dominici, F. (2019). All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously.Journal of Machine Learning Research

2019
[7]

Hooker, G. (2007). Generalized functional ANOVA diagnostics for high dimensional functions of dependent variables.Journal of Computational and Graphical Statistics

2007
[8]

Hooker, G., and Mentch, L. (2019). Please stop permuting features: An explanation and alternatives.arXiv preprint arXiv:1905.03151

arXiv 2019

Show all 18 references

[9]

D., Sturmfels, P., and Lee, S.-I

Janizek, J. D., Sturmfels, P., and Lee, S.-I. (2021). Explaining explanations: Axiomatic feature interactions for deep networks.Journal of Machine Learning Research

2021
[10]

Lipton, Z. C. (2018). The mythos of model interpretability.Queue, 16(3)

2018
[11]

M., and Lee, S.-I

Lundberg, S. M., and Lee, S.-I. (2017). A unified approach to interpreting model predictions. InAdvances in Neural Information Processing Systems

2017
[12]

M., Erion, G., Chen, H., DeGrave, A., Prutkin, J

Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., and Lee, S.-I. (2020). From local explanations to global understanding with explainable AI for trees.Nature Machine Intelligence

2020
[13]

(2022).Interpretable Machine Learning

Molnar, C. (2022).Interpretable Machine Learning. Springer

2022
[14]

Owen, A. B. (2014). Sobol’ indices and Shapley value.SIAM/ASA Journal on Uncertainty Quantification

2014
[15]

Why should I trust you?

Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). “Why should I trust you?” Explaining the predictions of any classifier. InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

2016
[16]

Sobol, I. M. (2001). Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates.Mathematics and Computers in Simulation

2001
[17]

Sundararajan, M., Taly, A., and Yan, Q. (2017). Axiomatic attribution for deep networks. InProceedings of the 34th International Conference on Machine Learning

2017
[18]

Yoon, J., Jordon, J., and van der Schaar, M. (2019). INVASE: Instance-wise variable selection using neural networks. InInternational Conference on Learning Representations. 20

2019

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.

[1] [1]

Baehrens, D., Schroeter, T., Harmeling, S., Kawanabe, M., Hansen, K., and Müller, K.-R. (2010). How to explain individual classification decisions.Journal of Machine Learning Research

2010

[2] [2]

Chen, T., and Guestrin, C. (2016). XGBoost: A scalable tree boosting system. InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data 19 IMEX Interaction-Based Model Explanation Mining

2016

[3] [3]

Covert, I., Lundberg, S., and Lee, S.-I. (2020). Understanding global feature contributions with additive importance measures. InAdvances in Neural Information Processing Systems

2020

[4] [4]

Datta, A., Sen, S., and Zick, Y. (2016). Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. InIEEE Symposium on Security and Privacy

2016

[5] [5]

Doshi-Velez, F., and Kim, B. (2017). Towards a rigorous science of interpretable machine learning.arXiv preprint arXiv:1702.08608

arXiv 2017

[6] [6]

Fisher, A., Rudin, C., and Dominici, F. (2019). All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously.Journal of Machine Learning Research

2019

[7] [7]

Hooker, G. (2007). Generalized functional ANOVA diagnostics for high dimensional functions of dependent variables.Journal of Computational and Graphical Statistics

2007

[8] [8]

Hooker, G., and Mentch, L. (2019). Please stop permuting features: An explanation and alternatives.arXiv preprint arXiv:1905.03151

arXiv 2019