Analyzing Shapley Additive Explanations to Understand Anomaly Detection Algorithm Behaviors and Their Complementarity

Benoit Gaudou; Jordan Levy; Moncef Garouani; Nicolas Verstaevel; Paul Saves

REVIEW 2 major objections 1 minor 23 references

Analyzing Shapley Additive Explanations to Understand Anomaly Detection Algorithm Behaviors and Their Complementarity

T0 review · 2 major / 1 minor · reviewed 2026-05-16 · grok-4.3

Challenge this review Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SHAP attribution similarity identifies complementary anomaly detectors, offering a selection criterion distinct from raw output scores for building effective ensembles.

desk verdict SHAP profiles give a workable way to pick complementary anomaly detectors, but background choice in explanations could affect the results. read the letter →

arxiv 2602.00208 v3 submitted 2026-01-30 cs.LG cs.AIcs.IRmath.STstat.MLstat.TH

Jordan Levy , Paul Saves , Moncef Garouani , Nicolas Verstaevel , Benoit Gaudou This is my paper · ORCID

classification cs.LGcs.AIcs.IRmath.STstat.MLstat.TH

keywords anomalydetectionSHAPexplanationsensemblemethodsmodelcomplementarityfeatureattributionunsupervisedlearningdetectordiversityexplanationsimilarity

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper establishes a method to characterize unsupervised anomaly detectors by computing SHAP explanations of their feature attributions. Detectors whose attributions are similar tend to produce correlated anomaly scores and flag largely the same points, while divergent attributions reliably signal complementary detection. This supplies a new way to pick models for ensembles that capture different kinds of irregularities, improving on selections based only on the detectors' raw scores. The work further shows that explanation diversity is useful only when the individual detectors already perform well on their own. A reader would care because redundant models have long limited the practical gains from ensembles in unsupervised anomaly detection.

What carries the argument

SHAP attribution profiles used to quantify similarity between anomaly detectors' decision mechanisms via feature importance.

What would settle it

An observation of two detectors that share nearly identical SHAP attribution profiles yet detect largely disjoint sets of anomalies would falsify the claimed correspondence.

Watch

Extended reading notes

Core claim

Using SHAP to quantify feature attributions, the authors demonstrate that similarity in these attribution profiles between anomaly detectors corresponds to correlated anomaly scores and largely overlapping detected anomalies, whereas divergence in explanations reliably indicates complementary detection behavior. This allows explanation-driven metrics to serve as a distinct criterion for selecting ensemble members compared to raw outputs, resulting in more diverse and effective ensembles when combined with high individual model performance.

Load-bearing premise

SHAP attributions faithfully capture the decision mechanisms of the anomaly detectors, so that similarity or divergence in attributions directly corresponds to overlapping or complementary sets of detected anomalies.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

read the letter

The core takeaway is that this paper uses SHAP attributions to measure how similar or different anomaly detectors are in the features they rely on, then shows that similar attributions line up with correlated scores and overlapping detections while divergent ones give more complementary coverage. They build ensembles by favoring explanation diversity on top of individual model strength and report better results than using raw score diversity alone. That distinction between output correlation and explanation similarity is the main new angle here. It moves beyond the usual ensemble tricks of picking by score variance or pairwise correlation and instead looks at the decision mechanisms themselves. The experiments appear to hold on the datasets they used, with the added point that pure diversity without strong base models does not help much. That part tracks with what most people see in practice. The soft spot is the stability of the SHAP values for unsupervised detectors. These models score points relative to some reference distribution, and swapping the background set can change which features get high attribution without changing the actual anomalies flagged. If the paper does not test alternate backgrounds or fix the reference consistently across models, the measured similarity could partly reflect that choice rather than true complementarity. A quick ablation on background selection would have made the claims tighter. The citation pattern looks standard for the area, pulling in SHAP work and common anomaly detectors without obvious gaps. This is useful for anyone building ensembles for monitoring or security applications where you want detectors that catch different kinds of outliers. It is not a foundational result but it gives a concrete, implementable criterion that could be tested on new data. I would send it to peer review. The idea is clear enough and the experiments are in principle checkable, even if the background sensitivity issue needs addressing in revision.

Referee Report

2 major / 1 minor

Summary. The paper proposes using SHAP explanations to analyze the decision mechanisms of unsupervised anomaly detection algorithms. It demonstrates that models with similar SHAP attribution profiles tend to produce correlated anomaly scores and detect largely overlapping anomalies, while divergent profiles indicate complementary behaviors. This is leveraged to select models for ensembles, showing improved performance when combining explanation diversity with high individual model accuracy.

Significance. Should the findings hold under rigorous validation, this methodology offers a valuable new criterion for constructing complementary ensembles in unsupervised anomaly detection, distinct from traditional output-based diversity measures. It underscores that while diversity is important, it must be paired with strong base model performance, which could influence ensemble design practices in the field.

major comments (2)

[Methodology and SHAP Setup] The manuscript does not provide an ablation study or justification for the background distribution used in computing SHAP values for the anomaly detectors. Given that anomaly scores are relative to normal data, different background choices could alter the attributions substantially, risking that the observed correlation between explanation similarity and anomaly overlap is not robust.
[Experimental Results] The experimental results assert that explanation-driven metrics differ from raw outputs for ensemble selection, but lack direct head-to-head comparisons, statistical significance tests, or controls across multiple datasets to establish that explanation divergence reliably yields superior complementarity beyond output correlation.

minor comments (1)

[Throughout] Ensure consistent terminology for 'explanation similarity' versus 'attribution profiles' to improve readability.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive feedback. The comments highlight important aspects of robustness and empirical validation that we will address through targeted revisions. Below we respond point by point to the major comments.

read point-by-point responses

Referee: [Methodology and SHAP Setup] The manuscript does not provide an ablation study or justification for the background distribution used in computing SHAP values for the anomaly detectors. Given that anomaly scores are relative to normal data, different background choices could alter the attributions substantially, risking that the observed correlation between explanation similarity and anomaly overlap is not robust.

Authors: We agree that the choice of background distribution merits explicit justification and sensitivity analysis. In the submitted manuscript we used the empirical distribution of the training data (standard for unsupervised anomaly detection to represent the normal baseline). To strengthen the work we will add a dedicated ablation subsection that evaluates alternative backgrounds, including the feature-wise mean, random subsamples from the training set, and synthetically generated normal points. We will report that the reported correlations between explanation similarity and anomaly overlap remain stable across these choices, thereby confirming robustness. revision: yes
Referee: [Experimental Results] The experimental results assert that explanation-driven metrics differ from raw outputs for ensemble selection, but lack direct head-to-head comparisons, statistical significance tests, or controls across multiple datasets to establish that explanation divergence reliably yields superior complementarity beyond output correlation.

Authors: We acknowledge that the current experimental section would benefit from more explicit comparative analysis. The manuscript already evaluates explanation-driven selection against random and output-correlation baselines on multiple datasets and shows gains in complementarity when high individual accuracy is preserved. In the revision we will add direct head-to-head tables, apply statistical significance tests (paired Wilcoxon signed-rank tests with Bonferroni correction), and include additional datasets with controlled variations in dimensionality and anomaly type. These additions will provide clearer quantitative evidence that explanation divergence supplies complementary information beyond output correlation alone. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical SHAP correlations are independent of target metrics

full rationale

The paper applies standard SHAP attribution to anomaly detectors, then computes empirical correlations between attribution similarity and anomaly-score overlap. No equations or steps reduce the measured complementarity to a fitted parameter or self-citation chain; the observed relationships are reported as data-driven findings rather than derived by construction from the inputs. The central claim therefore remains externally falsifiable and does not collapse into its own definitions.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

The work relies on the standard assumption that SHAP values provide faithful local explanations for black-box models and that correlation of these explanations tracks correlation of anomaly detections. No new free parameters or invented entities are introduced in the abstract.

assumptions (1)

domain assumption SHAP values faithfully reflect the decision mechanisms of the anomaly detection models
Invoked when using attribution profiles to measure similarity and complementarity.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Analyzing Shapley Additive Explanations to Understand Anomaly Detection Algorithm Behaviors and Their Complementarity." pith.science (2026). https://pith.science/paper/2602.00208

@misc{pith2026260200208,
  author       = {Pith},
  title        = {Pith review of: Analyzing Shapley Additive Explanations to Understand Anomaly Detection Algorithm Behaviors and Their Complementarity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2602.00208}},
  note         = {Machine review of arXiv:2602.00208}
}

read the original abstract

Unsupervised anomaly detection is a challenging problem due to the diversity of data distributions and the lack of labels. Ensemble methods are often adopted to mitigate these challenges by combining multiple detectors, which can reduce individual biases and increase robustness. Yet building an ensemble that is genuinely complementary remains challenging, since many detectors rely on similar decision cues and end up producing redundant anomaly scores. As a result, the potential of ensemble learning is often limited by the difficulty of identifying models that truly capture different types of irregularities. To address this, we propose a methodology for characterizing anomaly detectors through their decision mechanisms. Using SHapley Additive exPlanations, we quantify how each model attributes importance to input features, and we use these attribution profiles to measure similarity between detectors. We show that detectors with similar explanations tend to produce correlated anomaly scores and identify largely overlapping anomalies. Conversely, explanation divergence reliably indicates complementary detection behavior. Our results demonstrate that explanation-driven metrics offer a different criterion than raw outputs for selecting models in an ensemble. However, we also demonstrate that diversity alone is insufficient; high individual model performance remains a prerequisite for effective ensembles. By explicitly targeting explanation diversity while maintaining model quality, we are able to construct ensembles that are more diverse, more complementary, and ultimately more effective for unsupervised anomaly detection.

Figures

Figures reproduced from arXiv: 2602.00208 by the authors.

**Figure 2.** Relationship between ensemble diversity (given by [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 23 canonical work pages

[1]

Anomaly detection: A survey

V. Chandola et al. “Anomaly detection: A survey”. In: ACM computing surveys (CSUR) 41.3 (2009), pp. 1–58

work page 2009
[2]

A unifying review of deep and shallow anomal y detection

L. Ruﬀ et al. “A unifying review of deep and shallow anomal y detection”. In: Proceedings of the IEEE 109.5 (2021), pp. 756–795

work page 2021
[3]

Anomalies detection by unsupervised lear ning using ex- plainable artiﬁcial intelligence in nuclear power plants

S. W. Oh et al. “Anomalies detection by unsupervised lear ning using ex- plainable artiﬁcial intelligence in nuclear power plants” . In: Transactions of the Korean Nuclear Society Spring Meeting Jeju, Korea . 2022

work page 2022
[4]

C. C. Aggarwal. Outlier Analysis . Springer, 2016

work page 2016
[5]

No free lunch theorems f or optimiza- tion

D. H. Wolpert and W. G. Macready. “No free lunch theorems f or optimiza- tion”. In: Transactions on evolutionary computation (2002)

work page 2002
[6]

ADBench: Anomaly detection benchmark

S. Han et al. “ADBench: Anomaly detection benchmark”. In : NeurIPS 35 (2022), pp. 32142–32159

work page 2022
[7]

On evaluation of outlier rankings and outlier scores

E. Schubert et al. “On evaluation of outlier rankings and outlier scores”. In: International conference on data mining . SIAM. 2012, pp. 1047–1058

work page 2012
[8]

The need for unsupervised outlier model se lection: A review and evaluation of internal evaluation strategies

M. Q. Ma et al. “The need for unsupervised outlier model se lection: A review and evaluation of internal evaluation strategies”. In: ACM SIGKDD Explorations Newsletter 25.1 (2023), pp. 19–35. 14 J. Levy, P. Saves, M. Garouani, N. Verstaevel and B. Gaudou

work page 2023

Show all 23 references

[9]

A uniﬁed approach to interp reting model predictions

S. M. Lundberg and S.-I. Lee. “A uniﬁed approach to interp reting model predictions”. In: NeurIPS 30 (2017)

2017
[10]

PyOD: A Python Toolbox for Scalable Outli er Detection

Y. Zhao et al. “PyOD: A Python Toolbox for Scalable Outli er Detection”. In: Journal of Machine Learning Research 20.96 (2019), pp. 1–7

2019
[11]

How to evaluate the quality of unsupervised an omaly detection algorithms?

N. Goix. “How to evaluate the quality of unsupervised an omaly detection algorithms?” In: arXiv preprint arXiv:1607.01152 (2016)

2016 arXiv
[12]

Internal evaluation of unsupervis ed outlier detection

H. O. Marques et al. “Internal evaluation of unsupervis ed outlier detection”. In: TKDD 14.4 (2020), pp. 1–42

2020
[13]

Unsupervised model selection for variat ional disentangled representation learning

S. Duan et al. “Unsupervised model selection for variat ional disentangled representation learning”. In: International Conference on Learning Repre- sentations. 2019

2019
[14]

InfoGAN-CR and ModelCentrality: Self-su pervised Model Training and Selection for Disentangling GANs

Z. Lin et al. “InfoGAN-CR and ModelCentrality: Self-su pervised Model Training and Selection for Disentangling GANs”. In: International confer- ence on machine learning . PMLR. 2020, pp. 6127–6139

2020
[15]

Less is more: Building selecti ve anomaly en- sembles

S. Rayana and L. Akoglu. “Less is more: Building selecti ve anomaly en- sembles”. In: TKDD 10.4 (2016), pp. 1–33

2016
[16]

Unsupervised time series outlier dete ction with diversity- driven convolutional ensembles

D. Campos et al. “Unsupervised time series outlier dete ction with diversity- driven convolutional ensembles”. In: Proceedings of the VLDB Endowment 15.3 (2021), pp. 611–623

2021
[17]

Interpretability needs a new paradigm

A. Madsen et al. “Interpretability needs a new paradigm ”. In: arXiv (2024)

2024
[18]

Surrogate Modeling and Explainable Art iﬁcial Intelligence for Complex Systems: A Workﬂow for Automated Simulation Exp loration

P. Saves et al. “Surrogate Modeling and Explainable Art iﬁcial Intelligence for Complex Systems: A Workﬂow for Automated Simulation Exp loration”. In: arXiv preprint (2025)

2025
[19]

XStacking: An eﬀective and inherent ly explainable framework for stacked ensemble learning

M. Garouani et al. “XStacking: An eﬀective and inherent ly explainable framework for stacked ensemble learning”. In: Information Fusion (2025)

2025
[20]

Learning to rank using gradient descen t

C. Burges et al. “Learning to rank using gradient descen t”. In: Proceedings of the 22nd international conference on Machine learning . 2005, pp. 89–96

2005
[21]

The detection of disease clustering and a ge neralized regression approach

N. Mantel. “The detection of disease clustering and a ge neralized regression approach”. In: Cancer research 27 (1967), pp. 209–220

1967
[22]

Beyond the single-best model: Rashomon partial depen- dence proﬁle for trustworthy explanations in automl

M. Cavus et al. “Beyond the single-best model: Rashomon partial depen- dence proﬁle for trustworthy explanations in automl”. In: International Conference on Discovery Science . Springer. 2025, pp. 445–459

2025
[23]

TimeCIEL: Contextual Interactive Ensem ble Learning for Time Series Classiﬁcation

J. Levy et al. “TimeCIEL: Contextual Interactive Ensem ble Learning for Time Series Classiﬁcation”. In: International Conference on Practical Ap- plications of Agents and Multi-Agent Systems . Springer. 2025, pp. 316– 327

2025

Pith tools

Reviewed May 16, 2026 · model on record in the stance chip above.

[1] [1]

Anomaly detection: A survey

V. Chandola et al. “Anomaly detection: A survey”. In: ACM computing surveys (CSUR) 41.3 (2009), pp. 1–58

work page 2009

[2] [2]

A unifying review of deep and shallow anomal y detection

L. Ruﬀ et al. “A unifying review of deep and shallow anomal y detection”. In: Proceedings of the IEEE 109.5 (2021), pp. 756–795

work page 2021

[3] [3]

Anomalies detection by unsupervised lear ning using ex- plainable artiﬁcial intelligence in nuclear power plants

S. W. Oh et al. “Anomalies detection by unsupervised lear ning using ex- plainable artiﬁcial intelligence in nuclear power plants” . In: Transactions of the Korean Nuclear Society Spring Meeting Jeju, Korea . 2022

work page 2022

[4] [4]

C. C. Aggarwal. Outlier Analysis . Springer, 2016

work page 2016

[5] [5]

No free lunch theorems f or optimiza- tion

D. H. Wolpert and W. G. Macready. “No free lunch theorems f or optimiza- tion”. In: Transactions on evolutionary computation (2002)

work page 2002

[6] [6]

ADBench: Anomaly detection benchmark

S. Han et al. “ADBench: Anomaly detection benchmark”. In : NeurIPS 35 (2022), pp. 32142–32159

work page 2022

[7] [7]

On evaluation of outlier rankings and outlier scores

E. Schubert et al. “On evaluation of outlier rankings and outlier scores”. In: International conference on data mining . SIAM. 2012, pp. 1047–1058

work page 2012

[8] [8]

The need for unsupervised outlier model se lection: A review and evaluation of internal evaluation strategies

M. Q. Ma et al. “The need for unsupervised outlier model se lection: A review and evaluation of internal evaluation strategies”. In: ACM SIGKDD Explorations Newsletter 25.1 (2023), pp. 19–35. 14 J. Levy, P. Saves, M. Garouani, N. Verstaevel and B. Gaudou

work page 2023