On synthetic AND/OR/XOR datasets with perfectly accurate models, every tested saliency method sometimes ranks a truly irrelevant input above a necessary one, so the scores cannot be trusted as relevance rankings.
Investigating sanity checks for saliency maps with image and text classification
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Saliency maps have shown to be both useful and misleading for explaining model predictions especially in the context of images. In this paper, we perform sanity checks for text modality and show that the conclusions made for image do not directly transfer to text. We also analyze the effects of the input multiplier in certain saliency maps using similarity scores, max-sensitivity and infidelity evaluation metrics. Our observations reveal that the input multiplier carries input's structural patterns in explanation maps, thus leading to similar results regardless of the choice of model parameters. We also show that the smoothness of a Neural Network (NN) function can affect the quality of saliency-based explanations. Our investigations reveal that replacing ReLUs with Softplus and MaxPool with smoother variants such as LogSumExp (LSE) can lead to explanations that are more reliable based on the infidelity evaluation metric.
citation-role summary
background 1
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
background 1representative citing papers
citing papers explorer
-
Saliency Maps are Ambiguous: Analysis of Logical Relations on First and Second Order Attributions
On synthetic AND/OR/XOR datasets with perfectly accurate models, every tested saliency method sometimes ranks a truly irrelevant input above a necessary one, so the scores cannot be trusted as relevance rankings.