Pith. sign in

REVIEW 4 major objections 6 minor 28 references

Saliency maps that keep only the pixels needed to preserve a model's output beat sensitivity maps on faithfulness.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 04:58 UTC pith:FUWAHLJH

load-bearing objection Clean modular sufficiency saliency with competitive deletion numbers; the three-way self-composite is a real soft spot for the sufficiency claim, but not a collapse of the paper. the 4 major comments →

arxiv 2607.09164 v1 pith:FUWAHLJH submitted 2026-07-10 cs.CV

What Pixels Are Enough? SEAMS: Sufficiency Saliency via MSE-Preservation Soft-Masks

Magdalena Tr\k{e}dowicz , {\L}ukasz Struski , Arkadiusz Lewicki , Karolina Pachota , Andrzej Grudzie\'n , Mateusz Jag{\l}a , Jacek Tabor This is my paper
classification cs.CV
keywords sufficiency saliencysoft-mask optimisationpreservation MSEvisual explanationVision Transformersinsertion-deletionmodel-agnostic
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Most visual explanations measure how sensitive a model is to tiny input changes. This paper argues that the more useful question is which pixels are enough to keep the model’s behaviour the same. SEAMS finds a compact soft mask by optimising so that a frozen model output—class probability, CLS embedding, or patch tokens—stays close under a three-way composite built only from the query image: kept pixels, a self-augmented distractor, and heavy blur. The same pipeline, with no architecture-specific attribution rules or external distractor sets, produces object-level, class-conditioned, and token-level maps. The masks are sparse, competitive on insertion/deletion and ascending pixel-deletion tests, transfer to medical fundus images, and show that different backbones often select different sufficient pixels while matching preservation fidelity.

Core claim

A single soft-mask optimisation that minimises MSE preservation of a chosen differentiable model output, under a learnable budget and a three-way composite made only from the query image, yields compact sufficiency saliency that is competitive on faithfulness benchmarks and modular across targets and architectures.

What carries the argument

SEAMS: unconstrained logits turned into a soft mask by softplus, mean-normalisation with learnable budget α, and clip; the mask drives a three-way composite (original, heavyAug distractor, blur) whose augmented forward pass is matched in MSE to the fixed original target.

Load-bearing premise

That low preservation error under the paper’s self-made blur-and-distractor fill really means the kept pixels are enough for the model, rather than the fill still leaking usable context.

What would settle it

Re-evaluate the same optimised masks with a pure grey (or pure noise) fill that removes all non-mask context: if preservation fidelity collapses while insertion/deletion rankings stay high only under the training fill, the sufficiency claim fails for that composite.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes SEAMS, a post-hoc sufficiency saliency method that optimises a continuous soft mask so a frozen differentiable model output (class probability, CLS embedding, or dense tokens) is preserved under a three-way composite built only from the query image (original content, self-augmented distractor, and heavy blur). Masks are parameterised by unconstrained logits transformed via softplus, mean-normalisation, and clipping with a learnable budget α, and trained with an MSE preservation loss plus an α penalty, without auxiliary distractor datasets or differentiable top-k. Experiments on ViT and ConvNeXt families report competitive insertion/deletion AUCs, strong ascending pixel-deletion faithfulness versus DAVE/LeGrad and gradient baselines, multi-target explanations from one pipeline, low cross-architecture mask agreement at similar fidelity, and a qualitative transfer demo on private ROP fundus images.

Significance. If the sufficiency interpretation is made rigorous, the work is a useful and practical contribution to visual explanation: a single, architecture-agnostic optimisation loop that switches among object-level, class-conditioned, and token-level targets, needs no external reference set, and yields compact nearly-binary masks competitive on standard faithfulness benchmarks. The cross-architecture finding—that different backbones can preserve similar outputs with largely disjoint pixel sets—is a concrete, falsifiable observation that sensitivity methods do not naturally surface. Strengths include modular targets, a simple normalise-and-clip sparsity mechanism, insertion/deletion and ascending-deletion results that beat strong baselines (Tables 1–2, Fig. 4), and an explicit train/eval fill analysis in Supp. F. The main scientific value is therefore conditional on clarifying what “sufficient” means under the self-composite versus pure information removal.

major comments (4)
  1. §3 Composite image / Eq. (1) and Supp. F / Table 5: the central sufficiency claim is that low preservation MSE under the three-way self-composite means the retained pixels are enough for model behaviour. Supp. F shows a large train/eval gap when non-salient regions are grey-filled instead of blur-filled (e.g. mean CLS cos 0.740→0.918; spaniel p_masked 0.34→0.93). The paper itself states that masks rely on the blur fill’s low-frequency context. Because the optimisation objective (Eq. 2) always trains against blur+distractor, absolute “sufficiency” of high-M pixels alone is not established by the training loss. Main-text claims and the title should either (i) report primary preservation metrics under grey/mean fill as the default, (ii) include an ablation that optimises under grey fill, or (iii) reframe results as composite-conditioned sufficiency with the gap quantified in the main paper—
  2. §4 Faithfulness / Table 1 vs. pixel-deletion protocol (Fig. 4, Table 2): insertion/deletion and ascending deletion use grey or channel-mean replacement and still favour SEAMS, which supports ranking quality. However, SEAMS masks for the deletion benchmark are optimised with fewer steps (T=500) and n_copies=3 median aggregation, while main qualitative/ins-del settings use T=2000 and different targets. The paper should state whether the same mask generation protocol is used for all faithfulness tables, and report ins/del AUCs under a fixed, fully specified protocol (including fill, T, and aggregation) so that the large ascending-deletion gains cannot be attributed to protocol mismatch with baselines.
  3. Abstract and §4 claim masks are “stable across random initialisations,” but the main text and reported tables give no quantitative stability metric (e.g. IoU/cos across seeds, variance of α or of deletion AUC). Either add a short multi-seed study on a fixed image set or soften the claim. Without that evidence the stability statement is unsupported relative to the rest of the evaluation.
  4. §4 Transfer to medical imaging / Fig. 7 (and Supp. C): the ROP demo is only qualitative on private data with no insertion/deletion, preservation MSE, or clinical agreement metric, and no public subset or synthetic proxy. As written it cannot support a generalisation claim beyond “the optimiser runs and produces sparse maps.” Either add quantitative faithfulness on the ROP model (even on a held-out private split summarised by AUC/cos) or move the section to a brief qualitative appendix and avoid implying validated clinical transfer in the abstract/conclusions.
minor comments (6)
  1. Main text cites “Figure 19” for qualitative comparisons before later figures (e.g. Fig. 5); figure numbering appears out of order relative to the supplemental block. Renumber so main-text figures are sequential and self-contained.
  2. §3: M'' = min(1−M, α) is easy to misread as a second soft mask rather than a capped residual. A one-sentence intuition (why the cap equals α) would help readers who skip the partition-of-unity algebra.
  3. Hyperparameters λ_α=2.0, α_0=0.25, T=2000, D=4 are fixed for all main experiments; a short sensitivity plot (e.g. λ_α ∈ {0.5,2,8} on a small image set) would strengthen the “simple fixed pipeline” claim.
  4. Related work: Vision DiffMask and Extremal Perturbations are cited; a clearer one-paragraph contrast on modular targets and self-only composites would help position novelty without overstating uniqueness of soft-mask optimisation.
  5. Typos/notation: “desribed” (Supp. G), inconsistent spacing in “Fig. 1.Sufficiency”, and mixed use of g vs f for the encoder/target in places (e.g. Algorithm 1 uses f while §3 uses g).
  6. Limitations correctly note iterative cost (~40s/image); stating wall-clock and GPU model in the main experimental setup (not only Supp. A) would aid reproducibility comparisons with single-pass attribution.

Circularity Check

0 steps flagged

No circularity: SEAMS is an optimization method evaluated on independent insertion/deletion and pixel-deletion benchmarks, not a derivation that reduces to its own inputs.

full rationale

SEAMS defines a soft-mask optimization (preservation MSE under a three-way self-composite plus a learnable budget; Eq. 2 and Eq. 1) and reports empirical outcomes on external faithfulness protocols (RISE-style insertion/deletion AUCs in Table 1; ascending pixel-deletion curves and AUCs in Fig. 4 and Table 2) against independent baselines (IxG, SmoothGrad, IntGrad, LRP variants, LeGrad, DAVE). The training objective is not identical to the evaluation metrics: optimization uses the three-way composite with self-augmented distractor and blur, while reported insertion/deletion and deletion curves use grey or channel-mean fill. Supp. F and Table 5 even document a train/eval fill gap rather than hiding it, which is a validity concern, not a by-construction identity. There is no fitted parameter renamed as a prediction, no uniqueness theorem imported from the authors, and no load-bearing self-citation that forces the central claim. Overlap of co-author Tabor with the DAVE citation is ordinary baseline comparison, not a premise that SEAMS reduces to. The paper is a self-contained algorithmic contribution with external empirical checks; circularity score is 0.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

SEAMS is an empirical optimization method. Load-bearing choices are the preservation MSE objective, the self-composite fill, the softplus-normalize-clip mask, fixed regularizer weight, and the assumption that competitive insertion/deletion implies useful sufficiency explanations. No new physical entities; free parameters are optimization hyperparameters chosen by hand and held fixed.

free parameters (5)
  • sparsity weight λ_α = 2.0
    Fixed at 2.0 for all encoders and targets; directly trades mask size vs preservation and is not derived.
  • initial budget α_0 = 0.25
    Initial soft-mask mass; set to 0.25 and learned but strongly regularized.
  • optimization steps T and distractor count D = T=2000, D=4 (main)
    T=2000 (or 500 in deletion protocol), D=4; control convergence and stochasticity of the composite.
  • Adam learning rates η_ℓ, η_α = 0.05 / 0.01
    Hand-chosen (0.05 and 0.01) for logits and budget; affect mask sharpness and final α.
  • blur σ range and HeavyAug strength = σ∈[15,25]; HeavyAug as in Supp. D
    σ ~ U[15,25] and aggressive geometric/photometric distractor define the non-salient fill; change the meaning of sufficiency.
axioms (5)
  • domain assumption Preserving a chosen differentiable output under MSE on an augmented composite is a valid operational definition of sufficiency for model behaviour.
    Stated as the method objective in §3 Eq. (2); standard in sufficiency/perturbation XAI but not uniquely forced.
  • ad hoc to paper A three-way partition M + M'' + N = 1 with self-distractor and blur is an adequate information-removal model without external reference images.
    Eq. (1) and Composite image section; design choice that replaces auxiliary distractor datasets.
  • ad hoc to paper Normalise-and-clip of softplus logits with learnable α yields sufficiently sparse nearly-binary masks without top-k.
    Mask parametrization and Implicit sparsification in §3; empirical, not proved.
  • domain assumption Insertion/deletion and ascending pixel-deletion curves measure faithfulness of sufficiency masks.
    Used throughout §4 following RISE and related protocols.
  • domain assumption Frozen pretrained ImageNet (and private ROP) encoders are representative testbeds for architecture-agnostic post-hoc explanation.
    Experimental Setup §4.
invented entities (2)
  • SEAMS three-way self-composite (M, M'', N) with M''=min(1−M, α) no independent evidence
    purpose: Replace non-salient pixels using only the query image while discouraging reliance on weak texture under blur alone.
    Core mechanism of the paper; evaluated only inside this optimization loop, not as an independent theory of vision.
  • Normalise-and-clip soft-mask with learnable budget α no independent evidence
    purpose: Induce sparse continuous masks without differentiable top-k or Gumbel sorting.
    Parameterization introduced to avoid subset-selection machinery; behaviour is empirical.

pith-pipeline@v1.1.0-grok45 · 23727 in / 3308 out tokens · 50885 ms · 2026-07-13T04:58:28.119988+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of What Pixels Are Enough? SEAMS: Sufficiency Saliency via MSE-Preservation Soft-Masks." pith.science (2026). https://pith.science/paper/FUWAHLJH

@misc{pith2026260709164,
  author       = {Pith},
  title        = {Pith review of: What Pixels Are Enough? SEAMS: Sufficiency Saliency via MSE-Preservation Soft-Masks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FUWAHLJH}},
  note         = {Machine review of arXiv:2607.09164}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Saliency maps are most useful when they identify the image regions that are sufficient to preserve a model's behaviour. We introduce SEAMS, a sufficiency-based saliency method that directly optimises a soft mask using a preservation objective. Given a frozen differentiable model output, such as a class probability, CLS embedding, or token representation, SEAMS searches for a compact mask that preserves the selected output. The approach relies on a simple optimisation framework based on soft masks, a learnable budget, and a three-way image composite generated entirely from the query image. As a result, it requires no auxiliary distractor dataset, architecture-specific attribution mechanism, or differentiable top-k relaxation. Experiments with frozen ViT-S/16 and ConvNeXt models show that the same optimisation pipeline can generate object-level, class-conditioned, and token-level explanations by changing only the preserved target. The resulting masks are compact, interpretable, stable across random initialisations, and competitive on insertion and deletion benchmarks. Our results also indicate that different architectures often rely on different sufficient evidence while achieving similar preservation fidelity, highlighting the architecture-dependent nature of visual explanations.

Figures

Figures reproduced from arXiv: 2607.09164 by Andrzej Grudzie\'n, Arkadiusz Lewicki, Jacek Tabor, Karolina Pachota, {\L}ukasz Struski, Magdalena Tr\k{e}dowicz, Mateusz Jag{\l}a.

Figure 1
Figure 1. Figure 1: Sufficiency-based saliency across scales and ar￾chitectures. SEAMS identifies compact image regions that are sufficient to preserve a selected model representation. Top: Im￾ageNet image analysed with a supervised ViT-S/16 encoder. Bottom: DIV2K image analysed with a self-supervised DI￾NOv2 ViT-S/14 encoder. From left to right: original image, optimised soft mask, and the corresponding saliency overlay. Des… view at source ↗
Figure 2
Figure 2. Figure 2: One optimisation pipeline, multiple explanatory targets. Each row corresponds to a single ImageNet image, while each column preserves a different target output: a class probability, the CLS embedding, or the full patch-token rep￾resentation. The optimisation procedure, regularisation, and image composite remain unchanged. Different targets produce different saliency patterns, showing that distinct componen… view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the SEAMS framework. The target representation y ⋆ is first computed from the original image and then kept fixed. A learnable soft mask determines which pixels are preserved, replaced by a self-generated distractor, or substituted with blurred context. The resulting composite image is augmented and evaluated by the frozen model. Optimisation updates only the mask logits and the sparsity budget,… view at source ↗
Figure 4
Figure 4. Figure 4: Pixel-deletion faithfulness. Deletion curves for ViT￾B/16 (top) and DeiT-III-B/16 (bottom). Pixels are removed in ascending order of attribution score. Higher curves indicate better faithfulness. SEAMS (ours, gray) consistently preserves the highest target-class probability across the deletion range on both architectures. Additional results at 10% masking intervals are provided in Supplementary Material Se… view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison of attribution maps. For selected ImageNet-1K validation examples (rows), we display the input image (left) alongside attribution maps generated by C-LRP (Chefer-LRP), LeGrad, SmoothGrad, AttnLRP, IntGrad (Integrated Gradients), DAVE, and our method (columns). Both DAVE and SEAMS deliver sharper, more object-aligned, and spatially coherent explanations with significantly fewer patch-… view at source ↗
Figure 7
Figure 7. Figure 7: Generalisation beyond ImageNet. Qualitative re￾sults on a private retinopathy of prematurity (ROP) dataset. Columns left to right: original fundus image, soft mask, and binarised mask overlaid on the original. Each row shows a different patient. Without any modification to the optimisa￾tion procedure or hyperparameters, SEAMS produces sparse and clinically meaningful explanations, demonstrating trans￾ferab… view at source ↗
Figure 6
Figure 6. Figure 6: Cross-architecture saliency. Each row: one of four ImageNet validation images. Column 1: original. Columns 2– 4: ViT-S/16 masks under g = pc, g = fCLS, g = ftok. Columns 5–7: ConvNeXt-Tiny masks under the same three targets. Only the encoder (and its input normalisation) dif￾fers between the two sets of columns; all other hyperparam￾eters are identical. Quantitative agreement is in Tab. 3; inser￾tion/delet… view at source ↗
Figure 8
Figure 8. Figure 8: Cross-architecture saliency on the larger back [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Generalisation beyond ImageNet. Qualitative results on a private retinopathy of prematurity (ROP) dataset. Without any changes to the optimisation procedure or hyperparameters, SEAMS produces sparse and clinically meaningful explanations on retinal fundus images, demonstrating its transferability to a real-world medical imaging domain. D. Augmentation Details We employ two distinct augmentation pipelines, … view at source ↗
Figure 10
Figure 10. Figure 10: Multi-instance scenes: sufficiency vs. sensitivity. Each row shows an ImageNet validation image containing multiple instances of the same object class (left), the SEAMS soft mask (centre), and the DAVE attribution map (right). Sensitivity-based methods such as DAVE highlight all visible instances, since each one independently influences the model output. SEAMS instead identifies the minimal sufficient evi… view at source ↗
Figure 11
Figure 11. Figure 11: Multi-class scene panel: ImageNet ice-cream im [PITH_FULL_IMAGE:figures/full_fig_p016_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Evaluation composite makes a real difference. [PITH_FULL_IMAGE:figures/full_fig_p016_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Class-conditioned saliency on a multi-class scene (ImageNet validation image idx [PITH_FULL_IMAGE:figures/full_fig_p017_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Six DIV2K validation images at 518×518 under DINOv2 ViT-S/14 (cls-embed target). Row 1: originals (lion, marble statues, scuba diver, mountain hikers, portrait, skier). Row 2: masks (hot heatmap on black; rescaled to [0, 1] by per-image max and amplified by 1.1×). Row 3: M ·x + (1−M)·grey. Per-image numbers: lion α = 0.060, cosblur = 0.76; statues 0.059, 0.88; diver 0.058, 0.80; hikers 0.046, 0.86; portra… view at source ↗
Figure 15
Figure 15. Figure 15: Mean target-class probability at each 10% masking step for ViT-B/16. SEAMS (grey) achieves the highest probability [PITH_FULL_IMAGE:figures/full_fig_p019_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Mean target-class probability at each 10% masking step for DeiT-III-B/16. SEAMS (grey) consistently preserves the [PITH_FULL_IMAGE:figures/full_fig_p019_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Visual explanations across different ViT architectures. Using validation examples from ImageNet-1K (columns), we [PITH_FULL_IMAGE:figures/full_fig_p020_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Comparison of class-token representations across different ViT architectures. For each ImageNet-1K validation sample, [PITH_FULL_IMAGE:figures/full_fig_p021_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Comparison between SEAMS and post-hoc attribution methods using a DINO ViT-B/16-224 backbone. For selected [PITH_FULL_IMAGE:figures/full_fig_p022_19.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

28 extracted references · 5 linked inside Pith

  1. [1]

    On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation

    Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Woj- ciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10(7):e0130140, 2015. 2, 3

    2015

  2. [2]

    Legrad: An ex- plainability method for vision transformers via feature formation sensitivity

    Walid Bousselham, Angie Boggust, Sofian Chaybouti, Hendrik Strobelt, and Hilde Kuehne. Legrad: An ex- plainability method for vision transformers via feature formation sensitivity. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 20336–20345, 2025. 7

    2025

  3. [3]

    What made you do this? understanding black-box decisions with sufficient input subsets

    Brandon Carter, Jonas Mueller, Siddhartha Jain, and David Gifford. What made you do this? understanding black-box decisions with sufficient input subsets. InThe 22nd International Conference on Artificial Intelligence and Statistics, pages 567–576. PMLR, 2019. 2, 3

    2019

  4. [4]

    Grad-cam++: Gener- alized gradient-based visual explanations for deep con- volutional networks

    Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian. Grad-cam++: Gener- alized gradient-based visual explanations for deep con- volutional networks. In2018 IEEE winter conference on applications of computer vision (WACV), pages 839–847. IEEE, 2018. 3

    2018

  5. [5]

    Transformer inter- pretability beyond attention visualization

    Hila Chefer, Shir Gur, and Lior Wolf. Transformer inter- pretability beyond attention visualization. InProceedings of the IEEE/CVF conference on computer vision and pat- tern recognition, pages 782–791, 2021. 6

    2021

  6. [6]

    Real time image saliency for black box classifiers.Advances in neural information processing systems, 30, 2017

    Piotr Dabkowski and Yarin Gal. Real time image saliency for black box classifiers.Advances in neural information processing systems, 30, 2017. 3

    2017

  7. [7]

    Under- standing deep networks via extremal perturbations and smooth masks

    Ruth Fong, Mandela Patrick, and Andrea Vedaldi. Under- standing deep networks via extremal perturbations and smooth masks. InProceedings of the IEEE/CVF interna- tional conference on computer vision, pages 2950–2958,

  8. [8]

    Interpretable explana- tions of black boxes by meaningful perturbation

    Ruth C Fong and Andrea Vedaldi. Interpretable explana- tions of black boxes by meaningful perturbation. InPro- ceedings of the IEEE international conference on com- puter vision, pages 3429–3437, 2017. 2, 3

    2017

  9. [9]

    Axiom-based grad-cam: To- wards accurate visualization and explanation of cnns

    Ruigang Fu, Qingyong Hu, Xiaohu Dong, Yulan Guo, Yinghui Gao, and Biao Li. Axiom-based grad-cam: To- wards accurate visualization and explanation of cnns. arXiv preprint arXiv:2008.02312, 2020. 3

    Pith/arXiv arXiv 2008

  10. [10]

    Alignsam: Aligning seg- ment anything model to open context via reinforcement learning

    Duojun Huang, Xinyu Xiong, Jie Ma, Jichang Li, Zequn Jie, Lin Ma, and Guanbin Li. Alignsam: Aligning seg- ment anything model to open context via reinforcement learning. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3205– 3215, 2024. 3

    2024

  11. [11]

    Layercam: Exploring hierarchical class activation maps for localization.IEEE transactions on image processing, 30:5875–5888, 2021

    Peng-Tao Jiang, Chang-Bin Zhang, Qibin Hou, Ming- Ming Cheng, and Yunchao Wei. Layercam: Exploring hierarchical class activation maps for localization.IEEE transactions on image processing, 30:5875–5888, 2021. 3

    2021

  12. [12]

    igos++ integrated gradient optimized saliency by bilateral per- turbations

    Saeed Khorram, Tyler Lawson, and Li Fuxin. igos++ integrated gradient optimized saliency by bilateral per- turbations. InProceedings of the Conference on Health, Inference, and Learning, pages 174–182, 2021. 3

    2021

  13. [13]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. InProceedings of the IEEE/CVF international conference on computer vision, pages 4015–4026, 2023. 3

    2023

  14. [14]

    Vision diffmask: Faithful interpretation of vision trans- formers with differentiable patch masking

    Angelos Nalmpantis, Apostolos Panagiotopoulos, John Gkountouras, Konstantinos Papakostas, and Wilker Aziz. Vision diffmask: Faithful interpretation of vision trans- formers with differentiable patch masking. InProceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3756–3763, 2023. 3

    2023

  15. [15]

    Rise: Ran- domized input sampling for explanation of black-box models.arXiv preprint arXiv:1806.07421, 2018

    Vitali Petsiuk, Abir Das, and Kate Saenko. Rise: Ran- domized input sampling for explanation of black-box models.arXiv preprint arXiv:1806.07421, 2018. 3, 6

    Pith/arXiv arXiv 2018

  16. [16]

    Visualizing deep networks by optimizing with integrated gradients

    Zhongang Qi, Saeed Khorram, and Fuxin Li. Visualizing deep networks by optimizing with integrated gradients. InAAAI, pages 11890–11898, 2020. 3

    2020

  17. [17]

    Grad-cam: Visual explanations from deep net- works via gradient-based localization

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep net- works via gradient-based localization. InProceedings of the IEEE international conference on computer vision, pages 618–626, 2017. 3

    2017

  18. [18]

    Learning important features through propagating activation differences

    Avanti Shrikumar, Peyton Greenside, and Anshul Kun- daje. Learning important features through propagating activation differences. InInternational conference on machine learning, pages 3145–3153. PMlR, 2017. 2, 3

    2017

  19. [19]

    Deep inside convolutional networks: Visualising image classification models and saliency maps.arXiv preprint arXiv:1312.6034, 2013

    Karen Simonyan, Andrea Vedaldi, and Andrew Zisser- man. Deep inside convolutional networks: Visualising image classification models and saliency maps.arXiv preprint arXiv:1312.6034, 2013. 2, 3

    Pith/arXiv arXiv 2013

  20. [20]

    Smoothgrad: removing noise by adding noise.arXiv preprint arXiv:1706.03825,

    Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Vié- gas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise.arXiv preprint arXiv:1706.03825,

    Pith/arXiv arXiv

  21. [21]

    Ax- iomatic attribution for deep networks

    Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Ax- iomatic attribution for deep networks. InInterna- 9 tional conference on machine learning, pages 3319–

  22. [22]

    Diffuse attend and seg- ment: Unsupervised zero-shot segmentation using sta- ble diffusion

    Junjiao Tian, Lavisha Aggarwal, Andrea Colaco, Zsolt Kira, and Mar Gonzalez-Franco. Diffuse attend and seg- ment: Unsupervised zero-shot segmentation using sta- ble diffusion. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 3554–3563, 2024. 3

    2024

  23. [23]

    Score-cam: Score-weighted visual explanations for con- volutional neural networks

    Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr Mardziel, and Xia Hu. Score-cam: Score-weighted visual explanations for con- volutional neural networks. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 24–25, 2020. 3

    2020

  24. [24]

    Salient object detection: a mini review.Frontiers in Signal Pro- cessing, 4:1356793, 2024

    Xiuwenxin Wang, Siyue Yu, Eng Gee Lim, et al. Salient object detection: a mini review.Frontiers in Signal Pro- cessing, 4:1356793, 2024. 3

    2024

  25. [25]

    Dave: Distribution-aware attribution via vit gradient decompo- sition.arXiv preprint arXiv:2602.06613, 2026

    Adam Wróbel, Siddhartha Gairola, Jacek Tabor, Bernt Schiele, Bartosz Zieli´nski, and Dawid Rymarczyk. Dave: Distribution-aware attribution via vit gradient decompo- sition.arXiv preprint arXiv:2602.06613, 2026. 2, 6, 7

    arXiv 2026

  26. [26]

    Learning deep features for discriminative localization

    Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2921–2929, 2016. 3

    2016

  27. [27]

    Vision mamba: Efficient visual representation learning with bidirectional state space model.arXiv preprint arXiv:2401.09417,

    Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model.arXiv preprint arXiv:2401.09417,

    Pith/arXiv arXiv

  28. [28]

    informa- tion removal

    3 Supplemental Material A. Optimisation details For a single query imagex, the learnable parameters are the logit tensorℓ∈R H×W (initialised to zero) and the budget scalarα raw (initialised so thatα= 0.25). We use Adam with learning ratesη ℓ = 0.05for the mask logits andη α = 0.01for the budget, running forT= 2000 steps. At each step we (i) recomputeαandM...

    2000