Pith. sign in

REVIEW 3 major objections 6 minor 55 references

Faithful region-search attributions can train vision models to keep using the same evidence after flips, rotations, and other label-preserving transforms.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 10:55 UTC pith:ZGI5DAQD

load-bearing objection Solid methods paper: LIMA trajectories as self-supervision plus a ranking/truncation loss; flagship attribution gains are partly circular, independent robustness gains are real but small. the 3 major comments →

arxiv 2607.23835 v1 pith:ZGI5DAQD submitted 2026-07-26 cs.CV

Consistent Evidence, Robust Recognition: Faithful Attribution Regularization under Geometric Transformations

Xianghao Jiao , Ruoyu Chen , Wei Wang , Jiazi Hu , Jiawei Liang , Shangquan Sun , Shiming Liu , Qunli Zhang
show 1 more author
Xiaochun Cao
This is my paper
classification cs.CV
keywords attribution regularizationattribution equivariancesubmodular searchtransformation robustnessexplainable AIfaithfulnessgeometric transformations
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Vision models often keep the right class label after a flip or rotation while silently switching which image regions they rely on. That shift is a sign of brittle evidence use. This paper argues that you can fix it only if the training signal truly reflects decision evidence, not a convenient but unfaithful heatmap. It builds an annotation-free loop: greedily search image regions by how they change the model’s output, keep only compact high-confidence sequences, map them to transformed views, and train with a ranking loss that forces the transformed search to pick matching regions in order and stop at the same point. On ImageNet-scale tests the method raises attribution stability and faithfulness sharply and modestly improves accuracy under geometric transforms, with only a small clean-accuracy cost. A sympathetic reader cares because it turns explanations into a practical way to harden what the model actually uses, not just how the maps look.

Core claim

When compact, class-discriminative region sequences from submodular output-based search supervise training, and a Submodular Ranking Loss aligns both the ordered selection path and the stopping step on transformed inputs, models learn more consistent decision evidence under label-preserving geometric transforms—improving attribution stability and faithfulness and modestly raising transformed-input accuracy while barely hurting clean accuracy.

What carries the argument

Submodular Ranking Loss: a differentiable surrogate with a path-consistency (selection-ranking) term that ranks each transformed target region above remaining candidates at every search step, plus a termination-alignment (selection-truncation) term that pushes the terminal subset over the same consistency stop threshold—regularizing both final attributions and the discrete evidence-selection trajectory.

Load-bearing premise

The filtered greedy region-search sequences are faithful enough teaching targets that matching their order and stop point on transformed images really rewires the model’s decision evidence, not just the search procedure.

What would settle it

If after this training, models still flip their relied-on regions under the same geometric transforms while the search-based maps look more stable—or Insertion/Deletion and transformed-input accuracy fail to improve relative to gradient-map consistency baselines—the claim that faithful search supervision fixes evidence reliance would fail.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Attribution consistency objectives are only trustworthy when the attribution is decision-linked; gradient-map agreement alone need not improve transform robustness.
  • Self-supervised geometric equivariance of evidence can be trained without human relevance labels by recycling search trajectories as targets.
  • Regularizing the full selection path and stop condition beats matching only a final mask for both explanation quality and transform accuracy.
  • The same recipe transfers across ViTs and convolutional backbones with only small clean-accuracy cost on ImageNet-scale data.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Any post-hoc explainer that scores region subsets by output change could plug into the same two-stage query-and-rank loop.
  • If path alignment is doing real work, similar ranking losses might stabilize evidence under photometric or style shifts, not only geometry.
  • Deployment settings that already run multi-crop or flip test-time augmentation are natural places to harvest these paired trajectories at low extra label cost.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes an annotation-free attribution regularization framework in which a LIMA-style greedy submodular region search, run on the pretrained model itself, produces ordered evidence sequences that serve as self-generated supervision. A Submodular Ranking Loss (selection-ranking term, Eq. 7, plus selection-truncation term, Eq. 8) trains the model so that the search trajectory on a geometrically transformed input follows the spatially transformed query trajectory and terminates at the corresponding step. The authors argue this regularizes the underlying decision process rather than merely aligning gradient-based saliency maps, and report large gains in LIMA-based attribution stability/faithfulness on ImageNet-100 (Stability 0.14→0.27, Insertion +52.1%, Deletion −62.2% on ViT-B/16 at −0.28 pp accuracy), modest transformed-input accuracy gains on ImageNet-1K for ResNet-50/ConvNeXt-B, and improvements across corruption and OOD benchmarks (Table II).

Significance. The core idea — turning the ordered, path-dependent output of a search-based attribution method into a differentiable training signal via a pairwise ranking loss — is, to my knowledge, new and well-motivated; prior consistency regularizers (GC, CGC) align gradient maps whose faithfulness is questionable, and the authors correctly identify this gap. The formulation is clean, the paper is honest about the lack of a formal submodularity guarantee (§III-B, App. A), the path-constraint ablation (Table VI) is a genuinely informative control, and the empirical sweep is broad (two ViT scales, two CNNs, corruption/OOD suites). Some independent evidence is encouraging: the ObjectNet (+4.3 pp over the fine-tuned control) and ImageNet-C/P gains in Table II are not LIMA-derived and suggest real behavioral change. However, the headline attribution claims rest largely on metrics computed with the same search engine used to generate the training supervision, so the significance of the central claim is currently underdetermined; if confirmed on independent evaluators, this would be a useful contribution to explanation-guided learning.

major comments (3)
  1. [§IV-C, Tables IV–V, Appendix B] The headline attribution results are measured with the same procedure that generates the training supervision. The Submodular Ranking Loss (Eqs. 7–9) explicitly trains the model so that LIMA's greedy Ψ-trajectory on a transformed input follows the transformed query path and so that Ψ_cons reaches τ_cons at the target terminal step; Stability, Insertion, Deletion, PG_LIMA, EPG_LIMA, and Area are then all computed by re-running LIMA (App. B: Insertion AUC is literally the AUC of Ψ_cons along the LIMA path, which L_SelT directly optimizes). A model that merely made the Ψ objective surface more symmetric — without any change in true evidence reliance — would post large gains on these metrics, so the 0.14→0.27 Stability and +52.1% Insertion numbers are partly self-confirming. The paper's own independent check supports this concern: Grad-ECLIP localization transfer in Table V is mixed (ViT-B/1
  2. [§IV-B, Tables II–III] No seeds, error bars, or significance estimates are reported anywhere, and several of the load-bearing accuracy deltas are within typical fine-tuning noise: Table III's mean transformed-input gains are +0.55/+0.33 pp with per-transform regressions on ConvNeXt-B (crop −0.13, rotate −0.16), and Table II's clean-accuracy margins over the strongest baseline are 0.07–0.17 pp. Given that fine-tuning alone already moves these numbers substantially (the '+finetune' row shifts Sketch by −6 pp and ImageNet-A by +3.6 pp relative to pretrained), single-run deltas of this size cannot support the claim of 'improved transformed-input accuracy' without replication. Please report mean ± std over at least 3 seeds for Tables II–IV, or restrict claims to effects that are large relative to run-to-run variance (the ObjectNet and ImageNet-C/P gains may well survive this test).
  3. [§III-C–III-D, Algorithm 1 vs. §IV-B-2] The transformation distribution T used in training is under-specified relative to the evaluation. §III-C describes the loss 'using horizontal flipping as the geometric transformation,' and Algorithm 1 samples t ~ T without defining T, while evaluation uses translation, rotation, affine, and scaling. If training uses only horizontal flip, then the Table III results measure zero-shot transfer to unseen transformation families — which would strengthen the paper and should be stated explicitly; if T includes the evaluated transforms, the equivariance training directly targets the test conditions and this must be disclosed. Either way, the ambiguity is load-bearing for interpreting the robustness claims and must be resolved.
minor comments (6)
  1. [§IV-B-3] A leftover editorial placeholder remains in the text: 'The ImageNet-P metric name must be inserted after it is verified.' This must be fixed; it also leaves the ImageNet-P column of Table II without a defined metric.
  2. [§III-D / Appendix B] Key hyperparameters are not reported: τ_conf, τ_a, and the ranking margin δ appear in Eqs. 7–8 and 11 but no values are given in the main text or the training-details appendix. Since the filtering indicator g (Eq. 11) determines which samples receive regularization, the fraction of training samples passing the filter should also be reported.
  3. [Reference [53]] The Pointing Game metric is cited to a G-CAME paper (Nguyen et al.); the standard citation is Zhang et al., 'Top-Down Neural Attention by Excitation Backprop' (or the original pointing-game source). Please correct.
  4. [Tables V, VII, VIII] Bolding and cross-table consistency need a pass: in Table VII the best PG (0.9220, Conf.+Cons.) is bolded while the text argues the full objective is the best trade-off — please state the bolding convention; and Area for the full-objective ViT-B/16 configuration differs between Table V (31.04%, ImageNet-1K) and Table VII (37.03%, ImageNet-100), which is presumably a dataset difference but should be flagged in the captions to avoid confusion.
  5. [§III-C, Eq. (7)] Notation: q_i(˜r) is defined but the loss is written with Ψ directly; S̃_0 as 'the all-zero starting image' is unclear (all-zero pixel values? empty mask?). Also, p. 2, 'question of this work:how' is missing a space, and Fig. 2's caption refers to L_SelR formulas that should be cross-referenced to Eqs. 7–8.
  6. [§III-E, Table I] The efficiency claims (regularization every 20 iterations 'empirically preserves performance') are asserted without supporting data; a small ablation on the sparsity interval would strengthen this, since it affects how the reported results should be reproduced.

Circularity Check

3 steps flagged

Strongest attribution gains are LIMA-on-LIMA: the ranking loss trains equivariant LIMA trajectories, then Stability/Insertion/Deletion/PG are re-measured with the same LIMA search.

specific steps
  1. self definitional [Sec. III-C Eqs. 7–9; Sec. IV-C Tables IV–V; App. B Evaluation Implementation]
    "The target r̃*_i should score higher than every candidate in C̃_i. ... L_SelR = ... ReLU(q_i(r̃)−q_i(r̃*_i)+δ) ... L_SelT = ReLU(τ_cons − Ψ_cons(S̃_k,F)+δ). The resulting Submodular Ranking Loss is L_SubR = L_SelR + L_SelT. ... Stability ... IoU ... Insertion and Deletion AUC scores: We first employ LIMA to progressively search... Point Game: ... use LIMA to obtain the smallest image region..."

    Training defines success as making the transformed LIMA/Ψ greedy trajectory and terminal stopping set match the query LIMA sequence. Evaluation’s strongest claimed gains (Stability 0.14→0.27, Insertion/Deletion, PG/EPG_LIMA) re-run LIMA and score path/region agreement and LIMA insertion/deletion curves. Those metrics are not independent predictions; they largely re-read the quantity the loss was built to enforce. A model that only symmetrized the LIMA objective surface would post large Table IV–V gains by construction of the train/eval loop.

  2. self citation load bearing [Sec. III-B Preliminary; also Related Work II-A (LIMA [27],[2])]
    "Attribution regularization is effective only when the attribution used as supervision faithfully reflects the evidence underlying the model’s prediction. We therefore adopt search-based subset attribution for its high faithfulness [27], [2]. ... Prior studies, such as LIMA [27], [2] and EAGLE [1], have empirically shown that submodular subset-selection methods can identify compact, class-discriminative evidence with high faithfulness across diverse model architectures [27], [2]."

    The paper’s central premise—that map-consistency is invalid unless attribution is decision-faithful, and that LIMA search supplies such faithfulness—rests on citations whose author sets overlap heavily with this paper (Chen/Cao et al.). That premise is what licenses treating LIMA paths as ground-truth supervision and interpreting LIMA metric gains as evidence of improved decision processes. No independent, non-overlapping establishment of that uniqueness/faithfulness claim is provided in-chain; alternatives are dismissed by this self-lineage.

  3. fitted input called prediction [Sec. III-D Eq. 11 filtering; Abstract/Sec. IV-C claimed Stability & faithfulness improvements]
    "g(X,y)=1[arg max_c[F_θ(X)]_c=y, max_c[F_θ(X)]_c>τ_conf, Area(S_k)<τ_a]. ... our method increases attribution stability from 0.14 to 0.27, improves Insertion by 52.1%, and reduces Deletion by 62.2% ... LIMA attributions from our method reduce Area from 55.01% to 31.04% ... improve LIMA-based PG/EPG from 0.8244/0.5508 to 0.8906/0.6829"

    Supervision is retained only for samples whose LIMA terminal set is already correct, high-confidence, and compact (area filter), i.e., pre-selected easy LIMA successes. The loss then fits the model so transformed inputs reproduce those same compact LIMA sets/orders. Reporting large gains on LIMA Stability, Area, Insertion, Deletion, and PG_LIMA on this regime is statistically close to re-scoring the fitted target distribution, not an out-of-sample prediction of evidence consistency under a disjoint attribution definition. Grad-ECLIP transfer (Table V) is the nearer independent probe and is mixed, underscoring the dependence.

full rationale

The paper’s methodological chain is not a closed mathematical identity: classification loss, geometric transforms, and external benchmarks (ImageNet-C/P, ObjectNet, OOD suites, Grad-ECLIP, CNN transfer) are independent of the ranking objective and can falsify the method. What is partially circular is the load-bearing attribution-quality story. Supervision is defined as filtered LIMA ordered sequences and stopping sets (Sec. III-B–D, Eqs. 7–11); the Submodular Ranking Loss is explicitly a differentiable surrogate that forces the transformed greedy Ψ-path and terminal Ψ_cons to match those targets. The headline metrics in Tables IV–V and App. B—Stability (LIMA region IoU), Insertion/Deletion (LIMA consistency/collaboration curves), and PG/EPG_LIMA—are then obtained by re-running that same search. Large lifts on those metrics are therefore partly entailed by successful optimization of the training surrogate, not independent discovery that the underlying decision evidence changed. Faithfulness of LIMA as the right teaching signal is also imported largely from overlapping-author prior work rather than re-established here. Independent checks are real but weaker or mixed (small transformed-accuracy deltas, mixed Grad-ECLIP localization), so this is partial evaluation/premise circularity (score 5), not a fully self-sealed derivation (8–10).

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

This is an empirical methods paper. The load-bearing commitments are domain assumptions about attribution faithfulness and equivariance as a proxy for evidence consistency, plus hand-chosen search/loss hyperparameters. No new physical entities; the main invented constructs are the Submodular Ranking Loss and the filtered search-derived supervision pipeline built on prior LIMA machinery.

free parameters (5)
  • regularization weight λ = 0.01–1.0 ablated; ~0.10 favored
    Balances classification loss vs Submodular Ranking Loss; ablated in Table VIII, default operating point around 0.10.
  • consistency stop threshold τ_cons = 0.7
    Defines when LIMA query terminates and thus the supervision sequence length/content.
  • confidence filter τ_conf and area filter τ_a = not fully numeric in main text
    Gate which samples contribute attribution regularization via g(X,y) in Eq. 11; directly shape the training distribution of supervision.
  • ranking margin δ = unspecified numeric default
    Margin in selection-ranking and truncation ReLU losses (Eqs. 7–8).
  • Ψ weights λ1,λ2,λ3 and region/search limits = weights all 1.0; 16 regions; pool 4; every 20 iters
    Consistency/confidence/collaboration weights and constrained search (16 regions, pool of 4; sparse every-20-iter regularization) control both supervision generation and compute.
axioms (5)
  • domain assumption Attribution equivariance under label-preserving geometric transforms is a valid training target for consistent evidence reliance (Eqs. 1–2).
    Stated in Sec. I and III-A; standard in explanation-robustness literature but not guaranteed to capture all failure modes.
  • domain assumption Search-based subset attribution (LIMA) is sufficiently faithful that aligning its trajectories regularizes the true decision process, unlike Grad-CAM-style maps.
    Core premise in abstract/intro and Sec. III-B; supported by citations to LIMA/EAGLE rather than a proof in this paper.
  • domain assumption Greedy maximization of Ψ is an adequate practical surrogate even though strict submodularity may fail for neural responses.
    Explicitly conceded in Sec. III-B and Appendix A; method still treats greedy paths as supervision.
  • ad hoc to paper Filtered queries (correct, high-confidence, small-area) are reliable enough to use as annotation-free ground truth for transformed views.
    Eq. 11 and Sec. III-D; necessary for self-supervision quality control.
  • ad hoc to paper Pairwise ranking plus terminal truncation is a faithful differentiable surrogate for discrete path-dependent search equivariance.
    Sec. III-C Eqs. 7–9; central methodological commitment.
invented entities (2)
  • Submodular Ranking Loss (L_SubR = L_SelR + L_SelT) no independent evidence
    purpose: Provide differentiable path-consistency and termination-alignment supervision for discrete attribution search under transforms.
    Defined in Sec. III-C; primary algorithmic contribution.
  • Filtered search-derived supervision sequences from LIMA queries no independent evidence
    purpose: Replace human attribution labels and unfaithful gradient maps with compact ordered evidence targets.
    Query stage in Sec. III-D / Alg. 1; depends on prior LIMA but packaging as training targets is paper-specific.

pith-pipeline@v1.2.0-grok45-kimik3 · 23823 in / 3868 out tokens · 87559 ms · 2026-07-30T10:55:30.775954+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Consistent Evidence, Robust Recognition: Faithful Attribution Regularization under Geometric Transformations." pith.science (2026). https://pith.science/paper/ZGI5DAQD

@misc{pith2026260723835,
  author       = {Pith},
  title        = {Pith review of: Consistent Evidence, Robust Recognition: Faithful Attribution Regularization under Geometric Transformations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZGI5DAQD}},
  note         = {Machine review of arXiv:2607.23835}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Attribution methods are widely used to characterize the evidence underlying model predictions, yet their potential to improve model behavior remains underexplored. Attribution inconsistency under label-preserving geometric transformations may indicate transformation-sensitive evidence reliance, motivating attribution regularization. However, such supervision is valid only when attribution faithfully reflects the evidence driving predictions. Existing self-supervised methods typically align gradient-based maps such as Grad-CAM, whose limited faithfulness means that attribution consistency need not imply consistency of the underlying decision process, leaving transformation robustness unresolved. We propose an annotation-free attribution regularization framework based on submodular search over image regions. By measuring how candidate subsets affect model outputs, the search extracts compact, class-discriminative evidence as search-derived supervision. We further introduce a submodular ranking loss with path-consistency and termination-alignment terms that respectively align spatially corresponding candidate rankings along paired search trajectories and encourage the transformed trajectory to satisfy the stopping criterion at the target terminal step. The loss provides a differentiable surrogate for regularizing both final attributions and the otherwise discrete evidence-selection process. Experiments on ImageNet-100 show that our method substantially improves attribution stability, Insertion, and Deletion on ViT-B/16 with only a 0.28-point accuracy drop, with similar gains on ViT-L/16. On ImageNet-1K, it improves transformed-input accuracy on ResNet-50 and ConvNeXt-B while limiting the clean-accuracy drop to 0.30 points, demonstrating more consistent evidence reliance with minimal performance loss. Code will be released soon.

Figures

Figures reproduced from arXiv: 2607.23835 by Jiawei Liang, Jiazi Hu, Qunli Zhang, Ruoyu Chen, Shangquan Sun, Shiming Liu, Wei Wang, Xianghao Jiao, Xiaochun Cao.

Figure 1
Figure 1. Figure 1: Motivation and overview of the proposed attribution-regularization framework. Under label-preserving geometric transformations, a model may shift its [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the proposed attribution-regularization framework and its two-stage computation. In the query stage, the original image is searched to [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative comparison of the baseline and our model under four geometric transformations. Attribution maps are generated with Grad-ECLIP on [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of attribution results and corresponding saliency maps before and after image flipping using different methods. The model employed is [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Visualization of attribution stability evaluation. We present attribution results, perceptual heatmaps, and Insertion curves obtained via the LIMA [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Sanity check heatmaps based on Grad-ECLIP. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Grad-ECLIP attribution maps before and after the proposed regular [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

55 extracted references · 5 linked inside Pith

  1. [1]

    Where mllms attend and what they rely on: Explaining autoregressive token generation,

    R. Chen, X. Guo, K. Liu, S. Liang, S. Liu, Q. Zhang, L. Wang, H. Zhang, and X. Cao, “Where mllms attend and what they rely on: Explaining autoregressive token generation,” inCVPR, 2026, pp. 17 057–17 066. 1, 4

    2026

  2. [2]

    Less is more: Efficient black-box attribution via minimal interpretable subset selection,

    R. Chen, S. Liang, J. Li, S. Liu, L. Liu, H. Zhang, and X. Cao, “Less is more: Efficient black-box attribution via minimal interpretable subset selection,”arXiv preprint arXiv:2504.00470, 2025. 1, 3, 4

    Pith/arXiv arXiv 2025

  3. [3]

    Axiomatic attribution for deep networks,

    M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,” inICML, 2017. 1

    2017

  4. [4]

    Ig 2: Integrated gradient on iterative gradient path for feature attribution,

    Y . Zhuo and Z. Ge, “Ig 2: Integrated gradient on iterative gradient path for feature attribution,”IEEE Trans. Pattern Anal. Mach. Intell., 2024. 1

    2024

  5. [5]

    Path choice matters for clear attributions in path methods,

    B. Zhang, W. Zheng, J. Zhou, and J. Lu, “Path choice matters for clear attributions in path methods,” inICLR, 2024. 1

    2024

  6. [6]

    Gradient-based visual explanation for transformer-based CLIP,

    C. Zhao, K. Wang, X. Zeng, R. Zhao, and A. B. Chan, “Gradient-based visual explanation for transformer-based CLIP,” inICML, 2024. 1, 6, 7

    2024

  7. [7]

    A unified approach to interpreting model predictions,

    S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,”Advances in neural information processing systems, vol. 30,

  8. [8]

    Going beyond XAI: A systematic survey for explanation-guided learning,

    Y . Gao, S. Gu, J. Jiang, S. R. Hong, D. Yu, and L. Zhao, “Going beyond XAI: A systematic survey for explanation-guided learning,” ACM Comput. Surv., 2024. 1, 3

    2024

  9. [9]

    Generalized semantic contrastive learning via embedding side information for few- shot object detection,

    R. Chen, H. Zhang, J. Li, L. Liu, Z. Huang, and X. Cao, “Generalized semantic contrastive learning via embedding side information for few- shot object detection,”IEEE transactions on Pattern Analysis and Machine Intelligence, 2025. 1

    2025

  10. [10]

    Where not to learn: Prior-aligned training with subset-based attribution constraints for reliable decision-making,

    R. Chen, S. Sun, X. Guo, S. Zhang, K. Liu, S. Liu, Z. Wang, Q. Zhang, H. Zhang, and X. Cao, “Where not to learn: Prior-aligned training with subset-based attribution constraints for reliable decision-making,”arXiv preprint arXiv:2602.07008, 2026. 1

    Pith/arXiv arXiv 2026

  11. [11]

    Underspec- ification presents challenges for credibility in modern machine learning,

    A. D’Amour, K. Heller, D. Moldovan, B. Adlam, B. Alipanahi, A. Beu- tel, C. Chen, J. Deaton, J. Eisenstein, M. D. Hoffmanet al., “Underspec- ification presents challenges for credibility in modern machine learning,” Journal of Machine Learning Research, vol. 23, no. 226, pp. 1–61, 2022. 1

    2022

  12. [12]

    Imagenet-trained cnns are biased towards texture; in- creasing shape bias improves accuracy and robustness,

    R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel, “Imagenet-trained cnns are biased towards texture; in- creasing shape bias improves accuracy and robustness,” inInternational Conference on Learning Representations, 2018. 1

    2018

  13. [13]

    Evaluating the robustness of inter- pretability methods through explanation invariance and equivariance,

    J. Crabb ´e and M. van der Schaar, “Evaluating the robustness of inter- pretability methods through explanation invariance and equivariance,” in NeurIPS, 2023. 1

    2023

  14. [15]

    Hive: Evaluating the human interpretability of visual explanations,

    S. S. Kim, N. Meister, V . V . Ramaswamy, R. Fong, and O. Russakovsky, “Hive: Evaluating the human interpretability of visual explanations,” in European Conference on Computer Vision, 2022, pp. 280–298. 1

    2022

  15. [16]

    Making deep neural networks right for the right scientific reasons by interacting with their explanations,

    P. Schramowski, W. Stammer, S. Teso, A. Brugger, F. Herbert, X. Shao, H.-G. Luigs, A.-K. Mahlein, and K. Kersting, “Making deep neural networks right for the right scientific reasons by interacting with their explanations,”Nature Machine Intelligence, vol. 2, no. 8, pp. 476–486,

  16. [17]

    Right for the right rea- sons: Training differentiable models by constraining their explanations,

    A. S. Ross, M. C. Hughes, and F. Doshi-Velez, “Right for the right rea- sons: Training differentiable models by constraining their explanations,” inIJCAI, 2017. 1

    2017

  17. [18]

    Taking a HINT: leveraging explanations to make vision and language models more grounded,

    R. R. Selvaraju, S. Lee, Y . Shen, H. Jin, S. Ghosh, L. P. Heck, D. Batra, and D. Parikh, “Taking a HINT: leveraging explanations to make vision and language models more grounded,” inICCV, 2019. 1

    2019

  18. [19]

    Improving visual grounding by encouraging consistent gradient-based explanations,

    Z. Yang, K. Kafle, F. Dernoncourt, and V . Ordonez, “Improving visual grounding by encouraging consistent gradient-based explanations,” in CVPR, 2023. 1

    2023

  19. [20]

    Explainable models with consistent inter- pretations,

    V . Pillai and H. Pirsiavash, “Explainable models with consistent inter- pretations,” inAAAI, 2021. 1, 3, 6, 7, 8, 9

    2021

  20. [21]

    Consistent explanations by contrastive learning,

    V . Pillai, S. A. Koohpayegani, A. Ouligian, D. Fong, and H. Pirsiavash, “Consistent explanations by contrastive learning,” inCVPR, 2022. 1, 3, 6, 7, 8, 9

    2022

  21. [22]

    Are data-driven explanations robust against out-of-distribution data?

    T. Li, F. Qiao, M. Ma, and X. Peng, “Are data-driven explanations robust against out-of-distribution data?” inCVPR, 2023. 1, 3

    2023

  22. [23]

    ICEL: learning with inconsistent expla- nations,

    B. Liu, X. Wu, and B. Yuan, “ICEL: learning with inconsistent expla- nations,” inICASSP, 2023, pp. 1–5. 1, 3

    2023

  23. [24]

    Grad-cam: Visual explanations from deep networks via gradient-based localization,

    R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,”Int. J. Comput. Vis., 2020. 1

    2020

  24. [25]

    Sanity checks for saliency maps,

    J. Adebayo, J. Gilmer, M. Muelly, I. J. Goodfellow, M. Hardt, and B. Kim, “Sanity checks for saliency maps,” inNeurIPS, 2018. 1, 9

    2018

  25. [26]

    On the relationship between explanation and prediction: A causal view,

    A. Karimi, K. Muandet, S. Kornblith, B. Sch ¨olkopf, and B. Kim, “On the relationship between explanation and prediction: A causal view,” in ICML, ser. Proceedings of Machine Learning Research, 2023. 1

    2023

  26. [27]

    Less is more: Fewer interpretable region via submodular subset selection,

    R. Chen, H. Zhang, S. Liang, J. Li, and X. Cao, “Less is more: Fewer interpretable region via submodular subset selection,” inICLR, 2024. 2, 3, 4, 12

    2024

  27. [28]

    One explanation is not enough: structured attention graphs for image classification,

    V . Shitole, F. Li, M. Kahng, P. Tadepalli, and A. Fern, “One explanation is not enough: structured attention graphs for image classification,” in NeurIPS, 2021, pp. 11 352–11 363. 3

    2021

  28. [29]

    Identifying important group of pixels using interactions,

    K. Sumiyasu, K. Kawamoto, and H. Kera, “Identifying important group of pixels using interactions,” inCVPR, 2024, pp. 6017–6026. 3

    2024

  29. [30]

    Interpreting object-level foundation models via visual precision search,

    R. Chen, S. Liang, J. Li, S. Liu, M. Li, Z. Huang, H. Zhang, and X. Cao, “Interpreting object-level foundation models via visual precision search,” inCVPR, 2025. 3, 12

    2025

  30. [31]

    Grounding DINO: marrying DINO with grounded pre-training for open-set object detection,

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, Q. Jiang, C. Li, J. Yang, H. Su, J. Zhu, and L. Zhang, “Grounding DINO: marrying DINO with grounded pre-training for open-set object detection,” in ECCV, vol. 15105, 2024, pp. 38–55. 3

    2024

  31. [32]

    Florence-2: Advancing a unified representation for a variety of vision tasks,

    B. Xiao, H. Wu, W. Xu, X. Dai, H. Hu, Y . Lu, M. Zeng, C. Liu, and L. Yuan, “Florence-2: Advancing a unified representation for a variety of vision tasks,” inCVPR, 2024, pp. 4818–4829. 3

    2024

  32. [33]

    On the robustness of interpretabil- ity methods,

    D. Alvarez-Melis and T. S. Jaakkola, “On the robustness of interpretabil- ity methods,”arXiv preprint arXiv:1806.08049, 2018. 3

    Pith/arXiv arXiv 2018

  33. [34]

    Interpretation of neural networks is fragile,

    A. Ghorbani, A. Abid, and J. Zou, “Interpretation of neural networks is fragile,” inProceedings of the AAAI conference on artificial intelligence, vol. 33, no. 01, 2019, pp. 3681–3688. 3 12

    2019

  34. [35]

    Evaluating the robustness of inter- pretability methods through explanation invariance and equivariance,

    J. Crabb ´e and M. van der Schaar, “Evaluating the robustness of inter- pretability methods through explanation invariance and equivariance,” in NeurIPS, 2023, pp. 71 393–71 429. 3

    2023

  35. [36]

    An analysis of approximations for maximizing submodular set functions - I,

    G. L. Nemhauser, L. A. Wolsey, and M. L. Fisher, “An analysis of approximations for maximizing submodular set functions - I,”Math. Program., 1978. 4, 12

    1978

  36. [37]

    Boosting the visual interpretability of CLIP via adversarial fine-tuning,

    G. et al., “Boosting the visual interpretability of CLIP via adversarial fine-tuning,” inICLR, 2025. 6, 7

    2025

  37. [38]

    Imagenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” inCVPR, 2009. 6

    2009

  38. [39]

    Are we done with imagenet?

    L. Beyer, O. J. H ´enaff, A. Kolesnikov, X. Zhai, and A. v. d. Oord, “Are we done with imagenet?”arXiv preprint arXiv:2006.07159, 2020. 6

    Pith/arXiv arXiv 2006

  39. [40]

    Do imagenet classifiers generalize to imagenet?

    B. Recht, R. Roelofs, L. Schmidt, and V . Shankar, “Do imagenet classifiers generalize to imagenet?” inICML, 2019. 6

    2019

  40. [41]

    Benchmarking neural network robustness to common corrup- tions and perturbations,

    H. et al., “Benchmarking neural network robustness to common corrup- tions and perturbations,” inICLR, 2019. 6

    2019

  41. [42]

    Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models,

    A. Barbu, D. Mayo, J. Alverio, W. Luo, C. Wang, D. Gutfreund, J. Tenenbaum, and B. Katz, “Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models,” inNeurIPS, 2019, pp. 9448–9458. 6

    2019

  42. [43]

    Learning robust global representations by penalizing local predictive power,

    H. Wang, S. Ge, Z. C. Lipton, and E. P. Xing, “Learning robust global representations by penalizing local predictive power,” inNeurIPS, 2019. 6

    2019

  43. [44]

    The many faces of robustness: A critical analysis of out-of-distribution generalization,

    D. Hendrycks, S. Basart, N. Mu, S. Kadavath, F. Wang, E. Dorundo, R. Desai, T. Zhu, S. Parajuli, M. Guoet al., “The many faces of robustness: A critical analysis of out-of-distribution generalization,” in CVPR, 2021. 6

    2021

  44. [45]

    Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness,

    R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel, “Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness,” inICLR, 2018. 6

    2018

  45. [46]

    Natural adversarial examples,

    D. Hendrycks, K. Zhao, S. Basart, J. Steinhardt, and D. Song, “Natural adversarial examples,”CVPR, 2021. 6

    2021

  46. [47]

    Leaving reality to imagination: Robust classification via generated datasets,

    H. Bansal and A. Grover, “Leaving reality to imagination: Robust classification via generated datasets,” inICLR Workshop, 2023. 6

    2023

  47. [48]

    Are data-driven explanations robust against out-of-distribution data?

    T. Li, F. Qiao, M. Ma, and X. Peng, “Are data-driven explanations robust against out-of-distribution data?” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 3821–3831. 6, 7

    2023

  48. [49]

    Training for stable explanation for free,

    C. Chen, C. Guo, R. Chen, G. Ma, M. Zeng, X. Liao, X. Zhang, and S. Xie, “Training for stable explanation for free,”NeurIPS, 2024. 6, 7

    2024

  49. [50]

    Explanation-guided adversarial training for robust and interpretable models,

    C. Chen, Y . Chen, S. Lin, D. Hong, S. Wu, X. Liao, and C. Liu, “Explanation-guided adversarial training for robust and interpretable models,”IEEE Transactions on Circuits and Systems for Video Tech- nology, 2026. 6, 7

    2026

  50. [51]

    Rise: Randomized input sampling for explanation of black-box models,

    V . Petsiuk, A. Das, and K. Saenko, “Rise: Randomized input sampling for explanation of black-box models,” inBMVC, 2018, p. 151. 7

    2018

  51. [52]

    Benchmarking deletion metrics with the principled explanations,

    Y . Wang and X. Wang, “Benchmarking deletion metrics with the principled explanations,” inICML, 2024. 7

    2024

  52. [53]

    G-CAME: gaussian-class activation mapping explainer for object detectors,

    Q. K. Nguyen, T. T. H. Nguyen, V . T. K. Nguyen, V . B. Truong, and Q. H. Cao, “G-CAME: gaussian-class activation mapping explainer for object detectors,”CoRR, vol. abs/2306.03400, 2023. 7

    Pith/arXiv arXiv 2023

  53. [54]

    How to probe: Simple yet effective techniques for improving post-hoc explanations,

    S. Gairola, M. B ¨ohle, F. Locatello, and B. Schiele, “How to probe: Simple yet effective techniques for improving post-hoc explanations,” inICLR, 2025. 7

    2025

  54. [55]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inCVPR, 2016, pp. 770–778. 8

    2016

  55. [56]

    A convnet for the 2020s,

    Z. Liu, H. Mao, C.-Y . Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A convnet for the 2020s,” inCVPR, 2022, pp. 11 976–11 986. 8 APPENDIXA INTRODUCTION OFSUBMODULARSELECTION Submodularity is a property of set functions that character- izes diminishing marginal returns. LetVbe a finite ground set and letΨ : 2 V →Rbe a set function. For any two subsets Sa...

    2022