Pith. sign in

REVIEW 3 major objections 4 minor 49 references

A Vision Transformer's per-token influence, measured by deleting each image patch and watching the prediction confidence move, can flag images where the model predicts from background or artifact cues rather than from the object itself.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A token-discarding method for vision transformers measures whether predictions rely on features outside the object's bounding box, identifying spurious correlations and problematic ImageNet classes.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A useful token-dropping audit tool, but the M-TSI metrics are confounded by token-count imbalance and the quantitative claims need a serious rework before they can be taken at face value. the 3 major comments →

arxiv 2509.04009 v1 pith:TF7KJPIE submitted 2025-09-04 cs.CV cs.AI

Detecting Regional Spurious Correlations in Vision Transformers via Token Discarding

Solha Kang , Esla Timothy Anzaku , Wesley De Neve , Arnout Van Messem , Joris Vankerschaver , Francois Rameau , Utku Ozbulak This is my paper
classification cs.CV cs.AI
keywords vision transformersspurious correlationstoken discardingtoken influenceinterpretabilityImageNetself-supervised learningTSI metrics
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a Vision Transformer can be audited for spurious correlations by deleting each image token one at a time, recording how much the prediction confidence drops for the correct class, and comparing the influence of tokens inside versus outside the object's bounding box. The claim is that when outside influence dominates—the proposed TSI score exceeds 1—the model is relying on cues beyond the object, such as watermarks, backgrounds, or secondary objects, and the image can be flagged as spuriously correlated. Using the ImageNet validation set, the authors find that training method matters: the DINO self-supervised model shows lower spuriosity than supervised and MAE models on correctly classified images. They also identify classes and individual images where all tested models persistently rely on non-core features, and they provide a list of those images. If the method holds up, it gives a post-training, per-image way to detect when a transformer is guessing from context rather than looking at the object.

Core claim

The central claim is that token influence maps—formed by removing one patch token at a time from a ViT and measuring the absolute change in prediction confidence for the correct class—are faithful to the model's decision process, and that comparing the influence outside a known bounding box with the influence inside it yields a per-image spuriosity score. Two metrics are proposed: A-TSI, the ratio of mean outside influence to mean inside influence, and M-TSI, the ratio of maximum outside influence to maximum inside influence. A score below 1 means the model leans on the object; a score above 1 means the model leans more on regions outside the object, which the paper interprets as evidence of

What carries the argument

The load-bearing mechanism is leave-one-out token discarding, unique to token-based architectures: each of the 196 patch tokens is removed in turn, and the shift in the model's prediction confidence for the correct class defines that token's influence. The two TSI metrics are ratios over the token sets inside and outside the bounding box: A-TSI averages the influences, M-TSI takes their maxima. The identity that carries the argument is that TSI greater than 1 means outside tokens dominate, which the paper equates with spurious correlation, while TSI less than 1 indicates object-based prediction.

Load-bearing premise

The method assumes the labeled rectangle around the object is complete and correct, so any image patch outside it that sways the prediction counts as a spurious feature; the paper's own examples show that some of those rectangles are wrong or incomplete, which would make high scores misleading.

What would settle it

Build or find a dataset with known, controlled spurious cues, such as watermarks or backgrounds unrelated to the label, run the TSI procedure on a ViT, and check whether TSI greater than 1 flags exactly the images where the model's prediction depends on those cues. If high TSI appears when the model provably uses the object, or fails to appear when it provably uses the cue, the metric's claim is refuted.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A trained ViT can be audited per image without retraining: any image whose TSI exceeds 1 becomes a candidate for relying on background or artifacts rather than the object.
  • Training procedure, not just architecture, determines shortcut reliance: the paper's DINO model shows lower average TSI on correctly classified ImageNet images than the supervised ViT, while MAE shows the highest.
  • Certain ImageNet classes (space bar, ping-pong ball, puck) carry systematically detectable spurious signals across all three models, and the paper supplies an exhaustive list of the flagged images for future dataset curation.
  • The metrics extend beyond bounding boxes to other token annotations and to real-world data: the breast-mass case study uses the breast region and flags chest fat tissue as spuriously influential.
  • When annotations or compute are scarce, attention maps can substitute for token influence only partially: A-TSI from attention maps correlates moderately to strongly with the token-influence version, while M-TSI does not.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the bounding box is wrong or incomplete, TSI above 1 conflates a model's spurious reliance with an annotation error; a practical next step would be to filter or re-annotate the flagged images and measure how many high-TSI cases disappear.
  • The leave-one-out procedure could be accelerated by grouping tokens or using a cheaper attribution proxy, which would make the audit feasible on larger models without changing the core claim.
  • A controlled experiment with synthetic background shifts could turn TSI into a quantitative calibration tool: the fraction of TSI greater than 1 images should track the degree of background-label correlation.
  • Because the method only requires tokenization, the same audit could be applied to multimodal or NLP transformers where tokens correspond to words or patches, provided an annotation defines the 'core' region.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a post-hoc method for detecting spurious correlations in Vision Transformers (ViTs). For a given image, each of the 196 tokens is discarded individually and the absolute change in prediction confidence for the correct class is recorded, producing a token influence map. Using a bounding box annotation, tokens are partitioned into inside (Bin) and outside (Bout), and two metrics are defined: A-TSI, the ratio of average outside influence to average inside influence, and M-TSI, the ratio of the maximum outside influence to the maximum inside influence. The authors argue that TSI > 1 indicates that the model relies more on features outside the object bounding box, i.e., potential spurious correlations. Experiments on ImageNet with supervised ViT-B/16, DINO, and MAE report that DINO generally has lower TSI, that misclassified images have higher TSI, and that certain classes (e.g., ping-pong ball, space bar) consistently show high M-TSI. A case study on invasive breast mass classification in mammography is also presented. The paper additionally explores attention maps and GradCAM as cheaper proxies and concludes that M-TSI is the more appropriate metric when token influence maps are available.

Significance. If the proposed interpretation is valid, the method offers a simple, architecture-specific debugging tool that requires no fitted parameters and leverages the token-based structure of ViTs. The token influence computation is transparent and the qualitative examples—watermarks in ImageNet and chest-fat tissue in mammography—are visually compelling. The large-scale comparison across three training regimes and the public release of class-level TSI scores would be useful resources. However, the central claim depends on two untested assumptions: that bounding boxes correctly delineate core features, and that max-ratio comparisons are meaningful when the inside and outside token sets differ greatly in size. Both assumptions are load-bearing for the paper's main conclusions, and the current evidence does not yet separate genuine spurious correlations from annotation noise or from extreme-value artifacts.

major comments (3)
  1. [Eq. (2), Table 2, Section 4.3] M-TSI is systematically inflated by the imbalance between the number of outside and inside tokens. In Eq. (2), the maximum over Bout is compared with the maximum over Bin with no correction for |Bout| vs |Bin|. For small objects, Bout can be an order of magnitude larger than Bin, so even under a null model where token influence is independent of the bounding box, max_{Bout} z is stochastically larger than max_{Bin} z. Table 2 shows exactly the predicted signature: for the 1-40 token group, M-TSI is the highest for all three models in both DC and DI, while A-TSI is not elevated. The paper interprets this as evidence that small objects are more prone to spurious correlations, but it is equally explained by count imbalance. Because Section 4.3's threshold interpretation and Table 4's class rankings rely on M-TSI, and Section 5.2 recommends M-TSI as the more appropriate metric, the central e
  2. [Section 4.2, Table 1] The comparison between token influence and GradCAM is self-validating. Tokens are ranked by the confidence drop they cause when removed, and then the evaluation masks the top-ranked tokens and measures the resulting confidence drop. By construction, the method that selects tokens according to that same removal-based criterion will outperform a method trained on a different notion of importance (GradCAM). Table 1 therefore does not demonstrate that token influence identifies 'important' tokens in any independent sense; it only shows that the metric is consistent with its own definition. To validate the token influence maps, the authors need an external ground truth, such as human-annotated core regions, or they should evaluate whether token influence localizes objects better than GradCAM with respect to bounding boxes on a held-out set with reliable annotations. Without such a comparison,
  3. [Section 3.3.2, Fig. 10] The interpretation of TSI as a measure of spurious correlation assumes that the bounding box is a correct and complete delineation of the object of interest. The paper itself acknowledges in Fig. 10(a) that ImageNet bounding boxes suffer from label inconsistency and incomplete coverage. For such images, a high TSI reflects annotation error rather than model reliance on spurious features. The paper does not quantify how many images in the analyzed subsets are affected by this issue, nor does it report sensitivity of the results to the bbox quality. This is load-bearing because Section 4.3 and Table 4 use the bbox-vs-outside split to define spuriosity. The authors should either filter or re-label problematic annotations, or provide a robustness analysis showing that the main conclusions (e.g., class rankings, model comparisons) are unchanged when only high-confidence, manually verified bou
minor comments (4)
  1. [Section 6] The text reads 'As described in Section 4.1, two major limitations...' but the limitations are actually discussed in Section 5.1. Please fix the cross-reference.
  2. [Fig. 5] In the caption, 'can be entirely without causing missingness bias' appears to be missing a word; likely 'can be entirely removed' or 'can be entirely discarded'.
  3. [Table 2] In the DI, 121-160 tokens row for MAE, the reported standard deviation (0.76) is much larger than the mean (0.40) and inconsistent with the other models' standard deviations in the same row. Please verify this value.
  4. [Section 3.3.1] The definition z_k = |yhat - yhat^{(-k)}| uses 'the correct class' confidence. For images in DI (misclassified by at least one model), the correct-class confidence may be very small; the interpretation of z_k for such images should be clarified, as a small absolute change might still be relatively important for the model's decision.

Circularity Check

1 steps flagged

Token-influence validation is circular (same drop metric used for selection and evaluation); the central TSI bbox ratio itself is not circular.

specific steps
  1. self definitional [Section 3.3.1 (definition of z_k) and Section 4.2 (evaluation, Table 1)]
    "zk = |ŷ − ŷ(−k)|, with ŷ(−k) = gθ(X(−k)) being the prediction confidence of the image with one fewer token ... we mask the tokens with the highest importance scores. ... we record the resulting change in model’s prediction confidence for the correct class after the masking is applied."

    Token importance z_k is defined as the absolute drop in correct-class confidence when token k is discarded. In Section 4.2, tokens are selected by 'highest importance scores' (i.e., largest z_k) and then the drop after masking is measured to claim that token influence 'more effectively captures tokens critical to the model's decision-making process' than GradCAM. Because selection and evaluation use the same per-token confidence drop, the comparison is guaranteed by construction: the chosen tokens are exactly those that maximize the measured quantity, so any alternative attribution method will appear worse unless it selects the same tokens. This is not an independent faithfulness test.

full rationale

The central TSI computation (Eqs. 1-2) is a direct ratio of influence sums/maxima over bbox-derived token sets; it involves no fitted parameters, no imported uniqueness theorem, and no self-citation chain. The bbox-as-ground-truth assumption and M-TSI's sensitivity to token-count imbalance are validity concerns, not circularity. The only circular step is the Section 4.2 validation of token influence maps against GradCAM, where the evaluation metric is identical to the definition of token importance. Self-citations (e.g., [29] for dataset splitting, [30]/[32] for ImageNet issues) are non-load-bearing. Hence partial circularity in a supporting claim, while the main TSI proposal retains independent content.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 0 invented entities

The paper introduces no physical entities. It relies on three domain assumptions about bounding-box completeness, the unbiasedness of token discarding, and the validity of absolute confidence change as influence. The only hand-chosen quantities are analysis bins and top-k values, which do not violate the central metric but shape the reported empirical conclusions.

free parameters (3)
  • Bounding box token coverage groups = 1-40, 41-80, 81-120, 121-160
    Hand-chosen bins in Table 2 used to stratify TSI by object size; not fit to data but affect reported patterns.
  • Confidence bins for DI analysis = 0-25%, 25-50%, 50-75%, 75-100%
    Hand-chosen bins in Table 3 used to relate prediction confidence to TSI; arbitrary choices that shape the trend shown.
  • Top-k attention tokens for annotation-free proxy = 5, 10, 20, 40, 80
    Hand-chosen values in Section 5.1.2 for selecting Bin from attention maps; no justification of the specific k values.
axioms (3)
  • domain assumption ImageNet bounding boxes accurately and completely delineate the object of interest
    All TSI computation in Section 3.3.2 uses bbox membership to define core (inside) and spurious (outside) features; the paper itself shows label inconsistencies in Fig. 10a.
  • domain assumption Token discarding does not introduce missingness bias in ViTs
    Section 3.3.1 asserts ViTs offer a 'more natural implementation of missingness' through token discarding, but this is not rigorously demonstrated and could affect the faithfulness of the influence maps.
  • domain assumption Absolute confidence change |y_hat - y_hat(-k)| is a faithful measure of token influence
    Eq. (1) and (2) build entirely on z_k = |y_hat - y_hat(-k)|; the paper does not validate that this magnitude, rather than a signed or calibrated change, correctly identifies causal token importance.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Detecting Regional Spurious Correlations in Vision Transformers via Token Discarding." pith.science (2026). https://pith.science/paper/TF7KJPIE

@misc{pith2026250904009,
  author       = {Pith},
  title        = {Pith review of: Detecting Regional Spurious Correlations in Vision Transformers via Token Discarding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TF7KJPIE}},
  note         = {Machine review of arXiv:2509.04009}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Due to their powerful feature association capabilities, neural network-based computer vision models have the ability to detect and exploit unintended patterns within the data, potentially leading to correct predictions based on incorrect or unintended but statistically relevant signals. These clues may vary from simple color aberrations to small texts within the image. In situations where these unintended signals align with the predictive task, models can mistakenly link these features with the task and rely on them for making predictions. This phenomenon is referred to as spurious correlations, where patterns appear to be associated with the task but are actually coincidental. As a result, detection and mitigation of spurious correlations have become crucial tasks for building trustworthy, reliable, and generalizable machine learning models. In this work, we present a novel method to detect spurious correlations in vision transformers, a type of neural network architecture that gained significant popularity in recent years. Using both supervised and self-supervised trained models, we present large-scale experiments on the ImageNet dataset demonstrating the ability of the proposed method to identify spurious correlations. We also find that, even if the same architecture is used, the training methodology has a significant impact on the model's reliance on spurious correlations. Furthermore, we show that certain classes in the ImageNet dataset contain spurious signals that are easily detected by the models and discuss the underlying reasons for those spurious signals. In light of our findings, we provide an exhaustive list of the aforementioned images and call for caution in their use in future research efforts. Lastly, we present a case study investigating spurious signals in invasive breast mass classification, grounding our work in real-world scenarios.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 36 canonical work pages · 4 internal anchors

  1. [1]

    Sanity checks for saliency maps

    Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Advances in neural information processing systems, 31, 2018

    2018

  2. [2]

    Invari- ant risk minimization

    Martin Arjovsky, L´ eon Bottou, Ishaan Gul- rajani, and David Lopez-Paz. Invari- ant risk minimization. arXiv preprint arXiv:1907.02893, 2019

    Pith/arXiv arXiv 1907

  3. [3]

    Towards improved input masking for convo- lutional neural networks

    Sriram Balasubramanian and Soheil Feizi. Towards improved input masking for convo- lutional neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 1855–1865, 2023

    work page 2023

  4. [4]

    Are we done with imagenet? arXiv preprint arXiv:2006.07159, 2020

    Lucas Beyer, Olivier J H´ enaff, Alexander Kolesnikov, Xiaohua Zhai, and A¨ aron van den Oord. Are we done with imagenet? arXiv preprint arXiv:2006.07159, 2020

    Pith/arXiv arXiv 2006

  5. [5]

    Token merging: Your vit but faster

    Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. Token merging: Your vit but faster. arXiv preprint arXiv:2210.09461 , 2022

    Pith/arXiv arXiv 2022

  6. [6]

    Emerging properties in self-supervised vision transform- ers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv´ e J´ egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transform- ers. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision , pages 9650–9660, 2021

    work page 2021

  7. [7]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020

    Pith/arXiv arXiv 2010

  8. [8]

    Shortcut learning of large language models in natural language understanding: A survey

    M Du, F He, N Zou, D Tao, and X Hu. Shortcut learning of large language models in natural language understanding: A survey. preprint. arXiv preprint arXiv:2208.11857 , 2022

    Pith/arXiv arXiv 2022

  9. [9]

    Domain-adversarial training of neural networks

    Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Fran¸ cois Laviolette, Mario March, and Victor Lempitsky. Domain-adversarial training of neural networks. Journal of machine learning research, 17(59):1–35, 2016

    work page 2016

  10. [10]

    Shortcut learning in deep neu- ral networks

    Robert Geirhos, J¨ orn-Henrik Jacobsen, Clau- dio Michaelis, Richard Zemel, Wieland Bren- del, Matthias Bethge, and Felix A Wich- mann. Shortcut learning in deep neu- ral networks. Nature Machine Intelligence , 2(11):665–673, 2020

    work page 2020

  11. [11]

    Are vision transformers robust to spurious corre- lations? International Journal of Computer Vision, 132(3):689–709, 2024

    Soumya Suvra Ghosal and Yixuan Li. Are vision transformers robust to spurious corre- lations? International Journal of Computer Vision, 132(3):689–709, 2024

    work page 2024

  12. [12]

    Which tokens to use? investigating token reduction in vision transformers

    Joakim Bruslund Haurum, Sergio Escalera, Graham W Taylor, and Thomas B Moes- lund. Which tokens to use? investigating token reduction in vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 773– 783, 2023

    work page 2023

  13. [13]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yang- hao Li, Piotr Doll´ ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16000–16009, 2022

    work page 2022

  14. [14]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2016

    work page 2016

  15. [15]

    Missingness bias in model debugging

    Saachi Jain, Hadi Salman, Eric Wong, Pengchuan Zhang, Vibhav Vineet, Sai Vem- prala, and Aleksander Madry. Missingness bias in model debugging. arXiv preprint arXiv:2204.08945, 2022

    Pith/arXiv arXiv 2022

  16. [16]

    Feature alignment and restora- tion for domain generalization and adapta- tion

    Xin Jin, Cuiling Lan, Wenjun Zeng, and Zhibo Chen. Feature alignment and restora- tion for domain generalization and adapta- tion. arXiv preprint arXiv:2006.12009 , 2020

    Pith/arXiv arXiv 2006

  17. [17]

    Contrastive self- supervised learning: a survey on different architectures

    Adnan Khan, Sarah AlBarri, and Muham- mad Arslan Manzoor. Contrastive self- supervised learning: a survey on different architectures. In 2022 2nd International Conference on Artificial Intelligence (ICAI) , pages 1–6. IEEE, 2022

    work page 2022

  18. [18]

    The (un) reliability of saliency methods

    Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T Sch¨ utt, Sven D¨ ahne, Dumitru Erhan, and 18 Been Kim. The (un) reliability of saliency methods. Explainable AI: Interpreting, explaining and visualizing deep learning , pages 267–280, 2019

    work page 2019

  19. [19]

    Last layer re- training is sufficient for robustness to spurious correlations

    Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re- training is sufficient for robustness to spurious correlations. arXiv preprint arXiv:2204.02937, 2022

    Pith/arXiv arXiv 2022

  20. [20]

    Gradient-based learning applied to document recognition

    Yann LeCun, L´ eon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE , 1998

    work page 1998

  21. [21]

    Just train twice: Improving group robustness without training group information

    Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In Inter- national Conference on Machine Learning , pages 6781–6792. PMLR, 2021

    work page 2021

  22. [22]

    Deep learning face attributes in the wild

    Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE interna- tional conference on computer vision , pages 3730–3738, 2015

    work page 2015

  23. [23]

    Beyond attentive tokens: Incorporating token importance and diversity for efficient vision transformers

    Sifan Long, Zhen Zhao, Jimin Pi, Shengsheng Wang, and Jingdong Wang. Beyond attentive tokens: Incorporating token importance and diversity for efficient vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10334–10343, 2023

    work page 2023

  24. [24]

    Domain-invariant Feature Exploration for Domain Generalization

    Wang Lu, Jindong Wang, Haoliang Li, Yiqiang Chen, and Xing Xie. Domain- invariant feature exploration for domain gen- eralization. arXiv preprint arXiv:2207.12020, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  25. [25]

    Faithfulness Measurable Masked Language Models

    Andreas Madsen, Siva Reddy, and Sarath Chandar. Faithfulness measurable masked language models. arXiv preprint arXiv:2310.07819, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  26. [26]

    Hard imagenet: Segmentations for objects with strong spurious cues

    Mazda Moayeri, Sahil Singla, and Soheil Feizi. Hard imagenet: Segmentations for objects with strong spurious cues. Advances in Neural Information Processing Systems , 35:10068–10077, 2022

    work page 2022

  27. [27]

    Vindr-mammo: A large-scale benchmark dataset for computer-aided diagnosis in full- field digital mammography

    Hieu T Nguyen, Ha Q Nguyen, Hieu H Pham, Khanh Lam, Linh T Le, Minh Dao, and Van Vu. Vindr-mammo: A large-scale benchmark dataset for computer-aided diagnosis in full- field digital mammography. Scientific Data , 10(1):277, 2023

    work page 2023

  28. [28]

    Hidden stratification causes clinically meaningful fail- ures in machine learning for medical imaging

    Luke Oakden-Rayner, Jared Dunnmon, Gus- tavo Carneiro, and Christopher R´ e. Hidden stratification causes clinically meaningful fail- ures in machine learning for medical imaging. In Proceedings of the ACM conference on health, inference, and learning , pages 151– 159, 2020

    work page 2020

  29. [29]

    Selection of Source Images Heavily Influences the Effectiveness of Adversarial Attacks

    Utku Ozbulak, Esla Timothy Anzaku, Wesley De Neve, and Arnout Van Messem. Selection of source images heavily influences the effec- tiveness of adversarial attacks. arXiv preprint arXiv:2106.07141, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  30. [30]

    Utku Ozbulak, Esla Timothy Anzaku, Solha Kang, Wesley De Neve, and Joris Vanker- schaver. Self-supervised benchmark lot- tery on imagenet: Do marginal improve- ments translate to improvements on similar datasets? In 2024 International Joint Con- ference on Neural Networks (IJCNN) , pages 1–8. IEEE, 2024

    work page 2024

  31. [31]

    Know your self-supervised learning: A survey on image-based generative and discriminative training

    Utku Ozbulak, Hyun Jung Lee, Beril Boga, Esla Timothy Anzaku, Ho-min Park, Arnout Van Messem, Wesley De Neve, and Joris Vankerschaver. Know your self-supervised learning: A survey on image-based generative and discriminative training. Transactions on Machine Learning Research, 2023

    work page 2023

  32. [32]

    Evaluating Adversarial Attacks on ImageNet: A Reality Check on Misclassification Classes

    Utku Ozbulak, Maura Pintor, Arnout Van Messem, and Wesley De Neve. Eval- uating adversarial attacks on imagenet: A reality check on misclassification classes. arXiv preprint arXiv:2111.11056 , 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  33. [33]

    ia − red2: Interpretability-aware redundancy reduction for vision transform- ers

    Bowen Pan, Rameswar Panda, Yifan Jiang, Zhangyang Wang, Rogerio Feris, and Aude Oliva. ia − red2: Interpretability-aware redundancy reduction for vision transform- ers. Advances in Neural Information Process- ing Systems , 34:24898–24911, 2021

    work page 2021

  34. [34]

    Automated classifi- cation of model errors on imagenet

    Momchil Peychev, Mark M¨ uller, Marc Fis- cher, and Martin Vechev. Automated classifi- cation of model errors on imagenet. Advances in Neural Information Processing Systems , 36:36826–36885, 2023

    work page 2023

  35. [35]

    Finding and fixing spu- rious patterns with explanations

    Gregory Plumb, Marco T´ ulio Ribeiro, and Ameet Talwalkar. Finding and fixing spu- rious patterns with explanations. CoRR, abs/2106.02112, 2021

    Pith/arXiv arXiv 2021

  36. [36]

    Dynamicvit: Efficient vision transformers 19 with dynamic token sparsification

    Yongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu, Jie Zhou, and Cho-Jui Hsieh. Dynamicvit: Efficient vision transformers 19 with dynamic token sparsification. Advances in neural information processing systems , 34:13937–13949, 2021

    work page 2021

  37. [37]

    Learning to merge tokens in vision transformers

    Cedric Renggli, Andr´ e Susano Pinto, Neil Houlsby, Basil Mustafa, Joan Puigcerver, and Carlos Riquelme. Learning to merge tokens in vision transformers. arXiv preprint arXiv:2202.12015, 2022

    Pith/arXiv arXiv 2022

  38. [38]

    ” why should i trust you?” explaining the predictions of any classifier

    Marco Tulio Ribeiro, Sameer Singh, and Car- los Guestrin. ” why should i trust you?” explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD inter- national conference on knowledge discovery and data mining , pages 1135–1144, 2016

    work page 2016

  39. [39]

    Berg, and Li Fei-Fei

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexan- der C. Berg, and Li Fei-Fei. ImageNet large scale visual recognition challenge. International Journal of Computer Vision , 115(3):211–252, 2015

    work page 2015

  40. [40]

    Distribu- tionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization

    Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distribu- tionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731, 2019

    Pith/arXiv arXiv 1911

  41. [41]

    Grad-cam: Visual explanations from deep networks via gradient-based localization

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on com- puter vision , pages 618–626, 2017

    work page 2017

  42. [42]

    Salient imagenet: How to discover spurious fea- tures in deep learning? arXiv preprint arXiv:2110.04301, 2021

    Sahil Singla and Soheil Feizi. Salient imagenet: How to discover spurious fea- tures in deep learning? arXiv preprint arXiv:2110.04301, 2021

    Pith/arXiv arXiv 2021

  43. [43]

    Axiomatic attribution for deep networks

    Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning , pages 3319–3328. PMLR, 2017

    work page 2017

  44. [44]

    Rethinking the inception architec- ture for computer vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architec- ture for computer vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2016

    work page 2016

  45. [45]

    Over- coming simplicity bias in deep networks using a feature sieve

    Rishabh Tiwari and Pradeep Shenoy. Over- coming simplicity bias in deep networks using a feature sieve. In International Conference on Machine Learning , pages 34330–34343. PMLR, 2023

    work page 2023

  46. [46]

    Adversarial discrimina- tive domain adaptation

    Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discrimina- tive domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7167–7176, 2017

    work page 2017

  47. [47]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Par- mar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    work page 2017

  48. [48]

    Gen- eralizing to unseen domains: A survey on domain generalization

    Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, and S Yu Philip. Gen- eralizing to unseen domains: A survey on domain generalization. IEEE transactions on knowledge and data engineering , 35(8):8052– 8072, 2022

    work page 2022

  49. [49]

    Visu- alizing and understanding convolutional net- works

    Matthew D Zeiler and Rob Fergus. Visu- alizing and understanding convolutional net- works. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzer- land, September 6-12, 2014, Proceedings, Part I 13 , pages 818–833. Springer, 2014. 20

    work page 2014

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.