Are Vision Transformer Representations Semantically Meaningful? A Case Study in Medical Imaging

Chashi Mahiul Islam; Montasir Shams; Phat Tran; Shaeke Salman; Xiuwen Liu

REVIEW 4 major objections 5 minor 31 references

Are Vision Transformer Representations Semantically Meaningful? A Case Study in Medical Imaging

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that vision transformer representations in medical imaging are not semantically meaningful: a projected gradient-based procedure can reshape the embedding of a normal image to match a diseased one with imperceptible pixel…

desk verdict A believable case study of representation-level adversarial vulnerability in medical ViTs, but the sweeping 'not semantically meaningful' conclusion is not supported by the evidence as presented. read the letter →

arxiv 2507.01788 v2 pith:WN65RPO2 submitted 2025-07-02 cs.CV cs.AI

Montasir Shams , Chashi Mahiul Islam , Shaeke Salman , Phat Tran , Xiuwen Liu This is my paper

classification cs.CVcs.AI

keywords visiontransformersemanticmeaningfulnessrepresentationmatchingmedicalimagingadversarialperturbationembeddingspacefundusimagesmodelinterpretability

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to test whether the internal embeddings of vision transformers, when used for medical image classification, are organized by clinically meaningful concepts. It introduces a projected gradient-based procedure, PRM, that moves a source image's embedding to match that of a different-class target image while limiting pixel changes to a tiny allowed range. Under those constraints the images remain visually indistinguishable by standard metrics (PSNR above roughly 42 dB, SSIM above 0.97), yet the classifier's accuracy drops by more than 60% and the embedding of a normal image can be made to match that of a diseased one. The authors conclude that ViT representations lack semantic grounding: similar-looking images can carry very different embeddings while clinically different images can carry nearly identical ones, which they flag as a critical obstacle to safe deployment in medical settings.

What carries the argument

The load-bearing object is the Projected Representation Matching (PRM) procedure: an iterative gradient-descent optimization on the input image that minimizes half the squared Euclidean distance between the current embedding and a target embedding, followed by clipping each pixel change into an epsilon-ball. The success of the procedure depends on the structure of the transformer's Jacobian of the representation mapping with respect to the input; with that structure, the optimization converges reliably across learning rates, datasets, and architectures.

What would settle it

A perceptual study in which ophthalmologists or radiologists are shown original and PRM-modified fundus images in a same-different or forced-choice task; if experts reliably detect the modified image above chance, the imperceptibility premise fails and the representation change could be attributed to altered clinically relevant content. Conversely, a representation-probing study showing that the modified images' embeddings sit near the target-class centroid while the images remain clinically normal would strengthen the paper's claim.

Watch

Extended reading notes

Core claim

The paper claims to demonstrate, systematically and for the first time, that vision transformer representations for medical image classification are not semantically meaningful. The central experiment uses Projected Representation Matching (PRM), which performs gradient descent on the input image to minimize the squared Euclidean distance between the current embedding and a target embedding, while clamping each pixel change into an epsilon-ball. On the APTOS2019 diabetic retinopathy grading dataset, the fine-tuned MIL-VT model's accuracy falls from 81.3% to about 5–6% after matching; on RFMiD2020, it falls from 87.2% to about 27%, with average PSNR above 42 dB and SSIM above 0.97 between original and optimized images. Cosine similarity between the optimized embedding and the target embedding reaches 0.77–0.93, while similarity to the original embedding stays near 0.2–0.37. The authors interpret this as evidence that the representation space is not organized around clinically relevant concepts and that the vulnerability lives in the input-to-representation mapping itself, not merely in the classification head.

Load-bearing premise

The central conclusion depends on the premise that a PSNR above about 40 dB and an SSIM above about 0.97 guarantee the modified images are imperceptibly different to medical experts, so that a changed embedding reveals a lack of semantic grounding rather than a change in clinically relevant content.

Editorial extensions

If this is right

Deployed ViTs for medical triage could potentially be steered from a normal to a disease label by pixel perturbations that standard quality metrics deem invisible.
Classification accuracy alone is insufficient as a safety metric; the embedding geometry itself would need validation before clinical use.
Standard adversarial training aimed at the output layer will not repair the defect, because the vulnerability is located in the input-to-representation mapping.
Detection schemes based on label instability under added Gaussian noise may provide a practical defense in the short term.
Other vision models with a similar patch-embedding and self-attention structure are likely vulnerable in the same way, since the paper finds the behavior across MIL-VT, MedViT, and several datasets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural next experiment the paper does not run: apply the same PRM procedure to CNN-based medical classifiers on the same fundus datasets; if comparable embedding matching succeeds with similar PSNR, then the phenomenon is a general property of learned input-to-representation maps, not a transformer-specific failure.
The paper operationalizes semantic meaningfulness purely as robust distinctiveness. A stricter test would check whether the perturbed pixel region corresponds to a clinically salient structure such as microaneurysms or exudates, which would separate the claim that representations are non-robust from the claim that they are non-semantic.
The fine-tuned MIL-VT used 70% of each small dataset for training; whether the claim extends to ViTs trained from scratch on larger medical corpora, or to other modalities such as histopathology or radiology, remains untested.
If the finding generalizes, representation-level certification—for instance Lipschitz-like bounds on the embedding map or consistency checks across semantic augmentations—would be a more direct safety requirement than traditional output-level adversarial training.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this paper is a competent case study showing that white-box gradient attacks can move ViT embeddings in medical imaging with small pixel perturbations, and that this collapses classification accuracy. The experiments are straightforward and the results are believable as far as they go. But the paper overreaches when it concludes that representations 'are not semantically meaningful.' That conclusion requires evidence the paper doesn't provide.

What's actually new: applying representation-matching attacks (the PRM method, from the authors' earlier work) to medical ViTs (MIL-VT, MedViT) on fundus and MedMNIST data. The demonstration that accuracy drops by 60–76% under a small L-infinity budget is a useful data point for anyone deploying these models. The PSNR/SSIM numbers are reported honestly, and the qualitative figures give a sense of the effect. Credit where due: the method is clearly described, the experiments are reproducible in principle, and the authors don't hide the fact that the core optimization is from their prior paper.

The soft spots are significant. First, the central claim depends on the assumption that PSNR > 42 dB and SSIM > 0.97 mean the perturbed images are clinically unchanged. PSNR/SSIM are global image-similarity metrics; they are insensitive to small localized changes a clinician would care about (a microaneurysm, a small hemorrhage). No human perceptual study is reported, so 'imperceptible to medical experts' is asserted, not shown. Second, there's no baseline measuring natural cross-class embedding distances. Without knowing how far apart different classes are in the original embedding space, the statement that 'different semantic classes can have nearly identical representations' is not supported; the optimized images are the result of an optimization procedure designed to make embeddings similar, so that part is true by construction. Third, the paper reports no error bars or standard deviations, and the subset sizes for the accuracy tables are not stated. Fourth, the 'first to systematically demonstrate' claim is not credible given the existing adversarial robustness literature, and the paper even cites the relevant work.

Who is this for? A practitioner evaluating ViT robustness in medical imaging might read it as a warning. A researcher working on representation interpretability would find the framing too loose. The experiments are fine for a workshop or a short conference paper, but as a claim about semantic grounding, it needs a lot more support.

My recommendation: it deserves a referee if submitted to a venue that values empirical robustness case studies, but the referee should insist on either toning down the semantic meaningfulness conclusion or adding the missing baselines and a perceptual study. If the authors reframe it as 'representation-level adversarial vulnerability in medical ViTs,' the paper is solid enough to publish.

Referee Report

4 major / 5 minor

Summary. The paper proposes Projected Representation Matching (PRM), a gradient-based optimization that perturbs an input image within an L-infinity bound so that its ViT embedding is driven toward a target image's embedding. The authors apply PRM to MIL-VT and MedViT models on the APTOS2019, RFMiD2020, bloodMNIST, and dermaMNIST datasets, reporting large drops in classification accuracy, high cosine similarity between the optimized and target embeddings, and high PSNR/SSIM between the original and optimized images. From these results they conclude that ViT representations are not semantically meaningful, arguing that imperceptible changes can yield very different representations and that different semantic classes can have nearly identical embeddings. The paper also sketches a Gaussian-noise-based detection method as a potential mitigation in Section V.

Significance. If the central claim were established, the paper would have substantial implications for the deployment of vision transformers in medical imaging, where representation stability and distinctiveness are clinically relevant. The proposed PRM framework is a simple and potentially useful tool for stress-testing representation-level robustness, and the paper provides quantitative numbers (accuracy, match success rate, PSNR/SSIM, cosine similarity) that would allow comparisons with other attack methods. However, the evidence as presented does not support the strong conclusion about a lack of semantic meaningfulness: the experiments demonstrate a white-box adversarial capability, but they do not measure the natural distribution of cross-class embedding distances, and the imperceptibility premise relies on PSNR/SSIM rather than human or clinical evaluation. The contribution is therefore best viewed as an adversarial robustness study, not a demonstration that ViT representations lack semantic grounding.

major comments (4)

[Section IV-C, Table II] The claim that the optimized images are perceptually and clinically equivalent to the originals rests solely on PSNR and SSIM; no human or clinician perceptual study is reported. Because PSNR is a global pixel-error average and SSIM is not sensitive to localized changes relevant to fundus grading (e.g., microaneurysms or small hemorrhages), the reported values do not establish the 'not altered to any medical experts' condition stated in Section III-B. Without this validation, the observed embedding and label changes could reflect clinically meaningful content changes rather than a lack of semantic grounding.
[Section IV-C, Table III] The observation that optimized images have high cosine similarity to target embeddings is guaranteed by the optimization objective in Eq. (1); it does not support the inference that different semantic classes naturally have nearly identical embeddings. The paper does not report the distribution of cosine similarities between original images of different classes (e.g., no_DR vs severe), which is the relevant baseline for the 'nearly identical representations' claim. Table III only compares original-vs-optimized and optimized-vs-target similarities, so the key claim about colliding semantic classes is not supported by the reported data.
[Section IV-B, Fig. 2] The pairs (a)-(d), (b)-(e), and (c)-(f) are presented as 'visually indistinguishable' images with 'significantly different representations,' but these pairs are original and optimized images, so the large representation differences shown in the PCA projections are produced by the PRM optimization itself and are not evidence of a general instability of ViT representations. Without baseline projections of randomly sampled original image pairs (both same-class and different-class), the qualitative figure is anecdotal and does not quantify the claimed effect.
[Section V] The proposed Gaussian-noise detection mitigation is described qualitatively with no experimental results; including an untested mitigation in a paper claiming a fundamental vulnerability makes the paper hard to evaluate. Either remove this section or provide quantitative detection results with a ROC curve or accuracy table across noise levels and datasets.

minor comments (5)

[Section IV-A] The description of the datasets and implementation is incomplete; it does not state image resolution after preprocessing, class balance, or the fine-tuning details (number of epochs, optimizer, learning rate schedule, data augmentation). This limits reproducibility.
[Section IV-C, Table I] The 'bloodMNIST MedViT-L- 0.10' row is ambiguous about which model variant and which embedding type (ViT or MIL) was used; specify these details in the table or caption.
[Table II] The 'T arget' column header contains an obvious spacing typo and should be 'Target'; additionally, reporting only averages over 1000 examples without error bars or percentile ranges makes it difficult to assess variability.
[Algorithm 1] The convergence criterion is described only as 'e.g., if the representation distance is below threshold'; for reproducibility, specify the exact threshold and the maximum number of iterations used in the experiments.
[Section III-A] The paper refers to 'our earlier work [26]' when introducing the embedding alignment approach, but the relation between the PRM algorithm and reference [26] is not explained, and the novelty claim in the abstract is not positioned relative to that prior work.

Circularity Check

0 steps flagged · score 1.0 of 10

No load-bearing circularity; the matching result is an optimization output, while the semantic conclusion rests on an empirical imperceptibility proxy rather than a circular derivation.

full rationale

The paper's claim is an empirical demonstration, not a first-principles derivation. PRM (Section III) defines the loss in Eq. (1) as the squared distance between the embedding of the perturbed image and the target embedding, and Algorithm 1 explicitly minimizes that loss. The high cosine similarities reported in Table III are therefore execution checks of the optimizer, not independent predictions; an attack method reporting its own success condition is not circular, because the substantive content lies in achieving that match under the epsilon constraint and in the measured PSNR/SSIM fidelity. Neither the epsilon bound nor the PSNR/SSIM values are encoded in the loss, so the 'small changes' observation is not equivalent to the objective by construction. The interpretive step from high PSNR/SSIM to 'semantics ... not altered to any medical experts' (Section III-B) is an unsupported empirical assumption, but an unjustified proxy is a validity concern rather than a circular reduction. The only self-citation entering the method, [26], is used as provenance ('As outlined in our earlier work [26]'), and the algorithm is fully re-specified in the paper, so the citation is not load-bearing. No equation or theorem in the manuscript defines the central conclusion in terms of its own inputs.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

No invented entities. The main free parameters are the perturbation budget, learning rate, and convergence threshold; the imperceptibility proxy is a domain assumption, and the embedding-to-classifier link is also assumed.

free parameters (3)

pixel perturbation bound epsilon = 0.02 / 0.10
The claim of imperceptibility and the definition of small changes depend on this hand-chosen budget; results are shown for two values but no justification is given for their clinical relevance.
learning rate = 0.9 (MIL-VT), 0.09 (MedViT)
Hyperparameter that affects convergence; not fitted to the outcome, but the qualitative result may depend on achieving convergence.
convergence threshold = not specified
The convergence criterion is described as a threshold on embedding distance, but the numerical value is not reported, making the optimization endpoint underspecified.

assumptions (3)

domain assumption PSNR above 40 dB and SSIM above 0.97 imply perturbations are imperceptible to human experts
Used in Section IV-C to argue changes are visually negligible; no human study is performed, and PSNR/SSIM are not reliable proxies for clinical perceptibility.
domain assumption The ViT class-token embedding (or MIL aggregated embedding) is the representation that determines the model's classification decision
The attack targets these embeddings, and the accuracy drop is interpreted as a consequence of representation manipulation; if the classifier uses a different or combined feature, the interpretation is weakened.
domain assumption The selected test pairs, one from each of two different ground-truth classes, are representative of the model's behavior on the full test distribution
The paper randomly selects a subset but does not report its size or composition, so the reported accuracy drops may not generalize to the full test set.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Are Vision Transformer Representations Semantically Meaningful? A Case Study in Medical Imaging." pith.science (2026). https://pith.science/paper/WN65RPO2

@misc{pith2026250701788,
  author       = {Pith},
  title        = {Pith review of: Are Vision Transformer Representations Semantically Meaningful? A Case Study in Medical Imaging},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WN65RPO2}},
  note         = {Machine review of arXiv:2507.01788}
}

read the original abstract

Vision transformers (ViTs) have rapidly gained prominence in medical imaging tasks such as disease classification, segmentation, and detection due to their superior accuracy compared to conventional deep learning models. However, due to their size and complex interactions via the self-attention mechanism, they are not well understood. In particular, it is unclear whether the representations produced by such models are semantically meaningful. In this paper, using a projected gradient-based algorithm, we show that their representations are not semantically meaningful and they are inherently vulnerable to small changes. Images with imperceptible differences can have very different representations; on the other hand, images that should belong to different semantic classes can have nearly identical representations. Such vulnerability can lead to unreliable classification results; for example, unnoticeable changes cause the classification accuracy to be reduced by over 60\%. %. To the best of our knowledge, this is the first work to systematically demonstrate this fundamental lack of semantic meaningfulness in ViT representations for medical image classification, revealing a critical challenge for their deployment in safety-critical systems.

Figures

Figures reproduced from arXiv: 2507.01788 by the authors.

**Figure 2.** Typical examples from the APTOS2019 obtained using the proposed framework. Three pairs of visually indistinguishable [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references · 29 canonical work pages

[1]

On the opportunities and risks of foundation models,

Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Others, and et al., “On the opportunities and risks of foundation models,” CoRR, 2022

work page 2022
[2]

Gpt-4 technical report,

OpenAI, “Gpt-4 technical report,” 2023

2023
[3]

ChatGPT goes to law school,

Jonathan H. Choi, Kristin E. Hickman, Amy Monahan, and Daniel B. Schwarcz, “ChatGPT goes to law school,” Journal of Legal Education (Forthcoming) , 01 2023

work page 2023
[4]

ProteinBERT: a universal deep- learning model of protein sequence and function,

Nadav Brandes, Dan Ofer, Yam Peleg, Nadav Rappoport, and Michal Linial, “ProteinBERT: a universal deep- learning model of protein sequence and function,” Bioin- formatics, vol. 38, no. 8, pp. 2102–2110, 02 2022

work page 2022
[5]

Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models,

Tiffany H. Kung, Morgan Cheatham, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepa ˜no, Maria Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo, and Victor Tseng, “Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models,” PLOS Digital Health, 2023

work page 2023
[6]

Attention is all you need,

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems, 2017

work page 2017
[7]

An image is worth 16x16 words: Transformers for image recognition at scale,

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in International Conference on Learning Representations, 2021

2021
[8]

Bert: Pre-training of deep bidirectional trans- formers for language understanding,

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova, “Bert: Pre-training of deep bidirectional trans- formers for language understanding,” 2019

work page 2019

Show all 31 references

[9]

Transformers in medical imaging: A survey,

Fahad Shamshad, Salman Khan, Syed Waqas Zamir, Muhammad Haris Khan, Munawar Hayat, Fahad Shahbaz Khan, and Huazhu Fu, “Transformers in medical imaging: A survey,” Medical Image Analysis , vol. 88, pp. 102802, 2023

2023
[10]

A comparative study between vision transformers and cnns in digital pathology,

Luca Deininger, Bernhard Stimpel, Anil Yuce, Samaneh Abbasi-Sureshjani, Simon Sch ¨onenberger, Paolo Ocampo, Konstanty Korski, and Fabien Gaire, “A comparative study between vision transformers and cnns in digital pathology,” 2022

2022
[11]

Mil-vt: Multiple instance learning enhanced vision trans- former for fundus image classification,

Shuang Yu, Kai Ma, Qi Bi, Cheng Bian, Munan Ning, Nanjun He, Yuexiang Li, Hanruo Liu, and Yefeng Zheng, “Mil-vt: Multiple instance learning enhanced vision trans- former for fundus image classification,” in Medical Image Computing and Computer Assisted Intervention – MICCAI 20...

2021
[12]

Med- vit: A robust vision transformer for generalized medical image classification,

Omid Nejati Manzari, Hamid Ahmadabadi, Hossein Kashi- ani, Shahriar B. Shokouhi, and Ahmad Ayatollahi, “Med- vit: A robust vision transformer for generalized medical image classification,” Computers in Biology and Medicine , vol. 157, pp. 106791, May 2023

2023
[13]

A recent survey of vision transformers for medical image segmentation,

Asifullah Khan, Zunaira Rauf, Abdul Rehman Khan, Saima Rathore, Saddam Hussain Khan, Najmus Saher Shah, Umair Farooq, Hifsa Asif, Aqsa Asif, Umme Zahoora, Rafi Ullah Khalil, Suleman Qamar, Umme Hani Asif, Faiza Babar Khan, Abdul Majid, and Jeonghwan Gwak, “A recent survey of v...

2023
[14]

Comparing cnns and vits for medical image classification leveraging transfer learning,

Giovanni Lonia, Davide Ciraolo, Maria Fazio, Massimo Villari, and Antonio Celesti, “Comparing cnns and vits for medical image classification leveraging transfer learning,” in 2024 IEEE Symposium on Computers and Communica- tions (ISCC), 2024, pp. 1–7

2024
[15]

Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs,

Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu, Jaspreet Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naveen Valluri, Cliff Wong, Andrea Tupini, Yu Wang, Matt Mazzola, Swadheen Shukla, Lars Liden, Jianfeng Gao, Angela Crabtree, Brian Piening, Carlo Bi- fulco, Matthew ...

2025
[16]

Pmc-clip: Contrastive language-image pre-training using biomedical documents,

Weixiong Lin, Ziheng Zhao, Xiaoman Zhang, Chaoyi Wu, Ya Zhang, Yanfeng Wang, and Weidi Xie, “Pmc-clip: Contrastive language-image pre-training using biomedical documents,” 2023

2023
[17]

Explaining and harnessing adversarial examples,

Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy, “Explaining and harnessing adversarial examples,” in In- ternational Conference on Learning Representations, 2015

2015
[18]

Intriguing properties of neural networks,

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus, “Intriguing properties of neural networks,” CoRR, 2014

2014
[19]

Towards deep learn- ing models resistant to adversarial attacks,

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu, “Towards deep learn- ing models resistant to adversarial attacks,” 2019

2019
[20]

Survey on adversarial attack and defense for medical image analysis: Methods and challenges,

Junhao Dong, Junxi Chen, Xiaohua Xie, Jianhuang Lai, and Hao Chen, “Survey on adversarial attack and defense for medical image analysis: Methods and challenges,”ACM Computing Surveys, vol. 57, no. 3, pp. 1–38, Nov. 2024

2024
[21]

Generalizability vs. robustness: Ad- versarial examples for medical imaging,

Magdalini Paschali, Sailesh Conjeti, Fernando Navarro, and Nassir Navab, “Generalizability vs. robustness: Ad- versarial examples for medical imaging,” 2018

2018
[22]

Adversarial attacks on medical machine learning,

Samuel G. Finlayson, John D. Bowers, Joichi Ito, Jonathan L. Zittrain, Andrew L. Beam, and Isaac S. Ko- hane, “Adversarial attacks on medical machine learning,” Science, vol. 363, no. 6433, pp. 1287–1289, 2019

2019
[23]

Understanding adversarial attacks on deep learning based medical image analysis systems,

Xingjun Ma, Yuhao Niu, Lin Gu, Yisen Wang, Yitian Zhao, James Bailey, and Feng Lu, “Understanding adversarial attacks on deep learning based medical image analysis systems,” Pattern Recognition, vol. 110, pp. 107332, 2021

2021
[24]

Adversarial attacks and adversarial robustness in computational pathology,

Narmin Ghaffari Laleh, Daniel Truhn, Gregory Patrick Veldhuizen, Tianyu Han, Marko van Treeck, Roman D. Buelow, Rupert Langer, Bastian Dislich, Peter Boor, V olk- mar Schulz, and Jakob Nikolas Kather, “Adversarial attacks and adversarial robustness in computational pathology,”...

2022
[25]

Un- derstanding robustness of transformers for image classifica- tion,

Srinadh Bhojanapalli, Ayan Chakrabarti, Daniel Glasner, Daliang Li, Thomas Unterthiner, and Andreas Veit, “Un- derstanding robustness of transformers for image classifica- tion,” in IEEE/CVF International Conference on Computer Vision (ICCV), 2021

2021
[26]

Intriguing equivalence structures of the embedding space of vision transformers,

Shaeke Salman, Md Montasir Bin Shams, and Xiuwen Liu, “Intriguing equivalence structures of the embedding space of vision transformers,” 2024

2024
[27]

Medmnist v2 - a large-scale lightweight benchmark for 2d and 3d biomedical image classification,

Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni, “Medmnist v2 - a large-scale lightweight benchmark for 2d and 3d biomedical image classification,” Scientific Data , vol. 10, no. 1, Jan. 2023

2023
[28]

Image quality metrics: Psnr vs. ssim,

Alain Hor ´e and Djemel Ziou, “Image quality metrics: Psnr vs. ssim,” in 2010 20th International Conference on Pattern Recognition, 2010, pp. 2366–2369

2010
[29]

Feature forwarding for efficient single image dehazing,

Peter Morales, Tzofi Klinghoffer, and Seung Jae Lee, “Feature forwarding for efficient single image dehazing,” 2019

2019
[30]

Understanding zero-shot adversarial robustness for large-scale models,

Chengzhi Mao, Scott Geng, Junfeng Yang, Xin Wang, and Carl V ondrick, “Understanding zero-shot adversarial robustness for large-scale models,” 2023

2023
[31]

Malicious path manipu- lations via exploitation of representation vulnerabilities of vision-language navigation systems,

Chashi Mahiul Islam, Shaeke Salman, Montasir Shams, Xiuwen Liu, and Piyush Kumar, “Malicious path manipu- lations via exploitation of representation vulnerabilities of vision-language navigation systems,” 2024

2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

On the opportunities and risks of foundation models,

Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Others, and et al., “On the opportunities and risks of foundation models,” CoRR, 2022

work page 2022

[2] [2]

Gpt-4 technical report,

OpenAI, “Gpt-4 technical report,” 2023

2023

[3] [3]

ChatGPT goes to law school,

Jonathan H. Choi, Kristin E. Hickman, Amy Monahan, and Daniel B. Schwarcz, “ChatGPT goes to law school,” Journal of Legal Education (Forthcoming) , 01 2023

work page 2023

[4] [4]

ProteinBERT: a universal deep- learning model of protein sequence and function,

Nadav Brandes, Dan Ofer, Yam Peleg, Nadav Rappoport, and Michal Linial, “ProteinBERT: a universal deep- learning model of protein sequence and function,” Bioin- formatics, vol. 38, no. 8, pp. 2102–2110, 02 2022

work page 2022

[5] [5]

Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models,

Tiffany H. Kung, Morgan Cheatham, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepa ˜no, Maria Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo, and Victor Tseng, “Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models,” PLOS Digital Health, 2023

work page 2023

[6] [6]

Attention is all you need,

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems, 2017

work page 2017

[7] [7]

An image is worth 16x16 words: Transformers for image recognition at scale,

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in International Conference on Learning Representations, 2021

2021

[8] [8]

Bert: Pre-training of deep bidirectional trans- formers for language understanding,

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova, “Bert: Pre-training of deep bidirectional trans- formers for language understanding,” 2019

work page 2019