pith. sign in

arxiv: 1907.04774 · v1 · pith:Z2B7OPQFnew · submitted 2019-07-10 · 💻 cs.CV · cs.LG· eess.IV

Metamorphic Detection of Adversarial Examples in Deep Learning Models With Affine Transformations

Rohan Reddy Mekala , Gudjon Einar Magnusson , Adam Porter , Mikael Lindvall , Madeline Diep This is my paper

Pith reviewed 2026-05-24 23:43 UTC · model grok-4.3

classification 💻 cs.CV cs.LGeess.IV
keywords adversarial detectionmetamorphic testingaffine transformationsdeep neural networksimage classificationcomputer visionadversarial examples
0
0 comments X

The pith

Distance ratio preserving affine transformations detect adversarial examples by comparing deep learning model outputs on original and transformed images.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents a method to detect adversarial attacks on deep learning image classifiers using principles from metamorphic testing. It applies affine transformations that preserve distance ratios to an input image and checks if the model's classification remains the same. If the behavior is inconsistent, the image is likely adversarial. This is important because adversarial perturbations are tiny and can cause serious errors in applications like autonomous driving. The approach claims high accuracy without needing to know the specific attack used.

Core claim

By applying metamorphic relations based on distance ratio preserving affine image transformations which compare the behavior of the original and transformed image, the proposed approach can determine whether or not the input image is adversarial with a high degree of accuracy.

What carries the argument

Metamorphic relations using distance ratio preserving affine image transformations to compare model predictions on original versus transformed inputs.

If this is right

  • The method can identify adversarial manipulations that are imperceptible to humans.
  • It provides a way to guard against attacks in safety-critical industries such as self-driving cars and face recognition.
  • Detection works by checking consistency of model behavior under the transformations.
  • The approach is automatic and does not require knowledge of the attack details.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This detection strategy might extend to other data types like audio or text if similar transformations can be defined.
  • Combining it with existing defenses could improve overall robustness against adversarial attacks.
  • The accuracy might vary depending on the specific deep learning model architecture used.

Load-bearing premise

Distance ratio preserving affine transformations produce consistent model behavior on non-adversarial images but inconsistent behavior on adversarial images.

What would settle it

Finding a set of clean images where the model changes its prediction after applying the affine transformations, or adversarial images where the prediction stays the same.

Figures

Figures reproduced from arXiv: 1907.04774 by Adam Porter, Gudjon Einar Magnusson, Madeline Diep, Mikael Lindvall, Rohan Reddy Mekala.

Figure 9
Figure 9. Figure 9: We observe similar behavior to that using rotational transformation here [PITH_FULL_IMAGE:figures/full_fig_p006_9.png] view at source ↗
read the original abstract

Adversarial attacks are small, carefully crafted perturbations, imperceptible to the naked eye; that when added to an image cause deep learning models to misclassify the image with potentially detrimental outcomes. With the rise of artificial intelligence models in consumer safety and security intensive industries such as self-driving cars, camera surveillance and face recognition, there is a growing need for guarding against adversarial attacks. In this paper, we present an approach that uses metamorphic testing principles to automatically detect such adversarial attacks. The approach can detect image manipulations that are so small, that they are impossible to detect by a human through visual inspection. By applying metamorphic relations based on distance ratio preserving affine image transformations which compare the behavior of the original and transformed image; we show that our proposed approach can determine whether or not the input image is adversarial with a high degree of accuracy.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes a metamorphic testing approach to detect adversarial examples in deep learning image classifiers. It defines metamorphic relations based on distance-ratio-preserving affine transformations (e.g., rotations, scalings) and claims that comparing model behavior on an original image versus its transformed version allows reliable identification of adversarial inputs, achieving high accuracy.

Significance. If empirically validated with supporting experiments, the method could provide a useful, attack-agnostic detection technique for adversarial examples that does not require model retraining or knowledge of the perturbation, which would be relevant for safety-critical CV applications.

major comments (2)
  1. [Abstract] Abstract: the assertion that the approach determines whether an input is adversarial 'with a high degree of accuracy' is unsupported, as the manuscript supplies no experimental results, datasets, baselines, error bars, or implementation details.
  2. [Abstract] Abstract (final sentence): the core assumption that distance-ratio-preserving affine transformations produce consistent predictions on clean images but inconsistent predictions on adversarial images lacks any theoretical argument, invariance proof, or preliminary evidence; standard CNNs are known to change outputs under modest affine transforms unless explicitly trained for invariance.
minor comments (2)
  1. [Abstract] Abstract: punctuation and sentence structure issues, e.g., 'imperceptible to the naked eye; that when added' disrupts readability and should be rephrased.
  2. [Abstract] Abstract: the industry list ('self-driving cars, camera surveillance and face recognition') lacks an Oxford comma and parallel construction.

Simulated Author's Rebuttal

2 responses · 0 unresolved

Thank you for the constructive feedback on our manuscript. We appreciate the referee's identification of key issues in the abstract and will address them directly in our point-by-point response.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the assertion that the approach determines whether an input is adversarial 'with a high degree of accuracy' is unsupported, as the manuscript supplies no experimental results, datasets, baselines, error bars, or implementation details.

    Authors: We agree that the current manuscript does not include any experimental results, datasets, or implementation details to support the claim of determining adversarial inputs 'with a high degree of accuracy.' The abstract will be revised to remove this unsupported assertion. The revised version will either qualify the claim or defer it until supporting experiments can be added. revision: yes

  2. Referee: [Abstract] Abstract (final sentence): the core assumption that distance-ratio-preserving affine transformations produce consistent predictions on clean images but inconsistent predictions on adversarial images lacks any theoretical argument, invariance proof, or preliminary evidence; standard CNNs are known to change outputs under modest affine transforms unless explicitly trained for invariance.

    Authors: The referee correctly identifies that the manuscript provides no theoretical argument, invariance proof, or evidence for the core assumption. While the approach is motivated by the geometric properties of distance-ratio-preserving transformations, we acknowledge that standard CNNs are not inherently invariant to affine transforms. In the revision, we will expand the abstract and manuscript to include a more detailed rationale for the metamorphic relations, explicitly discuss the limitations regarding CNN invariance, and note this as requiring further investigation or preliminary validation. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical metamorphic testing method with no derivations or self-referential predictions

full rationale

The paper describes an application of metamorphic testing using distance-ratio-preserving affine transformations to detect adversarial examples by comparing model behavior on original and transformed images. No equations, derivations, fitted parameters presented as predictions, or load-bearing self-citations appear in the abstract or described approach. The central claim rests on an empirical assumption about model consistency under transforms, which is evaluated through experiments rather than reduced to a definition or prior self-citation by construction. This is a standard non-circular empirical method paper.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Insufficient information in the abstract to enumerate specific free parameters or invented entities; the approach rests on one domain assumption about metamorphic relations.

axioms (1)
  • domain assumption Distance ratio preserving affine transformations produce consistent classification behavior for non-adversarial images but inconsistent behavior for adversarial images.
    This premise is invoked in the final sentence of the abstract as the basis for detection.

pith-pipeline@v0.9.0 · 5687 in / 1220 out tokens · 21424 ms · 2026-05-24T23:43:55.596417+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

20 extracted references · 20 canonical work pages · 5 internal anchors

  1. [1]

    Imagenet classification with deep convolutional neural networks,

    A. Krizhevsky, I. Sutskever, and G. E. Hinton ,“Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 2012, pp. 1097– 1105

    work page 2012

  2. [2]

    Very Deep Convolutional Networks for Large-Scale Image Recognition

    K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in arXiv:1409.1556, 2014

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  3. [3]

    Intriguing properties of neural networks

    C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” in arXiv:1312.6199, 2014

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  4. [4]

    Practical Black -Box Attacks against Machine Learning

    N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, and A. Swami. “Practical Black -Box Attacks against Machine Learning”, ACM Asia Conference on Computer and Communications Security (ASIACCS), April 2017

    work page 2017

  5. [6]

    YOLOv3: An Incremental Improvement

    J. Redmon and A . Farhadi. “YOLOv3: An Incremental Improvement.” CoRRabs/1804.02767, 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  6. [7]

    Robust Physical -World Attacks on Deep Learning Visual Classification,

    K. Eykholt, I. Evtimov, E. Fernandes, B. Li, A. Rahmati, C. Xiao, A. Prakash, T. Kohno, and D. Song, “Robust Physical -World Attacks on Deep Learning Visual Classification,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 1625-1634

    work page 2018

  7. [8]

    Black-box Adversarial Attacks with Limited Queries and Information

    A. Ilyas, L. Engstrom, A. Athalye, and J. Lin, “Black-box Adversarial Attacks with Limited Queries and Information ”, in ICML, 2018

    work page 2018

  8. [9]

    IEEE Std 1012-2016 - IEEE Standard for System, Software, and Hardware Verification and Validation

    IEEE. “IEEE Std 1012-2016 - IEEE Standard for System, Software, and Hardware Verification and Validation ”, https://standards.ieee.org/findstds/standard/ 1012 -2016.html. [Online; accessed 7-Jan-2018]

    work page 2016

  9. [10]

    ImageNet: a large -scale hierarchical image database

    J. Deng, W. Dong, R. Socher, L-J. Li, K. Li, and L. Fei-Fei, “ImageNet: a large -scale hierarchical image database ”, in CVPR, 2009

    work page 2009

  10. [11]

    A Survey on Metamorphic Testing

    S. Segura, G. Fraser, A. Sánchez, and A. Ruiz-Cortés, “A Survey on Metamorphic Testing ”, in IEEE Transactions on Software Engineering. Vol 42(9), 2016, pp. 805-824

    work page 2016

  11. [12]

    Deep Residual Learning for Image Recognition

    K. He, X, Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition”, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770-778

    work page 2016

  12. [13]

    Identifying implementation bugs in machine learning based image classifiers using metamorphic testing

    A. Dwarakanath, M. Ahuja, S. Sikand, R. M. Rao, R. P. Bose, N. Dubash, and S. Podder, “Identifying implementation bugs in machine learning based image classifiers using metamorphic testing”, in Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis, 2018, pp. 118-128

    work page 2018

  13. [14]

    Detecting Adversarial Examples Through Image Transformation

    S. Tian, G. Yang, a nd Y. Cai, “Detecting Adversarial Examples Through Image Transformation”, in AAAI, 2018

    work page 2018

  14. [15]

    Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods

    N. Carlini and D. Wagner, “Adversarial examples are not easily detected: Bypassing ten detection methods ”, in arXiv preprint arXiv:1705.07263, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  15. [16]

    Explaining and Harnessing Adversarial Examples

    I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and Harnessing Adversarial Examples”, inCoRR, abs/1412.6572, 2014

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  16. [17]

    A S ystematic Way of Affine Transformation Using Image Registration

    J. Singla and G. Raman, “A S ystematic Way of Affine Transformation Using Image Registration”, 2012

    work page 2012

  17. [18]

    Krizhevsky, V

    A. Krizhevsky, V. Nair, and G . Hinton, ‘CIFAR -10 and CIFAR - 100’. https://www.cs.toronto.edu/~kriz/cifar.html

    work page

  18. [19]

    MNIST handwritten digit database

    Y. LeCun and C. Cortes, (2010), “MNIST handwritten digit database”, 2010. http://yann.lecun.com/exdb/mnist/

    work page 2010

  19. [20]

    DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks

    S-M. Moosavi-Dezfooli, A. Fawzi and P. Frossard, “DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks ”, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 2574-2582

    work page 2016

  20. [21]

    Metamorphic testing: a new approach for generating next test cases

    T. Y. Chen, S. C. Cheung, and S. M. Yiu. “Metamorphic testing: a new approach for generating next test cases ”, Technical Report HKUST-CS98-01, Department of Computer Science, Hong Kong University of Science and Technology, Hong Kong, 1998

    work page 1998