Pith. sign in

REVIEW 3 major objections 6 minor 15 references

Are classical deep neural networks weakly adversarially robust?

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Classical deep neural networks carry an inherent adversarial robustness, and a layer-wise feature-path method reaches 44-46% adversarial accuracy on CIFAR-10 without any adversarial training.

desk verdict Headline adversarial accuracies are post-selected on the test set, so the 'inherent robustness' claim is unsupported; the recognition extension is interesting but needs proper validation. read the letter →

arxiv 2506.02016 v1 pith:CMVMXG5G submitted 2025-05-28 cs.CV cs.LG

Nuolin Sun , Linyuan Wang , Dongyang Li , Bin Yan , Lei Li This is my paper
classification cs.CVcs.LG
keywords adversarialrobustnesslayer-wisefeaturepathsexampledetectionrecognitionProgressiveFeedforwardCollapseneuralCIFAR-10AutoPGD
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Adversarial robustness in deep networks is normally bought with adversarial training: generate attacks, retrain, and pay the compute cost. This paper claims that ordinary, unmodified networks already contain enough robustness to spot and recognize attacks, if classification uses the model's own layer-by-layer features instead of its final readout. It defines a layer-wise feature path for every image and compares that path against stored per-class average feature paths using cosine similarity. On CIFAR-10 under AutoPGD attacks, this method reaches 44.17% (ResNet-20) and 46.1% (ResNet-18) adversarial accuracy while keeping clean accuracy above 80%, trading a few points of adversarial accuracy against adversarial training (50.74-52.94%) for zero retraining cost. The authors take this as evidence that the widely accepted view of weak adversarial robustness in classical DNNs is wrong.

What carries the argument

The central object is the layer-wise feature path: for an example, the sequence of feature vectors it produces at the output of each residual block; for a class, the sequence of L2-normalized per-layer average features of its training samples (the class centroid path). The argument is carried by cosine similarity between an example's path and each class centroid path, aggregated across layers as a weighted sum. Detection thresholds the maximum aggregated similarity with Otsu's variance-maximizing rule, discarding the final layers where PFC occurs; recognition instead classifies by equal-weight voting over the most robust intermediate layers (layers 2, 3, and 4 in the experiments), exploiting the observation that adversarial features move toward the wrong class only in the last layers. Everything rests on the geometric premise that clean paths hug their true class path while adversarial paths are globally decorrelated.

What would settle it

Generate adversarial examples with an adaptive attack that optimizes against the feature-path cosine similarity (for example, minimizing the maximum similarity to the true class centroid path) rather than the cross-entropy loss, then rerun detection and recognition with the same thresholds and layer choices on CIFAR-10; if adversarial accuracy collapses toward the standard-training level of about 5%, the claimed inherent robustness is an artifact of the attack family tested. A cheaper check is to apply the exact threshold 0.6855 and the layers 2+3+4 voting rule to CIFAR-100 or a Transformer-based network, where the PFC separation premise is untested.

Watch

Extended reading notes

Core claim

The paper's central claim, in its own words, is that classical deep neural networks possess 'inherent adversarial robustness' that adversarial training is not required to unlock: it only takes using the right readout of the network's internal features. The evidence is the layer-wise feature-path experiment. For each test image the method collects the feature vectors at each residual-block output and measures their cosine similarity to per-class centroid paths built from training data. Clean images stay highly correlated with their true class path, especially in the final layers where Progressive Feedforward Collapse concentrates features around class centroids, while adversarial images show lowered correlation with every class path. Using the maximum similarity over classes with an Otsu-selected threshold, the detector flags 80.28% (ResNet-20) and 82.8% (ResNet-18) of adversarial examples; for recognition, the method drops the final few layers and classifies by equal-weight voting over the most robust intermediate layers, achieving 44.17% and 46.1% adversarial accuracy against the roughly 4.9% of standard training. The conclusion the authors draw is that the weak robustness attributed to DNNs is a failure of the classifier head, not of the features.

Load-bearing premise

The method stands or falls on the unproved geometric premise, stated in Section 2.1 and only illustrated in Fig. 1, that clean examples stay highly correlated with their true class feature path, especially in the final few layers with PFC, while adversarial examples show low correlation with any class path; if that separation fails for other attacks, depths, or datasets, both the Otsu threshold and the intermediate-layer voting rule lose their justification.

Editorial extensions

If this is right

  • Adversarial accuracy rises from about 4.9% (standard training) to 44.17% and 46.1% on CIFAR-10 with no retraining and no attack-specific knowledge, so standard-trained networks need not have near-zero adversarial accuracy.
  • The trade-off against adversarial training is favorable on clean accuracy: 82.77% and 80.01% clean versus 77.64% and 78.48%, with adversarial accuracy within roughly 5 to 9 points, at a fraction of the training cost.
  • The same optimal detection threshold, 0.6855, separates clean from adversarial examples on both ResNet-20 and ResNet-18, which the authors read as evidence that small architectural differences do not change the similarity statistics.
  • Because detection flags 80.28% and 82.8% of adversarial examples first, the reported recognition numbers apply to the examples the detector lets through, coupling the two tasks rather than treating them independently.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The claim of 'inherent' robustness is stronger than what the experiments directly show: they cover one dataset, one attack family (AutoPGD at 8/255, 20 iterations), and layer choices selected on the test distribution, so a natural extension is to test whether intermediate-layer voting survives adaptive attacks that optimize against the feature-path statistic itself.
  • If the geometric separation holds generally, tools that interpret features as trajectories through layers rather than as a final logit could gain robustness-minded uses, which the authors do not discuss.
  • The same machinery could serve as an inexpensive robustness diagnostic: if an off-the-shelf model's intermediate layers lose class correlation under attack, that predicts where adversarial training is needed.
  • The authors attribute the effect to the network rather than to the detection-and-voting procedure, but the procedure itself is doing the robustness work; a fair reading is that the method transfers robustness from the feature geometry to the output.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a post-hoc, feature-path-based mechanism for detecting adversarial examples and for classifying both clean and adversarial images. For each layer, class centroids are computed from training features (Eq. 1); a test example is compared with all centroids via cosine similarity (Eq. 2), aggregated over layers (Eq. 3), and the maximum over classes is used as a detection statistic (Eq. 5) with a threshold set by Otsu's method (Section 2.2). For recognition, the method selects the top-L' intermediate layers with the highest per-layer adversarial accuracy on the test set and uses equal-weight voting among them (Eqs. 6-7, Section 2.3). On CIFAR-10 with ResNet-20 and ResNet-18, the paper reports 82.77%/44.17% and 80.01%/46.1% clean/adversarial accuracy under AutoPGD, and claims this exposes 'inherent adversarial robustness' in standard DNNs.

Significance. If the claims were established, the result would be significant: it would offer a training-free, inference-time mechanism for recovering substantial accuracy under attack, and it would complicate the prevailing view that standard DNNs are uniformly fragile. The paper deserves credit for testing the idea on two architectures and for using AutoPGD, a relatively strong white-box attack, rather than a weak iterative attack. However, the significance of the claim is currently undermined by the evaluation protocol: the headline numbers are obtained after fitting both the detection threshold and the voting layers to the very test examples that are later scored, and no attack that adapts to the proposed defense is considered. With a proper holdout evaluation and adaptive attacks, the work could make a meaningful contribution.

major comments (3)
  1. [Section 2.2 and Table 3] The detection threshold tau is selected by Otsu's method on the distributions of Smax computed from the same 10,000 clean and 10,000 adversarial test examples that are subsequently used to report detection and recognition rates. In Section 2.3, the voting layers L' are likewise chosen by 'statistically evaluating the adversarial classification accuracy across all layers' on those same examples, and Table 3 reports the best observed configuration (layers 2+3+4). The 44.17% and 46.1% figures are therefore post-selected rather than predicted. A validation split, nested cross-validation, or at least a clear separation between model selection and final evaluation is needed before the accuracy numbers can support the paper's central claim.
  2. [Section 3.1] The evaluation considers only a single non-adaptive AutoPGD attack with epsilon=8/255 and 20 iterations. Because the proposed method consists of a detection rule and a voting rule, an adversary who knows these rules can optimize the perturbation against them, for example by maximizing Smax or by targeting the selected middle layers. Without such an adaptive attacker, the reported robustness numbers do not establish that the method is robust against adversarial examples in general, and the claim of 'inherent adversarial robustness' is overstated.
  3. [Section 2.2 and Eqs. (3), (6)] The layer weights omega_l are not defined. For detection, the text says 'we remove the layers where PFC occurs' but does not specify how many layers are removed or how this removal is determined; for recognition, equal weights are assumed only after a selection step. If the choice of layers and weights is part of the method, it must be specified before evaluation, not tuned on the test set.
minor comments (6)
  1. [Eq. (3)] The symbol omega_l is used but never defined; the recognition section later says equal weights are used, but the detection procedure leaves the weight assignment ambiguous.
  2. [Eq. (2)] The feature vector is written as h^l_{c,test} in the numerator, which is inconsistent with the definition h^l_{test} used elsewhere in the same equation.
  3. [Figure 1] The caption is too terse; it should state what the curves represent, which classes and examples are shown, and how the adversarial examples were generated.
  4. [References] Reference [15] is Otsu's 1979 paper, but the year is listed as 2007; please correct the citation.
  5. [Abstract and Section 1] The phrase 'priori information' should be 'a priori information'.
  6. [Section 3.2.1] The claim that ResNet-20 and ResNet-18 have exactly the same optimal threshold tau=0.6855 is reported without error bars or repeated runs, so it is difficult to assess whether this coincidence is meaningful.

Circularity Check

2 steps flagged · score 6.0 of 10

Headline adversarial accuracies are fitted to the same test set: the Otsu threshold and top-L' layers are chosen on the exact clean/adversarial examples that are then scored.

  1. fitted input called prediction [Section 2.2 (Detection), Eq. (4)-(5); results in Section 3.1.1, Fig. 2]
    "For each test example, compute its maximum similarity to all classes and compare it with a preset threshold τ. If Smax < τ, classify x as adversarial; otherwise, classify it as clean. We adopt the core principle of Otsu’s thresholding method [15], determining τ by maximizing the variance between layer-wise feature paths of clean and adversarial examples."

    The detection decision rule is parameterized by τ, and τ is chosen by Otsu on the distribution of Smax over the same 10,000 clean and 10,000 AutoPGD CIFAR-10 test examples whose detection accuracy is then reported (80.28% for ResNet-20, 82.8% for ResNet-18). The separation statistic is therefore fitted to the very examples that are later counted, so the detection accuracy is the value used to fit the threshold on that data, not an out-of-sample prediction. It also controls which examples enter the recognition stage, coupling the headline recognition numbers to this fitted split.

  2. fitted input called prediction [Section 2.3 (Recognition), Eq. (6)-(7); selection results in Section 3.1.2 and Table 3]
    "To identify more robust intermediate layers, we statistically evaluate the adversarial classification accuracy across all layers and select the top-L′ layers with higher accuracy."

    The voting set L' is selected by evaluating adversarial classification accuracy on the same 10,000 AutoPGD adversarial test examples that are later scored. Section 3.1.2 then reports that a weighted combination of layers 2, 3, and 4 achieves the highest classification accuracy of 44.17% in Table 3, and Section 3.2.2 reports 46.1% for the same layers on ResNet-18. These headline numbers are maxima over layer subsets screened on the test set (Table 3 lists layer2+3, layer2+4, layer3+4, layer2+3+4), i.e., test-set hyperparameter selection, and are evaluated on the post-detection subset defined by the likewise fitted Otsu threshold. The adversarial accuracy is therefore partly forced by construction rather than an independent measure of inherent robustness.

full rationale

The feature-path construction itself is self-contained: Eq. (1) defines class centers from training features, Eq. (2) computes cosine similarities, and Eq. (3) aggregates them with weights; the PFC/NC motivation is cited from external work. However, the two controls that produce the headline numbers are fitted on the evaluation set. The threshold τ in Section 2.2 is chosen by Otsu on the same clean/adversarial test Smax histogram that is scored, and the voted layer subset in Section 2.3 is chosen by ranking adversarial accuracy on the same adversarial test examples scored in Tables 3-4. Thus the claims "over 80% detection" and "44.17%/46.1% adversarial accuracy" are post-selected values, not predictions about unseen attacks or examples. No self-citation chain or renaming is involved; the circularity is the standard fitted-input-called-prediction pattern. Under the rubric this is partial circularity (6), because the underlying similarity statistic has independent content and is compared against adversarial training, but the central empirical claims reduce in part to test-set selection.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method's class-center paths are computed from training data, but its operating point (threshold) and recognition layers are fitted to the test set; PFC and similarity geometry are assumed rather than derived.

free parameters (2)
  • detection threshold tau = 0.6855
    Selected with Otsu's method by maximizing the variance between cosine-similarity distributions of clean and adversarial test examples; used in Section 2.2 Eq. 5 to convert Smax into a detection decision.
  • recognition voting layers and L' = layers 2, 3, 4 (L'=3)
    Chosen in Section 2.3 and Table 3 as the top-L' intermediate layers by adversarial classification accuracy computed on the test examples; this post hoc selection inflates reported adversarial accuracy.
assumptions (4)
  • domain assumption PFC holds for trained ResNet-20, so final-layer features collapse around class centers
    Invoked in Section 2.1 to justify comparing test features to class centers; supported only by citation [11].
  • domain assumption Clean examples correlate with their correct class path, adversarial examples do not
    The core separation premise stated in Section 2.1; unproved and only visualized in Fig. 1.
  • domain assumption Cosine similarity between layer-wise feature paths is an adequate adversarial indicator
    Adopted in Section 2.1 with reference to [10]; no theoretical or empirical support beyond the fitted evaluation.
  • ad hoc to paper Otsu threshold and layer choices fitted to the test set generalize to unseen data
    The paper assumes the test-set-derived threshold and layer choices transfer, without a held-out split or repeated trials.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Are classical deep neural networks weakly adversarially robust?." pith.science (2026). https://pith.science/paper/CMVMXG5G

@misc{pith2026250602016,
  author       = {Pith},
  title        = {Pith review of: Are classical deep neural networks weakly adversarially robust?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CMVMXG5G}},
  note         = {Machine review of arXiv:2506.02016}
}
read the original abstract

Adversarial attacks have received increasing attention and it has been widely recognized that classical DNNs have weak adversarial robustness. The most commonly used adversarial defense method, adversarial training, improves the adversarial accuracy of DNNs by generating adversarial examples and retraining the model. However, adversarial training requires a significant computational overhead. In this paper, inspired by existing studies focusing on the clustering properties of DNN output features at each layer and the Progressive Feedforward Collapse phenomenon, we propose a method for adversarial example detection and image recognition that uses layer-wise features to construct feature paths and computes the correlation between the examples feature paths and the class-centered feature paths. Experimental results show that the recognition method achieves 82.77% clean accuracy and 44.17% adversarial accuracy on the ResNet-20 with PFC. Compared to the adversarial training method with 77.64% clean accuracy and 52.94% adversarial accuracy, our method exhibits a trade-off without relying on computationally expensive defense strategies. Furthermore, on the standard ResNet-18, our method maintains this advantage with respective metrics of 80.01% and 46.1%. This result reveals inherent adversarial robustness in DNNs, challenging the conventional understanding of the weak adversarial robustness in DNNs.

Figures

Figures reproduced from arXiv: 2506.02016 by the authors.

Figure 1
Figure 1. Layer-wise feature paths comparison between clean and adversarial examples based on [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Feature similarity distribution histogram of ResNet-20 [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Layer-wise feature classification accuracy of ResNet-20 on the initial examples [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Feature similarity distribution histogram of ResNet-18 [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Layer-wise feature classification accuracy of ResNet-18 on the initial examples [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

15 extracted references · 9 canonical work pages

  1. [1]

    Intriguing properties of neural networks.Computer Science, 2013

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Good- fellow, and Rob Fergus. Intriguing properties of neural networks.Computer Science, 2013

    work page 2013

  2. [2]

    Univer- sal adversarial perturbations

    Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Univer- sal adversarial perturbations. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1765–1773, 2017

    2017

  3. [3]

    Goodfellow, Jonathon Shlens, and Christian Szegedy

    Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adver- sarial examples.Computer Science, 2014

    work page 2014

  4. [4]

    Ensemble adversarial training: Attacks and defenses.arXiv preprint arXiv:1705.07204, 2017

    Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses.arXiv preprint arXiv:1705.07204, 2017

    arXiv 2017

  5. [5]

    Towards deep learning models resistant to adversarial attacks.arXiv preprint arXiv:1706.06083, 2017

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks.arXiv preprint arXiv:1706.06083, 2017

    arXiv 2017

  6. [6]

    Towards evaluating the robustness of neural networks

    Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pages 39–57. Ieee, 2017

    2017

  7. [7]

    Distillation as a defense to adversarial perturbations against deep neural networks

    Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a defense to adversarial perturbations against deep neural networks. In2016 IEEE symposium on security and privacy (SP), pages 582–597. IEEE, 2016

    work page 2016

  8. [8]

    Deepfool: a simple and accurate method to fool deep neural networks

    Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2574–2582, 2016

    2016

Show all 15 references
  1. [9]

    Adversarial examples are not bugs, they are features

    Andrew Ilyas, Logan Engstrom, Shibani Santurkar, Brandon Tran, Dimitris Tsipras, and Aleksander Madry. Adversarial examples are not bugs, they are features. InAdvances in Neural Information Processing Systems 32, V olume 1 of 20: 32nd Conference on Neural Information Processin...

    2019

  2. [10]

    Adversarial sample detection through neural network transport dynamics

    Skander Karkar, Patrick Gallinari, and Alain Rakotomamonjy. Adversarial sample detection through neural network transport dynamics. InJoint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 164–181. Springer, 2023

    2023

  3. [11]

    Progressive feedforward collapse of resnet training

    Sicong Wang, Kuo Gai, and Shihua Zhang. Progressive feedforward collapse of resnet training. arXiv preprint arXiv:2405.00985, 2024

    2024 arXiv

  4. [12]

    Prevalence of neural collapse during the terminal phase of deep learning training.Proceedings of the National Academy of Sciences, 2020

    Vardan Papyan, X Y Han, and David L Donoho. Prevalence of neural collapse during the terminal phase of deep learning training.Proceedings of the National Academy of Sciences, 2020

    2020

  5. [13]

    A law of data separation in deep learning.Proceedings of the National Academy of Sciences, 120(36):e2221704120, 2023

    Hangfeng He and Weijie J Su. A law of data separation in deep learning.Proceedings of the National Academy of Sciences, 120(36):e2221704120, 2023

    2023

  6. [14]

    Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks

    Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In37th International Conference on Machine Learning: ICML 2020, Online, 13-18 July 2020, Part 3 of 15, 2021

    2020

  7. [15]

    A threshold selection method from gray-level histograms.IEEE Transactions on Systems Man & Cybernetics, 9(1):62–66, 2007

    Nobuyuki Otsu. A threshold selection method from gray-level histograms.IEEE Transactions on Systems Man & Cybernetics, 9(1):62–66, 2007. 9

    2007

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.