RECORD
Adversarial Robustness Overestimation and Instability in TRADES
Machine Learning · Submitted 10 Oct 2024
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim By examining metrics such as the First-Order Stationary Condition (FOSC), inner-maximization, and gradient information, we identify the underlying cause of this phenomenon as gradient masking and provide insights into it.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
This paper examines the phenomenon of probabilistic robustness overestimation in TRADES, a prominent adversarial training method. Our study reveals that TRADES sometimes yields disproportionately high PGD validation accuracy compared to the AutoAttack testing accuracy in the multiclass classification task. This discrepancy highlights a significant overestimation of robustness for these instances, potentially linked to gradient masking. We further analyze the parameters contributing to unstable models that lead to overestimation. Our findings indicate that smaller batch sizes, lower beta values (which control the weight of the robust loss term in TRADES), larger learning rates, and higher class complexity (e.g., CIFAR-100 versus CIFAR-10) are associated with an increased likelihood of robustness overestimation. By examining metrics such as the First-Order Stationary Condition (FOSC), inner-maximization, and gradient information, we identify the underlying cause of this phenomenon as gradient masking and provide insights into it. Furthermore, our experiments show that certain unstable training instances may return to a state without robust overestimation, inspiring our attempts at a solution. In addition to adjusting parameter settings to reduce instability or retraining when overestimation occurs, we recommend incorporating Gaussian noise in inputs when the FOSC score exceed the threshold. This method aims to mitigate robustness overestimation of TRADES and other similar methods at its source, ensuring more reliable representation of adversarial robustness during evaluation.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | This paper examines the phenomenon of probabilistic robustness overestimation in TRADES, a prominent adversarial training method. | contribution 0.58 | 0.40 | 0.04 | asserted 0.79 |
| s1 | Our study reveals that TRADES sometimes yields disproportionately high PGD validation accuracy compared to the AutoAttack testing accuracy in the multiclass classification task. | result 1.00 | 0.94 | 0.26 | asserted 0.68 |
| s2 | This discrepancy highlights a significant overestimation of robustness for these instances, potentially linked to gradient masking. | interpretation 0.97 | 0.73 | 0.26 | hedged possibility 0.68 |
| s3 | We further analyze the parameters contributing to unstable models that lead to overestimation. | contribution 0.91 | 0.87 | 0.04 | asserted 0.68 |
| s4 | Our findings indicate that smaller batch sizes, lower beta values (which control the weight of the robust loss term in TRADES), larger learning rates, and higher class complexity (e.g., CIFAR-100 versus CIFAR-10) are associated with an increased likelihood of robustness overestimation. | result 1.00 | 0.94 | 0.17 | asserted 0.55 |
| s5 | By examining metrics such as the First-Order Stationary Condition (FOSC), inner-maximization, and gradient information, we identify the underlying cause of this phenomenon as gradient masking and provide insights into it. | result 0.56 | 0.96 | 0.40 | asserted 1.00 |
| s6 | Furthermore, our experiments show that certain unstable training instances may return to a state without robust overestimation, inspiring our attempts at a solution. | result 0.97 | 0.94 | 0.06 | hedged possibility 0.71 |
| s7 | In addition to adjusting parameter settings to reduce instability or retraining when overestimation occurs, we recommend incorporating Gaussian noise in inputs when the FOSC score exceed the threshold. | method 0.62 | 0.67 | 0.16 | asserted 0.79 |
| s8 | This method aims to mitigate robustness overestimation of TRADES and other similar methods at its source, ensuring more reliable representation of adversarial robustness during evaluation. | contribution 0.66 | 0.42 | 0.37 | asserted 0.74 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
This paper examines the phenomenon of probabilistic robustness overestimation in TRADES, a prominent adversarial training method. Our study reveals that TRADES sometimes yields disproportionately high PGD validation accuracy compared to the AutoAttack testing accuracy in the multiclass classification task. This discrepancy highlights a significant overestimation of robustness for these instances, potentially linked to gradient masking. We further analyze the parameters contributing to unstable models that lead to overestimation. Our findings indicate that smaller batch sizes, lower beta values (which control the weight of the robust loss term in TRADES), larger learning rates, and higher class complexity (e.g., CIFAR-100 versus CIFAR-10) are associated with an increased likelihood of robustness overestimation. By examining metrics such as the First-Order Stationary Condition (FOSC), inner-maximization, and gradient information, we identify the underlying cause of this phenomenon as gradient masking and provide insights into it. Furthermore, our experiments show that certain unstable training instances may return to a state without robust overestimation, inspiring our attempts at a solution. In addition to adjusting parameter settings to reduce instability or retraining when overestimation occurs, we recommend incorporating Gaussian noise in inputs when the FOSC score exceed the threshold. This method aims to mitigate robustness overestimation of TRADES and other similar methods at its source, ensuring more reliable representation of adversarial robustness during evaluation.
Human review
How the literature uses it
1 arXiv paper cite this in 3 sentences
Papers citing it · 1 sentence without a confident label
Cites it as support 1 paper
-
Yet a recurring pattern undermines confidence in these numbers: defences are proposed with optimistic robustness claims that collapse under stronger attacks, richer threat models, or correct gradient accounting [ref] .
Auto-ART: Structured Literature Synthesis and Automated Adversarial Robustness Testing · Introduction · 0.99
What it cites · 7 sentences without a confident label
Cites as support 1 paper
-
The above hypothesis falls in line with the findings from the prior work by [ref] , which discusses the learnable patterns of simpler attacks like Fast Gradient Sign Method (FGSM) [ref] .
Explaining and Harnessing Adversarial Examples · TPGD Causes Over-fitting to Gradient-Based Attacks · 0.97
Cites as background 4 papers
-
Adversarial training [ref] , a method that involves training a model on adversarial examples, has been developed as a primary defense mechanism to enhance adversarial robustness.
Explaining and Harnessing Adversarial Examples · Introduction · 0.91
-
[ref] Adversarial training comes in different variants, including PGD-AT [ref] , ALP [ref] , and LSQ [ref] , MMA [ref] , among others.
Adversarial Logit Pairing · Related Work · 0.95
-
[ref] Adversarial training comes in different variants, including PGD-AT [ref] , ALP [ref] , and LSQ [ref] , MMA [ref] , among others.
Label Smoothing and Logit Squeezing: A Replacement for Adversarial Training? · Related Work · 0.95
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.