ZeroGrad : Mitigating and Explaining Catastrophic Overfitting in FGSM Adversarial Training
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:DR7YNDBFrecord.jsonopen to challenge →
read the original abstract
Making deep neural networks robust to small adversarial noises has recently been sought in many applications. Adversarial training through iterative projected gradient descent (PGD) has been established as one of the mainstream ideas to achieve this goal. However, PGD is computationally demanding and often prohibitive in case of large datasets and models. For this reason, single-step PGD, also known as FGSM, has recently gained interest in the field. Unfortunately, FGSM-training leads to a phenomenon called ``catastrophic overfitting," which is a sudden drop in the adversarial accuracy under the PGD attack. In this paper, we support the idea that small input gradients play a key role in this phenomenon, and hence propose to zero the input gradient elements that are small for crafting FGSM attacks. Our proposed idea, while being simple and efficient, achieves competitive adversarial accuracy on various datasets.
This paper has not been read by Pith yet.
Forward citations
Cited by 4 Pith papers
-
Mitigating Error Amplification in Fast Adversarial Training
DDG dynamically adjusts perturbation magnitude and supervision strength in fast adversarial training according to sample confidence at the ground-truth class, mitigating catastrophic overfitting and the robustness-acc...
-
SORA: Free Second-Order Attacks in Fast Adversarial Training
SORA is an adaptive step-size adversarial training algorithm that formalizes epsilon overfitting, introduces the PertAlign metric to predict catastrophic overfitting, and dynamically adjusts perturbations to achieve s...
-
Unveiling the Backdoor Mechanism Hidden Behind Catastrophic Overfitting in Fast Adversarial Training
Catastrophic overfitting in fast adversarial training is reinterpreted as a weak-trigger variant of unlearnable tasks, allowing backdoor-inspired recalibration and outlier suppression to restore robustness.
-
Catastrophic Overfitting, Entropy Gap and Participation Ratio: A Noiseless $l^p$ Norm Solution for Fast Adversarial Training
An adaptive l^p norm control in FGSM adversarial training, guided by participation ratio and entropy of gradients, mitigates catastrophic overfitting without noise or regularization.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.