REVIEW 16 cited by
Fast is better than free: Revisiting adversarial training
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Fast is better than free: Revisiting adversarial training
read the original abstract
Adversarial training, a method for learning robust deep networks, is typically assumed to be more expensive than traditional training due to the necessity of constructing adversarial examples via a first-order method like projected gradient decent (PGD). In this paper, we make the surprising discovery that it is possible to train empirically robust models using a much weaker and cheaper adversary, an approach that was previously believed to be ineffective, rendering the method no more costly than standard training in practice. Specifically, we show that adversarial training with the fast gradient sign method (FGSM), when combined with random initialization, is as effective as PGD-based training but has significantly lower cost. Furthermore we show that FGSM adversarial training can be further accelerated by using standard techniques for efficient training of deep networks, allowing us to learn a robust CIFAR10 classifier with 45% robust accuracy to PGD attacks with $\epsilon=8/255$ in 6 minutes, and a robust ImageNet classifier with 43% robust accuracy at $\epsilon=2/255$ in 12 hours, in comparison to past work based on "free" adversarial training which took 10 and 50 hours to reach the same respective thresholds. Finally, we identify a failure mode referred to as "catastrophic overfitting" which may have caused previous attempts to use FGSM adversarial training to fail. All code for reproducing the experiments in this paper as well as pretrained model weights are at https://github.com/locuslab/fast_adversarial.
Forward citations
Cited by 16 Pith papers
-
Uncovering and Understanding FPR Manipulation Attack in Industrial IoT Networks
FPR manipulation attack perturbs benign MQTT packets to flip labels to attacks in NIDS with 80-100% success, increasing SOC delays without gradient-based methods.
-
Learning Robustness at Test-Time from a Non-Robust Teacher
A test-time adaptation framework anchors adversarial training to a non-robust teacher's predictions, yielding more stable optimization and better robustness-accuracy trade-offs than standard self-consistency methods.
-
Adversarial Frontiers: Minimum-Norm Attack Ensembles for Robustness Evaluation
A greedy, query-budgeted attack-ensemble framework approximates the minimum-perturbation frontier and yields an ε-free defense ranking (DOI).
-
FastAT Benchmark: A Comprehensive Framework for Fair Evaluation of Fast Adversarial Training Methods
The FastAT Benchmark standardizes evaluation of over twenty fast adversarial training methods under unified conditions, showing that well-designed single-step approaches can match or exceed PGD-AT robustness at lower ...
-
Representation-Guided Parameter-Efficient LLM Unlearning
REGLU guides LoRA-based unlearning via representation subspaces and orthogonal regularization to outperform prior methods on forget-retain trade-off in LLM benchmarks.
-
GF-Score: Certified Class-Conditional Robustness Evaluation with Fairness Guarantees
GF-Score decomposes certified robustness into per-class profiles, adds fairness metrics like disparity index and Gini coefficient, and uses self-calibration on clean accuracy to avoid adversarial attacks.
-
Quantum Patches: Enhancing Robustness of Quantum Machine Learning Models
Random quantum circuits used as adversarial training data reduce successful attack rates on QML models for CIFAR-10 from 89.8% to 68.45% and for CINIC-10 from 94.23% to 78.68%.
-
Compression as an Adversarial Amplifier Through Decision Space Reduction
Compression acts as an adversarial amplifier by reducing the decision space of image classifiers, making attacks in compressed representations substantially more effective than pixel-space attacks under the same pertu...
-
SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks
SmoothLLM mitigates jailbreaking attacks on LLMs by randomly perturbing multiple copies of a prompt at the character level and aggregating the outputs to detect adversarial inputs.
-
Efficient LLM Adversarial Training via Low-Rank Defense and Circuit-Guided Surrogates
Combining suffix-window representation finetuning with an ActGrad-pruned surrogate cuts latent-adversarial-training FLOPs per step by 48.1% with only 0.0118% trainable parameters, while accepting higher attack success rates.
-
Foveation-Guided Dynamic Token Selection for Robust and Efficient Vision Transformers
FDT adds foveation and binary fixation modules to DeiT so multi-scale tokens are selected dynamically in one pass, improving ImageNet100 accuracy, MACs, and robustness without adversarial training.
-
A combination of noise and bilateral filters achieve supralinear and scalable adversarial robustness in CNNs
A preprocessor of Gaussian noise plus bilateral filtering yields supralinear adversarial robustness in CNNs and, when paired with adversarial training, ranks near the top of RobustBench while using far less compute, p...
-
Robust Bidirectional Associative Memory via Regularization Inspired by the Subspace Rotation Algorithm
B-SRA, a gradient-free orthogonal-rotation trainer, plus OWM/GPA regularizers, makes BAM markedly more robust to masking, noise, and FGSM/FFGSM/BIM/PGD attacks than Bidirectional Backpropagation.
-
Catastrophic Overfitting, Entropy Gap and Participation Ratio: A Noiseless $l^p$ Norm Solution for Fast Adversarial Training
An adaptive l^p norm control in FGSM adversarial training, guided by participation ratio and entropy of gradients, mitigates catastrophic overfitting without noise or regularization.
-
Robust Auto-associative Memory via Convolutional Restricted Hopfield Networks
CRHNs integrate convolutional extraction with subspace attractor retrieval trained via Subspace Rotation Algorithm and report order-of-magnitude lower reconstruction error than MHNs and PCNs on STL data under adversar...
-
Enhancing Adversarial Transferability through Block Stretch and Shrink
A block stretch-and-shrink input transformation improves black-box adversarial transferability in experiments on 1000 ImageNet images, but the submitted manuscript contains missing figures and an abstract describing a...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.