RECORD
Local Reweighting for Adversarial Training
Machine Learning · Submitted 30 Jun 2021
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Experiments show that LRAT works better than both IRAT (i.e., global reweighting) and the standard AT (i.e., no reweighting) when trained with an attack and tested on different attacks.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Instances-reweighted adversarial training (IRAT) can significantly boost the robustness of trained models, where data being less/more vulnerable to the given attack are assigned smaller/larger weights during training. However, when tested on attacks different from the given attack simulated in training, the robustness may drop significantly (e.g., even worse than no reweighting). In this paper, we study this problem and propose our solution--locally reweighted adversarial training (LRAT). The rationale behind IRAT is that we do not need to pay much attention to an instance that is already safe under the attack. We argue that the safeness should be attack-dependent, so that for the same instance, its weight can change given different attacks based on the same model. Thus, if the attack simulated in training is mis-specified, the weights of IRAT are misleading. To this end, LRAT pairs each instance with its adversarial variants and performs local reweighting inside each pair, while performing no global reweighting--the rationale is to fit the instance itself if it is immune to the attack, but not to skip the pair, in order to passively defend different attacks in future. Experiments show that LRAT works better than both IRAT (i.e., global reweighting) and the standard AT (i.e., no reweighting) when trained with an attack and tested on different attacks.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Instances-reweighted adversarial training (IRAT) can significantly boost the robustness of trained models, where data being less/more vulnerable to the given attack are assigned smaller/larger weights during training. | background 1.00 | 0.09 | 0.04 | asserted 0.92 |
| s1 | However, when tested on attacks different from the given attack simulated in training, the robustness may drop significantly (e.g., even worse than no reweighting). | gap 0.51 | 0.23 | 0.04 | hedged possibility 1.00 |
| s2 | In this paper, we study this problem and propose our solution--locally reweighted adversarial training (LRAT). | contribution 1.00 | 0.80 | 0.06 | asserted 0.84 |
| s3 | The rationale behind IRAT is that we do not need to pay much attention to an instance that is already safe under the attack. | background 0.96 | 0.09 | 0.03 | asserted 0.87 |
| s4 | We argue that the safeness should be attack-dependent, so that for the same instance, its weight can change given different attacks based on the same model. | background 0.27 | 0.25 | 0.06 | asserted 0.83 |
| s5 | Thus, if the attack simulated in training is mis-specified, the weights of IRAT are misleading. | background 0.48 | 0.15 | 0.04 | asserted 0.95 |
| s6 | To this end, LRAT pairs each instance with its adversarial variants and performs local reweighting inside each pair, while performing no global reweighting--the rationale is to fit the instance itself if it is immune to the attack, but not to skip the pair, in order to passively defend different attacks in future. | method 0.98 | 0.78 | 0.11 | asserted 0.98 |
| s7 | Experiments show that LRAT works better than both IRAT (i.e., global reweighting) and the standard AT (i.e., no reweighting) when trained with an attack and tested on different attacks. | result 1.00 | 0.96 | 0.86 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Instances-reweighted adversarial training (IRAT) can significantly boost the robustness of trained models, where data being less/more vulnerable to the given attack are assigned smaller/larger weights during training. However, when tested on attacks different from the given attack simulated in training, the robustness may drop significantly (e.g., even worse than no reweighting). In this paper, we study this problem and propose our solution--locally reweighted adversarial training (LRAT). The rationale behind IRAT is that we do not need to pay much attention to an instance that is already safe under the attack. We argue that the safeness should be attack-dependent, so that for the same instance, its weight can change given different attacks based on the same model. Thus, if the attack simulated in training is mis-specified, the weights of IRAT are misleading. To this end, LRAT pairs each instance with its adversarial variants and performs local reweighting inside each pair, while performing no global reweighting--the rationale is to fit the instance itself if it is immune to the attack, but not to skip the pair, in order to passively defend different attacks in future. Experiments show that LRAT works better than both IRAT (i.e., global reweighting) and the standard AT (i.e., no reweighting) when trained with an attack and tested on different attacks.
Human review
How the literature uses it
1 arXiv paper cite this in 1 sentence
Papers citing it
Cites it as background 1 paper
-
In recent years, there has been researched on sample reweighting [ref] in the adversarial robustness field.
Latent Feature Relation Consistency for Adversarial Robustness · Discussion and Future work · 1.00
What it cites · 2 sentences without a confident label
Cites as background 1 paper
-
By injecting adversarial examples into training data, adversarial training methods seek to train an adversarial-robust deep network whose predictions are locally invariant in a small neighborhood of its inputs [ref] .
Towards the Science of Security and Privacy in Machine Learning · Introduction · 0.86
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.