Pith. sign in

REVIEW 3 major objections 4 minor 77 references

Understanding Adversarial Training with Energy-based Models

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that catastrophic overfitting and robust overfitting in adversarial training can be diagnosed and mitigated through a single quantity, the change in classifier energy between a natural image and its adversarial…

desk verdict Energy-based view of overfitting in adversarial training has a useful new diagnostic and a plausible regularizer, but the multi-step RO results rest on an underspecified trigger that needs fixing before the claim is reproducible. read the letter →

arxiv 2505.22486 v1 pith:BFUOLKBI submitted 2025-05-28 cs.LG cs.CV

Mujtaba Hussain Mirza , Maria Rosaria Briglia , Filippo Bartolucci , Senad Beadini , Giuseppe Lisanti , Iacopo Masi This is my paper
classification cs.LGcs.CV
keywords adversarialtrainingenergy-basedmodelscatastrophicoverfittingrobustdeltaenergyregularizationsingle-stepexampleclassifiersasgenerative
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Adversarial training tends to fail in two ways: catastrophic overfitting, where single-step methods suddenly lose all robustness to multi-step attacks, and robust overfitting, where multi-step training keeps lowering training error while test robustness worsens. This paper argues that both failures show up in a scalar diagnostic derived from energy-based modeling: the change in marginal and joint energy between a natural sample and its adversarial counterpart, written as $\Delta E_{f_\theta}(x)$ and $\Delta E_{f_\theta}(x,y)$. In single-step training this delta energy spikes sharply when catastrophic overfitting begins, while in multi-step training with standard adversarial training it collapses when robust overfitting begins. The authors propose the $\Delta$ Energy Regularizer (DER), which punishes samples whose delta-energy vector is too large, and report that adding it prevents both failure modes on CIFAR-10, CIFAR-100, SVHN, and Tiny-ImageNet. If the claim holds, overfitting can be monitored by watching energy dynamics, and a simple smoothness penalty can replace elaborate regularization schemes.

What carries the argument

The load-bearing object is the delta-energy vector $(\Delta E_{f_\theta}(x), \Delta E_{f_\theta}(x,y))$, whose $\ell^2$ norm measures how far the adversarial perturbation bends the classifier's energy landscape around a sample. The energy definitions come from the standard energy-based reinterpretation of a softmax classifier, $E_{f_\theta}(x) = -\log \sum_k \exp(f_\theta(x)[k])$ and $E_{f_\theta}(x,y) = -f_\theta(x)[y]$, which makes the cross-entropy loss exactly $E_{f_\theta}(x,y) - E_{f_\theta}(x)$. The argument is carried by tracking this delta quantity epoch by epoch during adversarial training: a sharp increase marks catastrophic overfitting, a sharp decrease marks robust overfitting, and smoothing it with the regularizer $\max(\|(\Delta E_{f_\theta}(x), \Delta E_{f_\theta}(x,y))\|_2 - \gamma, 0)$ is what prevents both. The paper also uses this machinery to reinterpret TRADES as an energy alignment objective and to explain why its KL-based adversarial examples keep natural and adversarial energies close.

What would settle it

Track PGD-20 test accuracy and the mean delta-energy norm every epoch while training RS-FGSM with DER on CIFAR-10 at $\epsilon = 8/255$; if test robustness collapses to near zero while delta energy stays small, the claimed causal role fails. A second check is to apply DER with the same hyperparameters to a new dataset or larger perturbation radius and ask whether robust overfitting still appears while delta energy remains near zero.

Watch

Extended reading notes

Core claim

By treating a softmax classifier as an energy-based model, with marginal energy $E_{f_\theta}(x) = -\log \sum_k \exp(f_\theta(x)[k])$ and joint energy $E_{f_\theta}(x,y) = -f_\theta(x)[y]$, the paper defines the delta energy of a training sample as the directed shift from $x$ to its adversarial counterpart $x^\star$, namely $\Delta E_{f_\theta}(x) = E_{f_\theta}(x) - E_{f_\theta}(x^\star)$ and $\Delta E_{f_\theta}(x,y) = E_{f_\theta}(x,y) - E_{f_\theta}(x^\star,y)$. The central discovery is that the behavior of this delta energy diverges at the two overfitting phenomena: during catastrophic overfitting in RS-FGSM training it rises sharply because natural samples acquire much higher energy than their adversarial counterparts, while during robust overfitting in standard multi-step adversarial training it drops steeply. The paper further shows that methods that mitigate overfitting, including TRADES, keep delta energy near zero, and it rewrites the TRADES objective in energy terms to explain why. Building on that, the paper introduces DER, which adds a hinge penalty on the $\ell^2$ norm of the delta-energy vector to the classification loss, applies it to abnormal adversarial examples in single-step training and to all samples in multi-step training, and reports that it mitigates both catastrophic overfitting and robust overfitting across benchmarks.

Load-bearing premise

The regularizer assumes that the measured energy gap is not just a symptom of overfitting but a cause, so shrinking the gap with a penalty transfers to new datasets, attack budgets, and training schedules.

Editorial extensions

If this is right

  • A trainer can detect the onset of catastrophic overfitting by watching mean delta energy rise without running expensive PGD evaluation on test data every epoch.
  • Adding DER to RS-FGSM or N-FGSM keeps single-step training robust at perturbation radii where plain fast training collapses, including 16/255 and 32/255.
  • Applying DER only in the final phase of standard multi-step training is enough to reduce robust overfitting and, unlike reweighting baselines, does not sacrifice AutoAttack robustness.
  • The number of abnormal adversarial examples is not what triggers catastrophic overfitting; their energy and loss are, which redirects future fixes toward energy smoothness rather than counts of abnormal examples.
  • The energy-based rewriting of TRADES predicts that KL-based inner maximization inherently aligns natural and adversarial energies, which explains why TRADES is less prone to robust overfitting than standard adversarial training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the energy-gap signal is causal rather than merely correlated, then applying DER from the first epoch, without waiting for an estimated robust-overfitting onset, should give the same or better results; the paper mostly evaluates an onset-triggered schedule.
  • Delta energy could serve as a cheap, attack-free robustness monitor for other training schemes and architectures, but that requires checking whether its divergence threshold is stable across datasets and perturbation budgets.
  • The local class-wise PCA initialization used for generation suggests that any sampling-based generative procedure could use an energy-based stopping rule as a general diversity-preserving early exit.
  • The paper's evidence that down-weighting correctly classified samples hurts AutoAttack robustness implies that some reported gains from reweighting methods may be partly due to loss scaling or implicit weight decay rather than the reweighting itself.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies adversarial training (AT) through the lens of energy-based models. It defines the delta energy ΔE_fθ(x) = E_fθ(x) − E_fθ(x*) between a natural sample and its adversarial counterpart, and reports that this quantity diverges systematically when catastrophic overfitting (CO) or robust overfitting (RO) occurs: it increases sharply at CO and decreases sharply at RO. The authors propose the Delta Energy Regularizer (DER), which penalizes large L2 norms of the marginal-plus-joint delta-energy vector, applying it to abnormal adversarial examples in single-step AT and to all samples in multi-step AT, the latter starting 'preferably at the onset of RO.' They report that DER mitigates CO and RO across CIFAR-10, CIFAR-100, SVHN, and Tiny-ImageNet, and they also propose a local class-wise PCA initialization and energy-guided adaptive stopping to improve the diversity of images generated by robust classifiers. A secondary contribution is an interpretation of the TRADES objective as an energy-based model (Proposition 1).

Significance. If the central claims hold, the paper offers both a diagnostic signal for two well-known AT failure modes and a very simple regularizer that mitigates them, which would be practically useful. The paper's strengths are its breadth of experiments (four datasets, multiple attack budgets, PGD and AutoAttack evaluation, ablations) and a clean, machine-checkable derivation of the KL-energy identity in Proposition 1. The energy analysis of CO in single-step AT is well supported by the figures and by the comparison with AAER, and the single-step DER results are internally consistent. The main risk is in the multi-step RO claim, where the activation schedule of the regularizer appears to rely on information that is only available in hindsight; this must be resolved before the central claim can be accepted.

major comments (3)
  1. [Section VI-A, Eq. (7), Tables III-IV] The multi-step DER protocol is not reproducible as stated. The text says 'we initially train the model using SAT [3] and apply the regularizer during the final epochs, preferably starting at the onset of RO,' but no train-time criterion for detecting the onset of RO is given. The shaded/unshaded regions in Fig. 1 appear to delimit RO using the test error curve, which is not available during training. If the onset epoch is selected after inspecting the test PGD/AA curve, the DER-AT rows in Tables III and IV conflate the effect of the regularizer with an oracle early-stopping schedule. Please specify a practical protocol that uses only training data (e.g., a validation split, or a threshold on the training-set delta-energy statistic), or report results for a fixed schedule (e.g., regularizer active from epoch 60), and show that the reported gains persist without oracle information.
  2. [Section VI-A and Table Vb] The claim that DER is a 'minimal intervention' with 'fewer hyperparameters' is weakened by the per-dataset and per-epsilon tuning of β and γ. Section VI-A reports β = 0.5 for single-step at ε = 8/255 but β = 4 (CIFAR-10) and 1.5 (CIFAR-100) at higher ε, and β = 6, 3, 1 for multi-step across datasets. Table Vb shows that β strongly changes APGD-CE accuracy (52.33 to 54.24 as β goes from 2.5 to 8) and clean accuracy (82.99 to 81.31). Because the headline contribution is a simple regularizer, the paper should provide a principled selection rule for β and γ, or at least a sensitivity analysis showing that the qualitative RO/CO mitigation is robust across a range of values rather than a single tuned point.
  3. [Section IV-D and Table Va] The causal interpretation that energy smoothing is the mechanism behind DER's effectiveness is underdetermined by the present experiments. The ablation in Table Va shows that existing regularizers such as KL divergence and ALP, which also affect the energy landscape, improve robustness over SAT, but the paper does not show that DER's benefit exceeds what a generic late-stage regularizer of comparable strength would provide. To support the 'understanding' claim, the authors should include a control experiment that penalizes a different, non-energy quantity with the same schedule and magnitude, or otherwise make a falsifiable prediction that distinguishes the energy-smoothing mechanism from a generic regularization effect.
minor comments (4)
  1. [Section IV-A, text near Fig. 4] The text says 'confirmed from fig. 4a where we show that right before CO, the mean energy across all abnormal samples increases, and fig. 4a where we observe certain AAE samples appearing with higher loss'; the second reference should likely be to a different panel, since Fig. 4a alone cannot show both the mean energy trend and the loss distribution. Please correct the cross-reference.
  2. [Eq. (7) and Section IV-D] The notation L_CE(x⋆) is used without a formal definition in Eq. (7), and the sentence 'our method implicitly penalizes such samples more heavily' is misleading: the regularizer directly penalizes the norm of the delta-energy vector, so the effect on low-energy samples is explicit through the energy-dependent magnitude of that norm. Please rephrase for clarity.
  3. [Section VI-A and Eq. (8)] For the generative experiments, the values of ζ, γ, η, the SSIM threshold for the KNN selection, and the local PCA neighborhood size K are not fully specified in the text (only ζ=0.8, γ=0.001, η=0.05 and 99% variance are given). Since the generation results are a secondary contribution, please provide the complete settings to enable reproduction.
  4. [Fig. 2 caption] The caption states that all attacks are generated with an ℓ∞ constraint of ε = 8/255, including for ImageNet. This is an unusually large perturbation for ImageNet and may indicate a typo; please clarify whether ε = 4/255 was used for ImageNet, as is standard.

Circularity Check

0 steps flagged · score 2.0 of 10

No construction-level circularity: DER is an intervention on the observed diagnostic and is evaluated against external AutoAttack; the under-specified RO-onset trigger is a reproducibility caveat, not a circular derivation.

full rationale

The paper's central derivation is not circular in the construction-level sense. The energy analysis measures Delta E from logits and correlates it with CO/RO measured by test robustness; this is an empirical observation, not an identity. DER (eq. 7) penalizes the same Delta E quantity used as a diagnostic, but that is a motivated intervention rather than a prediction forced by the diagnostic: the claim is that penalizing large energy shifts improves held-out PGD/AA accuracy, which is tested against external benchmarks (Tables I-IV) and against regularizers with similar objectives (Table V). Proposition 1 restates the KL divergence using the energy definitions in eqs. (2)-(4); it is direct algebra and does not import a result. Self-citations to the authors' prior ECCV work [15] are used as a baseline and as an initialization scheme for the generation study, not as the justification for the robustness claims, so they do not make the argument circular. One caveat should be weighed: Section VI-A says the multi-step DER is applied 'preferably starting at the onset of RO' but does not specify how a practitioner detects that onset. If the onset were chosen by inspecting the test PGD curve, as Fig. 1's unshaded regions suggest, the DER-AT rows in Tables III-IV would be confounded by oracle scheduling. This is a serious reproducibility and validity risk, but the paper also documents a training-set signal (Delta E drops at RO onset, Section IV-B3) that could serve as a non-oracle trigger. Because the paper does not explicitly fit a parameter and rename it a prediction, and because the single-step CO results and the generation results are independently evaluated, this concern is not counted as construction-level circularity; it is a correctness risk that keeps the score at the low end.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The main free quantities are the DER hyperparameters, which are tuned per dataset and epsilon; the paper assumes the EBM interpretation and the representativeness of PGD/AA attacks without independent support.

free parameters (6)
  • beta_DER_single_step = 0.5 (eps 8/255), 4 (CIFAR-10) / 1.5 (CIFAR-100) at larger eps
    Weight of the DER penalty in single-step AT (RS-DER and N-DER); tuned per dataset and epsilon, critical to the reported robustness.
  • beta_DER_multi_step = 6 (CIFAR-10), 3 (CIFAR-100, TinyImageNet), 1 (SVHN)
    Weight of DER in multi-step AT; chosen per dataset.
  • gamma_DER = 0.2, except 0 for SVHN single-step
    Threshold below which no penalty is applied; tuned per setting.
  • sigma_PCA = 0.005, 0.01, 0.02 (figure); implementation says 99% retained variance
    Noise scale in local PCA initialization for generation; affects diversity/quality trade-off.
  • local_PCA_K = Not stated in text
    Number of nearest neighbors in the KNN set for local PCA; required by eq. (8) but K is not reported.
  • SGLD_hyperparameters_zeta_gamma_eta = 0.8, 0.001, 0.05
    Friction, noise variance, step size for Langevin dynamics; chosen for generation quality.
assumptions (5)
  • standard math The softmax classifier can be interpreted as an energy-based model with E(x,y)=-log f(x)[y] and E(x)=-log sum_k f(x)[k].
    Section III eq. (3); follows from [21], JEM.
  • standard math KL divergence identity in Proposition 1 is a correct algebraic expansion.
    Section IV-B1 eq. (6); verified directly from the energy definitions.
  • domain assumption The set of PGD/AA adversarial examples under L_inf is representative of the threats that matter for robust generalization.
    Used throughout experiments; the entire evaluation relies on PGD-20 and AutoAttack.
  • domain assumption Delta energy divergence observed on CIFAR-10/100, SVHN, Tiny-ImageNet with PreActResNet-18 generalizes to other architectures and datasets.
    The central correlation is claimed 'across multiple benchmarks' but tested on one architecture (ResNet-18) mostly.
  • ad hoc to paper The onset of robust overfitting can be identified during training and intervening only then is a fair evaluation.
    Section VI: 'preferably starting at the onset of RO'; this timing is a design choice that could inflate gains.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Understanding Adversarial Training with Energy-based Models." pith.science (2026). https://pith.science/paper/BFUOLKBI

@misc{pith2026250522486,
  author       = {Pith},
  title        = {Pith review of: Understanding Adversarial Training with Energy-based Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BFUOLKBI}},
  note         = {Machine review of arXiv:2505.22486}
}
read the original abstract

We aim at using Energy-based Model (EBM) framework to better understand adversarial training (AT) in classifiers, and additionally to analyze the intrinsic generative capabilities of robust classifiers. By viewing standard classifiers through an energy lens, we begin by analyzing how the energies of adversarial examples, generated by various attacks, differ from those of the natural samples. The central focus of our work is to understand the critical phenomena of Catastrophic Overfitting (CO) and Robust Overfitting (RO) in AT from an energy perspective. We analyze the impact of existing AT approaches on the energy of samples during training and observe that the behavior of the ``delta energy' -- change in energy between original sample and its adversarial counterpart -- diverges significantly when CO or RO occurs. After a thorough analysis of these energy dynamics and their relationship with overfitting, we propose a novel regularizer, the Delta Energy Regularizer (DER), designed to smoothen the energy landscape during training. We demonstrate that DER is effective in mitigating both CO and RO across multiple benchmarks. We further show that robust classifiers, when being used as generative models, have limits in handling trade-off between image quality and variability. We propose an improved technique based on a local class-wise principal component analysis (PCA) and energy-based guidance for better class-specific initialization and adaptive stopping, enhancing sample diversity and generation quality. Considering that we do not explicitly train for generative modeling, we achieve a competitive Inception Score (IS) and Fr\'echet inception distance (FID) compared to hybrid discriminative-generative models.

Figures

Figures reproduced from arXiv: 2505.22486 by the authors.

Figure 1
Figure 1. Analysis of ∆Efθ (x) .= Efθ (x) − Efθ (x ⋆ ), computed on the training set (green, right axis), and error (PGD) on test data (blue, left axis) for CIFAR-10 (left column) and CIFAR-100 (right column) across catastrophic overfitting and robust overfitting. The onset of CO or RO is indicated by the start of the unshaded region. (a) For RS-FGSM, ∆Efθ (x) increases sharply after CO, aligning with a sudden rise in test er… view at source ↗
Figure 2
Figure 2. Marginal — Efθ (x) — and joint — Efθ (x, y) — energy distributions for natural and adversarial inputs (with y as the ground-truth label), shown for CIFAR-10 (rows 1–2) and ImageNet (rows 3–4) under various untargeted and targeted (-T) attacks. For this analysis, we use non-robust models trained on CIFAR-10 and ImageNet respectively. All attacks are generated with an input deformation constraint of ℓ∞ ≤ ϵ = 8/255. in… view at source ↗
Figure 3
Figure 3. Quiver plot illustrates the energy shift between [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: (a, b) Mean ∆Efθ (x) values (left axis) and mean energy values for original samples Efθ (x) (right axis) across training epochs for AAEs and NAEs, respectively. (c) The number of AAEs (left axis) and the distribution of their losses (right axis) are tracked over the co…
Figure 5
Figure 5. Figure 5: Quiver plot now only showing all the AAEs present in [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: (a) Distribution of Cross-Entropy loss and [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Difference in the energy between natural data [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Left: Comparison of different PCA retained variances and [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: The figures display generations from the reference method [ [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

77 extracted references · 65 canonical work pages

  1. [3]

    Towards deep learning models resistant to adversarial attacks,

    A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” in ICLR, 2018

    2018

  2. [1]

    Intriguing properties of neural networks,

    C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” inICLR, 2014

    work page 2014

  3. [2]

    Explaining and harnessing adversarial examples,

    I. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” in ICLR, 2015

    work page 2015

  4. [4]

    Theoretically principled trade-off between robustness and accuracy,

    H. Zhang, Y . Yu, J. Jiao, E. P. Xing, L. E. Ghaoui, and M. I. Jordan, “Theoretically principled trade-off between robustness and accuracy,” in ICML, 2019

    work page 2019

  5. [5]

    Adversarial training for free!

    A. Shafahi, M. Najibi, M. A. Ghiasi, Z. Xu, J. Dickerson, C. Studer, L. S. Davis, G. Taylor, and T. Goldstein, “Adversarial training for free!” in NeurIPS, 2019

    work page 2019

  6. [6]

    Fast is better than free: Revisiting adversarial training,

    E. Wong, L. Rice, and J. Z. Kolter, “Fast is better than free: Revisiting adversarial training,” in ICLR, 2020

    work page 2020

  7. [7]

    Improving adversarial robustness requires revisiting misclassified examples,

    Y . Wang, D. Zou, J. Yi, J. Bailey, X. Ma, and Q. Gu, “Improving adversarial robustness requires revisiting misclassified examples,” in ICLR, 2020

    work page 2020

  8. [8]

    Towards efficient and effective adversarial training,

    G. Sriramanan, S. Addepalli, A. Baburaj et al., “Towards efficient and effective adversarial training,” NeurIPS, 2021

    work page 2021

Show all 77 references
  1. [9]

    Guided adversarial attack for evaluating and enhancing adversarial defenses,

    G. Sriramanan, S. Addepalli, A. Baburaj, and R. V . Babu, “Guided adversarial attack for evaluating and enhancing adversarial defenses,” in NeurIPS, 2020

    2020

  2. [10]

    Unlabeled data improves adversarial robustness,

    Y . Carmon, A. Raghunathan, L. Schmidt, P. Liang, and J. Duchi, “Unlabeled data improves adversarial robustness,” in NeurIPS, 2019

    2019

  3. [11]

    Improving robustness using generated data,

    S. Gowal, S.-A. Rebuffi, O. Wiles, F. Stimberg, D. A. Calian, and T. A. Mann, “Improving robustness using generated data,” in NeurIPS, 2021

    2021

  4. [12]

    Better diffusion models further improve adversarial training,

    Z. Wang, T. Pang, C. Du, M. Lin, W. Liu, and S. Yan, “Better diffusion models further improve adversarial training,” in ICML, 2023

    2023

  5. [13]

    Probabilistic margins for instance reweighting in adversarial training,

    F. Liu, B. Han, T. Liu, C. Gong, G. Niu, M. Zhou, M. Sugiyama et al., “Probabilistic margins for instance reweighting in adversarial training,” in NeurIPS, 2021

    2021

  6. [14]

    Overfitting in adversarially robust deep learning,

    L. Rice, E. Wong, and Z. Kolter, “Overfitting in adversarially robust deep learning,” in ICML, 2020

    2020

  7. [15]

    Shedding more light on robust classifiers under the lens of energy- based models,

    M. Mujtaba Hussain, B. Maria Rosaria, B. Senad, and M. Iacopo, “Shedding more light on robust classifiers under the lens of energy- based models,” in ECCV, 2024

    2024

  8. [16]

    Understanding catastrophic overfitting in single-step adversarial training,

    H. Kim, W. Lee, and J. Lee, “Understanding catastrophic overfitting in single-step adversarial training,” in AAAI Conference on Artificial Intelligence, 2021

    2021

  9. [17]

    Robustbench: a standardized adversarial robustness benchmark,

    F. Croce, M. Andriushchenko, V . Sehwag, E. Debenedetti, N. Flammar- ion, M. Chiang, P. Mittal, and M. Hein, “Robustbench: a standardized adversarial robustness benchmark,” in ICLR Workshops 2021 - Workshop on Security and Safety in Machine Learning Systems , 2021

    2021

  10. [18]

    Robust principles: Architectural design principles for adversarially robust CNNs,

    S. Peng, W. Xu, C. Cornelius, M. Hull, K. Li, R. Duggal, M. Phute, J. Martin, and D. H. Chau, “Robust principles: Architectural design principles for adversarially robust CNNs,” in BMVC, 2023

    2023

  11. [19]

    Adversarial weight perturbation helps robust generalization,

    D. Wu, S.-T. Xia, and Y . Wang, “Adversarial weight perturbation helps robust generalization,” in NeurIPS, 2020

    2020

  12. [20]

    Towards understanding the generative capability of adversarially robust classifiers,

    Y . Zhu, J. Ma, J. Sun, Z. Chen, R. Jiang, Y . Chen, and Z. Li, “Towards understanding the generative capability of adversarially robust classifiers,” in ICCV, 2021

    2021

  13. [21]

    Your classifier is secretly an energy based model and you should treat it like one,

    W. Grathwohl, K.-C. Wang, J.-H. Jacobsen, D. Duvenaud, M. Norouzi, and K. Swersky, “Your classifier is secretly an energy based model and you should treat it like one,” in ICLR, 2020

    2020

  14. [22]

    Exploring the connection between robust and generative models,

    S. Beadini and I. Masi, “Exploring the connection between robust and generative models,” in Italian Conference on AI - Ital-IA - Workshop on AI for Cybersecurity , 2023

    2023

  15. [23]

    Towards evaluating the robustness of neural networks,

    N. Carlini and D. Wagner, “Towards evaluating the robustness of neural networks,” in IEEE Symposium on Security and Privacy (SP) , 2017

    2017

  16. [24]

    Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks,

    F. Croce and M. Hein, “Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks,” in ICML, 2020

    2020

  17. [25]

    Diffusion-based adversarial sample generation for improved stealthiness and controllability,

    H. Xue, A. Araujo, B. Hu, and Y . Chen, “Diffusion-based adversarial sample generation for improved stealthiness and controllability,” in NeurIPS, 2023

    2023

  18. [26]

    Adversarial logit pairing,

    H. Kannan, A. Kurakin, and I. Goodfellow, “Adversarial logit pairing,” arXiv preprint arXiv:1803.06373 , 2018

    2018 arXiv

  19. [27]

    Eliminating catastrophic overfitting via abnormal adversarial examples regularization,

    R. Lin, C. Yu, and T. Liu, “Eliminating catastrophic overfitting via abnormal adversarial examples regularization,” in NeurIPS, 2024

    2024

  20. [28]

    On adversarial training without perturbing all examples,

    M. Losch, M. Omran, D. Stutz, M. Fritz, and B. Schiele, “On adversarial training without perturbing all examples,” in ICLR, 2024

    2024

  21. [29]

    On the over-memorization during natural, robust and catastrophic overfitting,

    R. Lin, C. Yu, B. Han, and T. Liu, “On the over-memorization during natural, robust and catastrophic overfitting,” in ICLR, 2024

    2024

  22. [30]

    Square attack: a query-efficient black-box adversarial attack via random search,

    M. Andriushchenko, F. Croce, N. Flammarion, and M. Hein, “Square attack: a query-efficient black-box adversarial attack via random search,” in ECCV, 2020

    2020

  23. [31]

    Decou- pled kullback-leibler divergence loss,

    J. Cui, Z. Tian, Z. Zhong, X. Qi, B. Yu, and H. Zhang, “Decou- pled kullback-leibler divergence loss,” arXiv preprint arXiv:2305.13948, 2023

    2023 arXiv

  24. [32]

    Ensemble adversarial training: Attacks and defenses,

    F. Tram `er, A. Kurakin, N. Papernot, I. Goodfellow, D. Boneh, and P. McDaniel, “Ensemble adversarial training: Attacks and defenses,” in ICLR, 2018

    2018

  25. [33]

    Single-step adversarial training with dropout scheduling,

    B. S. Vivek and R. Venkatesh Babu, “Single-step adversarial training with dropout scheduling,” in CVPR, 2020

    2020

  26. [34]

    Make some noise: Reliable and efficient single-step adversarial training,

    P. de Jorge Aranda, A. Bibi, R. V olpi, A. Sanyal, P. Torr, G. Rogez, and P. Dokania, “Make some noise: Reliable and efficient single-step adversarial training,” NeurIPS, 2022

    2022

  27. [35]

    Reliably fast adversarial training via latent adversarial perturbation,

    G. Y . Park and S. W. Lee, “Reliably fast adversarial training via latent adversarial perturbation,” in ICCV, 2021

    2021

  28. [36]

    Zerograd: Costless conscious remedies for catastrophic overfitting in the fgsm adversarial training,

    Z. Golgooni, M. Saberi, M. Eskandar, and M. H. Rohban, “Zerograd: Costless conscious remedies for catastrophic overfitting in the fgsm adversarial training,” Intelligent Systems with Applications , 2023

    2023

  29. [37]

    Understanding and improving fast adversarial training,

    M. Andriushchenko and N. Flammarion, “Understanding and improving fast adversarial training,” NeurIPS, 2020

    2020

  30. [38]

    Subspace adversarial training,

    T. Li, Y . Wu, S. Chen, K. Fang, and X. Huang, “Subspace adversarial training,” in CVPR, 2022

    2022

  31. [39]

    Fast adversarial training with adaptive step size,

    Z. Huang, Y . Fan, C. Liu, W. Zhang, Y . Zhang, M. Salzmann, S. S ¨usstrunk, and J. Wang, “Fast adversarial training with adaptive step size,” IEEE Transactions on Image Processing , 2023

    2023

  32. [40]

    Adver- sarially robust generalization requires more data,

    L. Schmidt, S. Santurkar, D. Tsipras, K. Talwar, and A. Madry, “Adver- sarially robust generalization requires more data,” in NeurIPS, 2018

    2018

  33. [41]

    Are labels required for improving adversarial robustness?

    J.-B. Alayrac, J. Uesato, P.-S. Huang, A. Fawzi, R. Stanforth, and P. Kohli, “Are labels required for improving adversarial robustness?” in NeurIPS, 2019

    2019

  34. [42]

    Adversarially robust generalization just requires more unlabeled data,

    R. Zhai, T. Cai, D. He, C. Dan, K. He, J. Hopcroft, and L. Wang, “Adversarially robust generalization just requires more unlabeled data,” arXiv preprint arXiv:1906.00555 , 2019

    1906 arXiv

  35. [43]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in NeurIPS, 2020

    2020

  36. [44]

    Exploring memorization in adversarial training,

    Y . Dong, K. Xu, X. Yang, T. Pang, Z. Deng, H. Su, and J. Zhu, “Exploring memorization in adversarial training,” in ICLR, 2021

    2021

  37. [45]

    Enhancing adversarial training via reweighting optimization trajectory,

    T. Huang, S. Liu, T. Chen, M. Fang, L. Shen, V . Menkovski, L. Yin, Y . Pei, and M. Pechenizkiy, “Enhancing adversarial training via reweighting optimization trajectory,” in Joint European Conference on Machine Learning and Knowledge Discovery in Databases , 2023

    2023

  38. [46]

    Sparsity winning twice: Better robust generaliztion from more efficient training,

    T. Chen, Z. Zhang, P. Wang, S. Balachandra, H. Ma, Z. Wang, and Z. Wang, “Sparsity winning twice: Better robust generaliztion from more efficient training,” arXiv preprint arXiv:2202.09844 , 2022

    2022 arXiv

  39. [47]

    Relating adversarially robust generalization to flat minima,

    D. Stutz, M. Hein, and B. Schiele, “Relating adversarially robust generalization to flat minima,” in ICCV, 2021. 15

    2021

  40. [48]

    Low curvature activations reduce overfitting in adversarial training,

    V . Singla, S. Singla, S. Feizi, and D. Jacobs, “Low curvature activations reduce overfitting in adversarial training,” in ICCV, 2021

    2021

  41. [49]

    Robust overfitting may be mitigated by properly learned smoothening,

    T. Chen, Z. Zhang, S. Liu, S. Chang, and Z. Wang, “Robust overfitting may be mitigated by properly learned smoothening,” in ICLR, 2020

    2020

  42. [50]

    Adversarial robustness through the lens of causality,

    Y . Zhang, M. Gong, T. Liu, G. Niu, X. Tian, B. Han, B. Sch ¨olkopf, and K. Zhang, “Adversarial robustness through the lens of causality,” in ICLR, 2022

    2022

  43. [51]

    Understanding robust overfitting of adversarial training and beyond,

    C. Yu, B. Han, L. Shen, J. Yu, C. Gong, M. Gong, and T. Liu, “Understanding robust overfitting of adversarial training and beyond,” in ICML, 2022

    2022

  44. [52]

    Jem++: Improved techniques for training jem,

    X. Yang and S. Ji, “Jem++: Improved techniques for training jem,” in ICCV, 2021, pp. 6494–6503

    2021

  45. [53]

    Sharpness-aware minimization for efficiently improving generalization,

    P. Foret, A. Kleiner, H. Mobahi, and B. Neyshabur, “Sharpness-aware minimization for efficiently improving generalization,” in ICLR, 2021

    2021

  46. [54]

    A unified contrastive energy- based model for understanding the generative ability of adversarial training,

    Y . Wang, Y . Wang, J. Yang, and Z. Lin, “A unified contrastive energy- based model for understanding the generative ability of adversarial training,” in ICLR, 2022

    2022

  47. [55]

    Towards bridging the performance gaps of joint energy-based models,

    X. Yang, Q. Su, and S. Ji, “Towards bridging the performance gaps of joint energy-based models,” in CVPR, 2023

    2023

  48. [56]

    M-ebm: Towards understanding the manifolds of energy-based models,

    X. Yang and S. Ji, “M-ebm: Towards understanding the manifolds of energy-based models,” in Pacific-Asia Conference on Knowledge Discovery and Data Mining . Springer, 2023, pp. 291–302

    2023

  49. [57]

    Invert- ing adversarially robust networks for image synthesis,

    R. A. Rojas-Gomez, R. A. Yeh, M. N. Do, and A. Nguyen, “Invert- ing adversarially robust networks for image synthesis,” arXiv preprint arXiv:2106.06927, 2021

    2021 arXiv

  50. [58]

    MAGIC: Mask-guided image synthesis by inverting a quasi-robust classifier,

    M. Rouhsedaghat, M. Monajatipoor, C.-C. J. Kuo, and I. Masi, “MAGIC: Mask-guided image synthesis by inverting a quasi-robust classifier,” in AAAI Conference on Artificial Intelligence , 2023

    2023

  51. [59]

    Effective single-step adversarial training with energy-based models,

    K. Tang, T. Lou, W. Peng, N. Chen, Y . Shi, and W. Wang, “Effective single-step adversarial training with energy-based models,” IEEE Trans- actions on Emerging Topics in Computational Intelligence , 2024

    2024

  52. [60]

    Geometry-aware instance-reweighted adversarial training,

    J. Zhang, J. Zhu, G. Niu, B. Han, M. Sugiyama, and M. Kankanhalli, “Geometry-aware instance-reweighted adversarial training,” in ICLR, 2020

    2020

  53. [61]

    Evaluating the robustness of geometry-aware instance-reweighted adversarial training,

    D. Hitaj, G. Pagnotta, I. Masi, and L. V . Mancini, “Evaluating the robustness of geometry-aware instance-reweighted adversarial training,” arXiv preprint arXiv:2103.01914 , 2021

    2021 arXiv

  54. [62]

    Entropy weighted adversarial training,

    M. Kim, J. Tack, J. Shin, and S. J. Hwang, “Entropy weighted adversarial training,” in ICML Workshop on Adversarial Machine Learning , 2021

    2021

  55. [63]

    Memorization weights for instance reweighting in adversarial training,

    J. Zhang, Y . Hong, and Q. Zhao, “Memorization weights for instance reweighting in adversarial training,” in AAAI Conference on Artificial Intelligence, 2023

    2023

  56. [64]

    A tutorial on energy-based learning,

    Y . LeCun, S. Chopra, R. Hadsell, M. Ranzato, and F. Huang, “A tutorial on energy-based learning,” Predicting structured data , 2006

    2006

  57. [65]

    Generative modeling by estimating gradients of the data distribution,

    Y . Song and S. Ermon, “Generative modeling by estimating gradients of the data distribution,” in NeurIPS, 2019

    2019

  58. [66]

    Diffusion models beat gans on image synthesis,

    P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” in NeurIPS, 2021

    2021

  59. [67]

    Attacks which do not kill training make adversarial learning stronger,

    J. Zhang, X. Xu, B. Han, G. Niu, L. Cui, M. Sugiyama, and M. Kankan- halli, “Attacks which do not kill training make adversarial learning stronger,” in ICML, 2020

    2020

  60. [68]

    Mma training: Direct input space margin maximization through adversarial training,

    G. W. Ding, Y . Sharma, K. Y . C. Lui, and R. Huang, “Mma training: Direct input space margin maximization through adversarial training,” in ICLR, 2020

    2020

  61. [69]

    Image quality assess- ment: From error visibility to structural similarity,

    Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image quality assess- ment: From error visibility to structural similarity,” IEEE Transactions on Image Processing , vol. 13, no. 4, pp. 600–612, 2004

    2004

  62. [70]

    Improved techniques for training gans,

    T. Salimans, I. Goodfellow, W. Zaremba, V . Cheung, A. Radford, X. Chen, and X. Chen, “Improved techniques for training gans,” in NeurIPS, 2016

    2016

  63. [71]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium,

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” in NeurIPS, 2017

    2017

  64. [72]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky, G. Hinton et al. , “Learning multiple layers of features from tiny images,” Citeseer, Tech. Rep., 2009

    2009

  65. [73]

    Reading digits in natural images with unsupervised feature learning,

    N. Yuval, “Reading digits in natural images with unsupervised feature learning,” in NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011

    2011

  66. [74]

    Imagenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, Kai Li, and Li Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in CVPR, 2009

    2009

  67. [75]

    Averaging weights leads to wider optima and better generalization,

    P. Izmailov, D. Podoprikhin, T. Garipov, D. Vetrov, and A. G. Wilson, “Averaging weights leads to wider optima and better generalization,” arXiv preprint arXiv:1803.05407 , 2018

    2018 arXiv

  68. [76]

    Learning energy-based models by diffusion recovery likelihood,

    R. Gao, Y . Song, B. Poole, Y . N. Wu, and D. P. Kingma, “Learning energy-based models by diffusion recovery likelihood,” in ICLR, 2021

    2021

  69. [77]

    Image synthesis with a single (robust) classifier,

    S. Santurkar, D. Tsipras, B. Tran, A. Ilyas, L. Engstrom, and A. Madry, “Image synthesis with a single (robust) classifier,” in NeurIPS, 2019. Mujtaba Mirza Hussain is a PhD student in Com- puter Science at Sapienza University of Rome. He holds a Bachelor’s degree from the Uni...

    2019

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.