Pith. sign in

REVIEW 3 major objections 5 minor 34 references

RCR-AF: Enhancing Model Generalization via Rademacher Complexity Reduction Activation Function

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RCR-AF, a clipped activation with two tuning parameters, claims higher CIFAR-10 accuracy and adversarial robustness than ReLU, GELU, and Swish.

desk verdict Test-set-selected hyperparameters and an unfair robustness comparison sink the empirical claim; the activation is softplus in disguise, and the theory doesn't prove what it says. read the letter →

arxiv 2507.22446 v1 pith:TJAA6KE2 submitted 2025-07-30 cs.LG cs.AIcs.CV

Yunrui Yu , Kafeng Wang , Hang Su , Jun Zhu This is my paper
classification cs.LGcs.AIcs.CV MSC 68T0768Q32
keywords activationfunctionsadversarialrobustnessRademachercomplexitygeneralizationboundsCIFAR-10AutoAttacktrainingclippedactivations
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a single activation function, RCR-AF, can improve both clean generalization and adversarial robustness at the same time. RCR-AF is the softplus-like map $\frac{1}{\alpha}\ln(1+e^{-\alpha x})+x$ with outputs clipped to $[-\gamma/\alpha,\gamma/\alpha]$, so it stays smooth and monotone while bounding layer outputs. The authors argue that the two hyperparameters directly control the model's Rademacher complexity, giving a principled reason for better generalization, and they report that on CIFAR-10 with ResNet-18 the activation reaches 96.50% clean accuracy at $\alpha=43$ and 51.96% AutoAttack robustness at $\alpha=36$, surpassing ReLU, GELU, and Swish in both settings. If correct, the result would make activation-function choice a practical, theory-motivated lever for robust deep learning.

What carries the argument

The load-bearing object is the RCR-AF activation $\mathrm{RCR}(x;\alpha,\gamma)=\frac{1}{\alpha}\ln(1+e^{-\alpha x})+x$ restricted to the clipping interval $[-\gamma/\alpha,\gamma/\alpha]$. Its derivative is the logistic sigmoid $\sigma'(x)=1/(1+e^{-\alpha x})$, which is bounded by $1/(1+e^{-\gamma})<1$ on the clipped domain, so clipping makes the activation a strict contraction. The paper feeds this contraction factor, together with the reduced operator norm and reduced covering number, into the standard Rademacher complexity bound, and the strict contraction is what turns the ReLU bound into the strictly smaller bound of Eq. (20).

What would settle it

Retrain the CIFAR-10 comparisons with $\alpha$ and $\gamma$ fixed by validation before seeing the test set, and rerun the adversarial training with three trials and error bars; if RCR-AF no longer beats ReLU and GELU, the empirical claim fails. Separately, recompute the operator-norm step in Section 3.2.2 with consistent dimensions; if the clipped bound is not strictly smaller than the ReLU bound, the theoretical claim fails.

Watch

Extended reading notes

Core claim

The central claim is that RCR-AF yields a strictly smaller Rademacher complexity bound than ReLU-based networks. Substituting the clipped operator norm $k_{i,\mathrm{clip}}=\zeta_{\mathrm{clip}} k_i$ and the clipped covering-number coefficient $b_{i,\mathrm{clip}}=\eta_{i,\mathrm{clip}} b_i$ into the usual bound gives an inequality (Eq. 20) that is strictly smaller than the ReLU bound, with the difference controlled by $\alpha$ and $\gamma$. The same mechanism preserves GELU-like smoothness and negative-information retention while adding ReLU's monotonicity, and the paper reports that this translates into higher clean accuracy and higher adversarial robustness in CIFAR-10 experiments.

Load-bearing premise

The reported improvement rests on treating the best $\alpha$ found on the test set (and, for adversarial training, a single trial) as the activation's true effect, and on the derivation that clipping strictly reduces the Rademacher complexity bound.

Editorial extensions

If this is right

  • With fixed $\gamma$, larger $\alpha$ moves RCR-AF closer to ReLU while shrinking the Rademacher bound, so the paper predicts a tunable accuracy-versus-capacity trade-off within one activation family.
  • Adversarial training with RCR-AF should inherit the complexity reduction, which the paper reports as 51.96% AutoAttack accuracy versus 49.82% for ReLU under otherwise matched training.
  • Clean training should also improve with a well-chosen $\alpha$, reported as 96.50% versus 95.98% for ReLU on CIFAR-10.
  • The mechanism gives a concrete way to choose activation parameters by complexity control instead of by exhaustive search.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test is to fix $\alpha$ and $\gamma$ on a validation split and compare against baselines; if the gap persists, the activation effect is real rather than a selection artifact.
  • Because the bound depends only on the contraction and output range, the same argument should transfer to other smooth monotone activations with bounded output; one could test whether any strictly contracting activation reproduces the reported gains.
  • The joint optimization of $\alpha$ and $\gamma$ is left to future work, and exploring it could uncover operating points beyond the single-parameter sweep reported here.
  • The complexity-reduction mechanism suggests a possible link to certified robustness radii, since bounded activations constrain the Lipschitz constant of the network; this connection is not developed in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes RCR-AF, an activation function defined as (1/α) ln(1 + e^{αx}) with an input clipping operation clip(x, [-γ/α, γ/α]), where α and γ are tunable hyperparameters. The central claims are threefold: (1) RCR-AF combines GELU-like smoothness and negative-information retention with ReLU-like monotonicity; (2) through the clipping parameters, the model's Rademacher complexity can be directly controlled and strictly reduced relative to ReLU, yielding a theoretical justification for improved generalization; and (3) experiments on CIFAR-10 with ResNet-18 show that RCR-AF outperforms ReLU, GELU, and Swish in both clean accuracy under standard training and AutoAttack robustness under adversarial training. The theoretical development (Section 3) derives Lipschitz constants, covering numbers, and a modified Rademacher bound, while the experiments (Section 4) sweep α over a range and report peak accuracies at α=43 for clean accuracy and α=36 for robustness.

Significance. If the claims were valid, the paper would contribute a new activation function with a principled, theoretically grounded mechanism for controlling model capacity and robustness, which is a timely and potentially useful direction in adversarial robustness. The act of linking an activation function's clipping hyperparameters to Rademacher complexity is a reasonable idea and the paper attempts a formal derivation. However, the significance is severely undermined by two load-bearing problems: the empirical evaluation selects α directly on the test set and uses unequal trial counts between RCR-AF and baselines, making the reported performance gains unreliable; and the theoretical derivation contains a dimensionally inconsistent operator-norm bound and a parameter error in the clipping factor, so the claimed strict complexity reduction is not proven. The paper's central claims are therefore not supported as written. The absence of error bars, validation split, and code further limits the reproducibility and evidentiary value of the experiments.

major comments (3)
  1. [§4.1, §4.2, Figs. 2–3] The empirical evaluation selects the hyperparameter α directly on the CIFAR-10 test set: no validation split is used, and the reported peaks (α=43 for clean accuracy in Fig. 2, α=36 for AutoAttack robustness in Fig. 3) are maxima over a sweep performed on the evaluation set. In addition, Fig. 3 compares a single RCR-AF trial against baselines reported as best-of-three, and no error bars or seed variance are reported anywhere. Consequently, the claimed improvements (0.52%–1.51% clean, 2.14%–4.51% robust) cannot be distinguished from selection artifacts, and the headline empirical claim is not supported.
  2. [§3.2.2, Eqs. (11)–(13), Eq. (20)] Equation (12) is dimensionally inconsistent: the term √d_i · γ/(α∥x∥_2) has units inverse to ∥x∥_2 and therefore cannot be an operator-norm bound, and the equality ∥clip(W_i)∥_op = sup ∥clip(W_i x)∥_2 omits the required normalization by ∥x∥_2 (the supremum must be taken over unit vectors, or the expression must include a division by ∥x∥_2). Equation (13) uses λ (the overflow threshold) in place of γ in the definition of ζ_clip, despite Eq. (9) establishing the Lipschitz constant as 1/(1+e^{-γ}). Because k_{i,clip} and b_{i,clip} are not correctly derived, the strict inequality in Eq. (20) is not established; the theory does not prove that RCR-AF strictly reduces the Rademacher complexity bound relative to ReLU.
  3. [§3.2.3 vs. §4.1, Fig. 2] The theoretical analysis predicts a monotonic decrease of the Rademacher complexity bound with increasing α for fixed γ, which would suggest monotone improvement in generalization. However, the empirical clean accuracy in Fig. 2 is non-monotonic: it rises to a peak at α=43 and then declines for α>50. The paper's explanation that large α 'over-constrain the model's representational capacity' is not derived from the Rademacher analysis, so the causal link between complexity reduction and the observed accuracy improvements is not validated.
minor comments (5)
  1. [Eq. (3)] The definition of RCR-AF is ambiguous about whether the clipping operation applies to the input x or to the output of the softplus expression; Section 3.2 later clarifies that the pre-activation is clipped, but the definition should state this explicitly.
  2. [§4.2 vs. Fig. 3] The robustness result for Swish is inconsistent: the text reports 47.45% for Swish, while the Figure 3 caption lists Swish as 49.36% (identical to GELU); this discrepancy should be corrected.
  3. [Figs. 2–3] The y-axis labels in Figures 2 and 3 contain placeholder text ('YULU(,; .)') that should be replaced with the actual RCR-AF notation, e.g., 'RCR-AF(α; γ)'.
  4. [§3.1, Eqs. (5)–(6)] The sparsity probability computations assume Gaussian pre-activations without empirical justification; the text should state that this is a modeling assumption rather than a property of the training distribution.
  5. [Abstract and §1] The claim that RCR-AF retains 'negative information' is misleading because the function is strictly positive for all inputs; what is retained is information about negative pre-activations through the gradient, not negative output values. The wording should be clarified.

Circularity Check

2 steps flagged · score 6.0 of 10

The reported RCR-AF gains are test-set-selected maxima, and the strict Rademacher reduction in Eq. (20) is imported through a definitional clipping factor, so the central empirical and theoretical claims are partly circular.

  1. fitted input called prediction [Section 4.1, Figure 2; Section 4.2, Figure 3]
    "For the RCR-AF activation function, we systematically tested its key parameterα across values ranging from 5 to 100. ... Notably, with α set to 43, the ResNet-18 model equipped with RCR-AF activation achieves 96.50% Clean Accuracy on the CIFAR-10 test set."

    The reported 96.50% is the maximum of an α-sweep evaluated directly on the CIFAR-10 test set; no validation split is used to select α. Comparing this test-selected maximum against fixed ReLU/GELU/Swish baselines is biased, because the RCR-AF number is max_α TestAcc(α) whereas each baseline is a single fixed-configuration accuracy. The same protocol is used for robustness (α=36), and Figure 3's caption admits baselines are best-of-three while RCR-AF is single-trial. The conclusion that RCR-AF consistently outperforms therefore reduces to the test-set selection rule, not to an independently predicted property of the activation or to the Rademacher mechanism.

  2. self definitional [Section 3.2.2, Eq. (13); Section 3.2.3, Eq. (20)]
    "To maintain compatibility with the Rademacher complexity derivation presented in [34] (Chapter 5, pp. 62–67), we define the operator norm for the i-th network layer as: ki,clip = ζclipki, (13) where ζclip = min(1,√di· γ/(α∥x∥2 ki))/(1+e^{-λ}) < 1 the clipping factor that captures the reduction in operator norm."

    Equation (20) asserts a strict Rademacher reduction after substituting ki,clip = ζclipki and bi,clip = ηi,clipbi. The strict inequality rests on ζclip<1, which is stipulated in the definition of ki,clip in Eq. (13), not derived from a valid operator-norm bound: Eq. (12) is dimensionally inconsistent and Eq. (13) uses λ instead of the γ from the Lipschitz computation in Eq. (9). Hence the theoretical prediction that RCR-AF strictly lowers the bound is true by construction of the clipping factor; without accepting that definitional ζclip<1, no derivation is supplied.

full rationale

The paper does not rest on a self-citation chain: the Rademacher framework is taken from an external Stanford lecture-notes source [34], and the authors' own prior papers are not load-bearing for the central claim. The two substantive circularities are empirical and theoretical. Empirically, Section 4.1 sweeps α over 5–100 and reports the peak at α=43 on the CIFAR-10 test set itself, with no validation split; Section 4.2 does the same at α=36 for AutoAttack, and Figure 3's own caption states that RCR-AF is single-trial while baselines are best-of-three. Consequently, the headline 'RCR-AF outperforms ReLU/GELU/Swish' is the maximum of a test-set hyperparameter search compared against fixed baselines, i.e., a selected statistic rather than a prediction of the activation function's behavior. Theoretically, the strict inequality in Eq. (20) is made true by defining the clipping factor ζclip<1 in Eq. (13); the preceding derivation is not an independent bound because Eq. (12) is dimensionally inconsistent and Eq. (13) substitutes λ for γ. Because the central empirical claim reduces to test-set selection and the theoretical claim partly reduces to a definitional inequality, a score of 6 is appropriate rather than a lower score for minor self-citation or a higher score for full circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on two free hyperparameters (α tuned on the test set, γ fixed by hand), a standard Rademacher bound that is borrowed from lecture notes, a Gaussian assumption for sparsity estimates, and an unproven substitution of modified weight norms. No new physical or model entities are introduced.

free parameters (2)
  • α = 43 (clean accuracy), 36 (robust accuracy), swept over [5,100] and [5,50] respectively
    The central empirical claim depends on choosing the best α on the test set; the paper reports the peak of the sweep, so this is a free parameter fitted to the data.
  • γ = 66.72 (approximately, held constant)
    γ is fixed throughout; the paper does not explore joint tuning. Its value is chosen by hand, and the theory's reduction factor depends on it.
assumptions (4)
  • standard math Standard Rademacher complexity bound for deep networks (Theorem 5.18 and related bounds from Ma's lecture notes [34])
    Eq. (20) substitutes k_i,clip and b_i,clip into this existing bound; the paper does not re-derive it.
  • domain assumption Pre-activations z_j follow a Gaussian distribution N(0, σ_j^2) for sparsity probability calculations
    Eq. (5) and (6) compute P(|z|>γ/α) under a Gaussian assumption on pre-activations, an empirical approximation not justified for trained networks.
  • standard math Weight matrices satisfy ∥W_i∥_op ≤ k_i and ∥W_i^T∥_{2,1} ≤ b_i, and inputs satisfy ∥x∥₂ ≤ c
    Standard assumptions in Rademacher complexity analysis; used to define the function class F_i in Section 3.2.
  • ad hoc to paper The clipping operation is 1-Lipschitz and the Rademacher bound remains valid after replacing k_i and b_i with k_i,clip and b_i,clip
    Section 3.2.2 claims direct substitution into [34]'s framework; this is not proven and the derived k_i,clip is based on a dimensionally inconsistent bound.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RCR-AF: Enhancing Model Generalization via Rademacher Complexity Reduction Activation Function." pith.science (2026). https://pith.science/paper/TJAA6KE2

@misc{pith2026250722446,
  author       = {Pith},
  title        = {Pith review of: RCR-AF: Enhancing Model Generalization via Rademacher Complexity Reduction Activation Function},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TJAA6KE2}},
  note         = {Machine review of arXiv:2507.22446}
}
abstract

Despite their widespread success, deep neural networks remain critically vulnerable to adversarial attacks, posing significant risks in safety-sensitive applications. This paper investigates activation functions as a crucial yet underexplored component for enhancing model robustness. We propose a Rademacher Complexity Reduction Activation Function (RCR-AF), a novel activation function designed to improve both generalization and adversarial resilience. RCR-AF uniquely combines the advantages of GELU (including smoothness, gradient stability, and negative information retention) with ReLU's desirable monotonicity, while simultaneously controlling both model sparsity and capacity through built-in clipping mechanisms governed by two hyperparameters, $\alpha$ and $\gamma$. Our theoretical analysis, grounded in Rademacher complexity, demonstrates that these parameters directly modulate the model's Rademacher complexity, offering a principled approach to enhance robustness. Comprehensive empirical evaluations show that RCR-AF consistently outperforms widely-used alternatives (ReLU, GELU, and Swish) in both clean accuracy under standard training and in adversarial robustness within adversarial training paradigms.

Figures

Figures reproduced from arXiv: 2507.22446 by the authors.

Figure 1
Figure 1. Comparison of nonlinear activation functions: ReLU (σ(x) = max(0, x)), GELU (σ(x) ≈ x/(1 +e −1.702x )), Swish (σ(x) = x/(1 + e −βx) with β = 1), and RCR-AF (σ(x) = α −1 ln(1+e −αx)+x) with α ∈ {1, 10, 20}. RCR-AF preserves GELU’s smoothness and Swish’s gradient continuity while maintaining ReLU’s strict monotonicity - a property neither GELU nor Swish possess. As α increases to 10 (green dash-dot) and 20 (yellow das… view at source ↗
Figure 2
Figure 2. Clean accuracy comparison on CIFAR-10 (ResNet-18) with all values representing the best [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Robustness comparison of activation functions under PGD adversarial training on CIFAR [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

34 extracted references · 22 canonical work pages

  1. [1]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770–778, 2016

    2016

  2. [2]

    Imagenet classification with deep convolutional neural networks

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84–90, 2017

    2017

  3. [3]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017

    2017

  4. [4]

    Deep speech 2: End-to-end speech recognition in english and mandarin

    Dario Amodei, Sundaram Ananthanarayanan, Rishita Anubhai, Jingliang Bai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Qiang Cheng, Guoliang Chen, et al. Deep speech 2: End-to-end speech recognition in english and mandarin. In International conference on machine learning , pages 173–182. PMLR, 2016

    work page 2016

  5. [5]

    Intriguing properties of neural networks

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013

    arXiv 2013

  6. [6]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  7. [7]

    Explaining and harnessing adversarial examples

    Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations , 2015

    work page 2015

  8. [8]

    Deepfool: a simple and accurate method to fool deep neural networks

    Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2574–2582, 2016

    work page 2016

Show all 34 references
  1. [9]

    Carlini and D

    N. Carlini and D. Wagner. Towards evaluating the robustness of neural networks. In2017 IEEE Symposium on Security and Privacy (SP) , pages 39–57, 2017

    2017

  2. [10]

    To- wards deep learning models resistant to adversarial attacks

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. To- wards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018

    2018

  3. [11]

    Y . Dong, F. Liao, T. Pang, H. Su, J. Zhu, X. Hu, and J. Li. Boosting adversarial attacks with momentum. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9185–9193, 2018

    2018

  4. [12]

    Generating adversarial examples with adversarial networks

    Chaowei Xiao, Bo Li, Jun-Yan Zhu, Warren He, Mingyan Liu, and Dawn Song. Generating adversarial examples with adversarial networks. In IJCAI, 2018

    2018

  5. [13]

    Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks

    Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In ICML, 2020

    2020

  6. [14]

    Adversarial training for free! In Advances in Neural Information Processing Systems, volume 32, pages 3358–3369, 2019

    Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In Advances in Neural Information Processing Systems, volume 32, pages 3358–3369, 2019

    2019

  7. [15]

    Are labels required for improving adversarial robustness? In Advances in Neural Information Processing Systems, volume 32, 2019

    Jean-Baptiste Alayrac, Jonathan Uesato, Po-Sen Huang, Alhussein Fawzi, Robert Stanforth, and Pushmeet Kohli. Are labels required for improving adversarial robustness? In Advances in Neural Information Processing Systems, volume 32, 2019

    2019

  8. [16]

    Theoreti- cally principled trade-off between robustness and accuracy

    Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoreti- cally principled trade-off between robustness and accuracy. In ICML, 2019

    2019

  9. [17]

    Boosting adversarial training with hypersphere embedding

    Tianyu Pang, Xiao Yang, Yinpeng Dong, Kun Xu, Hang Su, and Jun Zhu. Boosting adversarial training with hypersphere embedding. In NeurIPS, 2020

    2020

  10. [18]

    Improving adver- sarial robustness requires revisiting misclassified examples

    Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adver- sarial robustness requires revisiting misclassified examples. In International Conference on Learning Representations, 2020

    2020

  11. [19]

    Adversarial weight perturbation helps robust generalization

    Dongxian Wu, Shu tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. In NeurIPS, 2020

    2020

  12. [20]

    Does network width really help adversarial robustness? arXiv 2010.01279, 2020

    Boxi Wu, Jinghui Chen, Deng Cai, Xiaofei He, and Quanquan Gu. Does network width really help adversarial robustness? arXiv 2010.01279, 2020. 10

    2010 arXiv

  13. [21]

    Lafeat: Piercing through adversarial defenses with latent features

    Yunrui Yu, Xitong Gao, and Cheng-Zhong Xu. Lafeat: Piercing through adversarial defenses with latent features. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 5735–5745, 2021

    2021

  14. [22]

    Mora: Improving ensemble robustness evaluation with model reweighing attack

    Xitong Gao, Cheng-Zhong Xu, et al. Mora: Improving ensemble robustness evaluation with model reweighing attack. Advances in Neural Information Processing Systems , 35:26955–26965, 2022

    2022

  15. [23]

    Lafit: Efficient and reliable evaluation of adversarial defenses with latent features

    Yunrui Yu, Xitong Gao, and Cheng-Zhong Xu. Lafit: Efficient and reliable evaluation of adversarial defenses with latent features. IEEE Transactions on Pattern Analysis and Machine Intelligence , 46(1):354– 369, 2023

    2023

  16. [24]

    Boosting adversarial training with hypersphere embedding

    Tianyu Pang, Xiao Yang, Yinpeng Dong, Kun Xu, Jun Zhu, and Hang Su. Boosting adversarial training with hypersphere embedding. Advances in Neural Information Processing Systems , 33:7779–7792, 2020

    2020

  17. [25]

    Fixing data augmentation to improve adversarial robustness

    Sylvestre-Alvise Rebuffi, Sven Gowal, Dan A Calian, Florian Stimberg, Olivia Wiles, and Timothy Mann. Fixing data augmentation to improve adversarial robustness. arXiv preprint arXiv:2103.01946, 2021

    2021 arXiv

  18. [26]

    Towards deep learning models resistant to adversarial attacks

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017

    2017 arXiv

  19. [27]

    Theo- retically principled trade-off between robustness and accuracy

    Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theo- retically principled trade-off between robustness and accuracy. In International conference on machine learning, pages 7472–7482. PMLR, 2019

    2019

  20. [28]

    Improving adver- sarial robustness requires revisiting misclassified examples

    Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adver- sarial robustness requires revisiting misclassified examples. In International conference on learning representations, 2019

    2019

  21. [29]

    Uncovering the limits of adversarial training against norm-bounded adversarial examples

    Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint arXiv:2010.03593, 2020

    2010 arXiv

  22. [30]

    Mitigating adversarial effects through randomization

    Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. arXiv preprint arXiv:1711.01991, 2017

    2017 arXiv

  23. [31]

    Gaussian error linear units (gelus)

    Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016

    2016 arXiv

  24. [32]

    Adversarial weight perturbation helps robust generalization

    Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. Advances in neural information processing systems , 33:2958–2969, 2020

    2020

  25. [33]

    Searching for activation functions

    Prajit Ramachandran, Barret Zoph, and Quoc V Le. Searching for activation functions. arXiv preprint arXiv:1710.05941, 2017

    2017 arXiv

  26. [34]

    Lecture notes for machine learning theory

    Tengyu Ma. Lecture notes for machine learning theory. Stanford University Course Notes, 2020. Accessed: 2023-08-20. 11 6 Technical Appendices and Supplementary Material 6.1 Limitations of Hyperparameter Exploration While the proposed RCR-AF activation function demonstrates sig...

    2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.