Pith. sign in

REVIEW 4 major objections 5 minor 64 references

Enhancing Robust Fairness via Confusional Spectral Regularization

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper derives a PAC-Bayesian bound showing that worst-class adversarial error is controlled by the spectral norm of the empirical robust confusion matrix plus a model-complexity term, and then introduces a regularizer that shrinks…

desk verdict A worthwhile empirical paper on worst-class robust accuracy, but the regularizer's connection to the stated PAC-Bayes bound rests on an unproven surrogate assumption that a referee should push on hard. read the letter →

arxiv 2501.13273 v1 pith:H5DZGZZ6 submitted 2025-01-22 cs.LG

Gaojie Jin , Sihao Wu , Jiaxu Liu , Tianjin Huang , Ronghui Mu This is my paper · ORCID
classification cs.LG
keywords robustfairnessworst-classerrorconfusionmatrixspectralnormPAC-Bayesianboundadversarialtrainingregularizationclassimbalance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper takes on robust fairness, the observation that adversarial robustness varies sharply across classes, and replaces explicit class reweighting with a spectral-norm regularizer. It derives a PAC-Bayesian generalization bound for the worst-class robust error, showing that this error is bounded by the spectral norm of the empirical robust confusion matrix plus a term capturing model and data complexity. The paper then constructs a differentiable surrogate confusion matrix, whose off-diagonal entries are average KL divergences, and regularizes its spectral norm during training. The key algorithmic assumption is that descending the surrogate also descends the true confusion matrix; under that assumption, the regularizer should improve worst-class robust accuracy without the train-test divergence that plagues reweighting methods. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet report consistent gains in worst-class robust accuracy.

What carries the argument

The central object is the robust confusion matrix $C_{S',\gamma}^{f_w}$, whose off-diagonal entry $(i,j)$ counts how often a true class-$j$ sample is classified as class $i$ under adversarial perturbation within margin $\gamma$; its $\ell^1$ norm equals the worst-class robust error, and its spectral norm is the largest singular value. The proof machinery is a chain of PAC-Bayesian inequalities over these matrices, culminating in Proposition 3.1, together with Perron-Frobenius monotonicity, which lets the proof compare spectral norms entrywise. The algorithmic machinery is the KL-surrogate matrix $L_{S',\gamma}^{f_w}$, which replaces binary error indicators with average KL divergences so that the spectral norm's gradient can be computed by backpropagation; the load-bearing sign approximation $\mathrm{sign}\bigl(\partial (C_{S',\gamma}^{f_w})_{ij}/\partial (L_{S',\gamma}^{f_w})_{ij}\bigr)\approx 1$ connects the optimized surrogate to the true matrix.

What would settle it

Take a trained model and a minibatch of adversarial examples; compute the spectral norm of the true binary confusion matrix before and after one optimization step that decreases the spectral norm of the KL surrogate. If the true spectral norm increases on a substantial fraction of such steps, the sign approximation in Eq. (11) is violated and the regularizer can move the bound in the wrong direction.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is a robust generalization bound for the worst-class robust error: $\|C_{D'}^{f_w}\|_1 \le \nu\|C_{S',\gamma}^{f_w}\|_2 + O\!\left(\sqrt{\frac{\nu^2 d_y}{(m_{\min}-8d_y)\gamma^2}\left(\Phi'(f_w)+\ln\frac{n m_{\min}}{\delta}\right)}\right)$. The first term is the spectral norm of the empirical robust confusion matrix; the second term is a weight-norm and sample-complexity term familiar from PAC-Bayesian generalization theory. The proof chains a Gibbs-classifier PAC-Bayesian bound on confusion matrices, a deterministic margin-based spectral bound, and a local-perturbation step that transfers the clean bound to adversarial inputs. The paper concludes that robust fairness is not primarily a weighting problem but a spectral problem, and that regularizing the confusion matrix's spectral norm should improve worst-class robust accuracy.

Load-bearing premise

The method depends on the assumption that the gradient direction of the true binary confusion matrix aligns with the gradient direction of the differentiable KL surrogate; if they point opposite ways in common situations, shrinking the surrogate can inflate the true matrix's spectral norm and hurt worst-class accuracy.

Editorial extensions

If this is right

  • If the bound holds, any method that shrinks the empirical robust confusion matrix's spectral norm also shrinks an upper bound on the worst-class robust error, independent of how classes are weighted.
  • The regularizer should remain effective when training-set and test-set worst classes disagree, because it does not require identifying a worst class at all.
  • Fine-tuning a strong adversarially pretrained model for a few epochs with the regularizer should raise worst-class robust accuracy while roughly preserving average accuracy.
  • The same reasoning should apply to non-adversarial fairness: regularizing the clean confusion matrix's spectral norm improves hard-class accuracy, as the paper's clean-training and long-tail experiments suggest.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the theory bounds the true binary confusion matrix while the method optimizes a KL surrogate, the most direct stress test is to log both spectral norms during training; the paper reports end results but not this comparison curve.
  • The two terms of the bound are additive, so combining this confusion-spectral regularizer with existing weight spectral-normalization techniques may yield orthogonal gains; that combination is not tested in the paper.
  • The surrogate-matrix trick could be transferred to other nonsmooth fairness objectives by redefining the matrix over groups rather than classes, which would make the same spectral argument applicable to group-conditional error rates.
  • A quantitative prediction of the bound is that worst-class robust error should grow with class imbalance through the $m_{\min}$ term; experiments varying class frequency directly would isolate that effect.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper derives a PAC-Bayesian bound on the worst-class robust error of a ReLU feedforward network, expressing it as the spectral norm of an empirical robust margin confusion matrix plus a weight-norm and sample-complexity term (Prop. 3.1). It then proposes a regularizer that aims to reduce this spectral norm by replacing the nondifferentiable binary confusion matrix with a smooth KL-divergence surrogate (Eq. (10) and (11)), and validates the approach through fine-tuning and from-scratch adversarial training experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet, reporting improved worst-class AutoAttack accuracy over existing robust-fairness methods.

Significance. If the bound and the surrogate connection were both rigorous, this would be a valuable contribution: it would be the first PAC-Bayesian characterization of worst-class robust error and would provide a principled alternative to explicit reweighting, which the paper shows can worsen train/test class-wise divergence. The experimental study is extensive (fine-tuning on DDPM-trained models, from-scratch training, multiple attacks and datasets, sensitivity analysis, long-tail and clean-training extensions), and the authors provide a public code link. The main weaknesses are in the theoretical chain: a missing condition that breaks the bound on two of the three evaluated datasets, an invalid intermediate step in the proof of Lemma 3.2, an incompletely specified constant ν in the leading term, and an unvalidated surrogate-gradient approximation that is the linchpin of the method. These issues currently block the paper's central claim that the regularizer directly implements the bound.

major comments (4)
  1. [Prop. 3.1 and Lem. 3.2–3.4] The bound is stated without the necessary condition mmin > 8dy. The denominator mmin − 8dy appears in Thm. 2.1 and in Eqs. (6), (7), and (8), and is positive only when mmin > 8dy. On CIFAR-100 (mmin ≈ 500, dy = 100) and Tiny-ImageNet (mmin ≈ 500, dy = 200) the denominator is negative, so the square root is not defined over the reals and the claimed probability bound cannot hold. The experimental sections, particularly Table 2 and Table 3, include these datasets. The condition must be stated explicitly, and the theoretical claims should be restricted to settings where it holds, or a separate argument must be provided for the missing regime.
  2. [App. B, Proof B.2] The proof of Lemma 3.2 claims entrywise domination (C^fw_D)_{ij} ≤ (C^{\tilde Q}_{D,γ/2})_{ij} for all i ≠ j, relying on the perturbation bound |fw+u(x) − fw(x)|∞ < γ/4. This entrywise claim is false because the row index of the margin confusion matrix is the argmax over non-true classes of fw+u, which can shift away from the argmax of fw. Concretely, with dy = 3, take a sample of true class 3 that fw misclassifies as class 1, and suppose after the perturbation the top non-true class becomes class 2. Then (C^fw_D)_{1,3} includes this sample, while (C^{\tilde Q}_{D,γ/2})_{1,3} does not; the sample may be counted in row 2 instead. Thus the entrywise inequality used to apply Perron–Frobenius is not valid in general. The proof of Lemma 3.2, and therefore the chain leading to Prop. 3.1, requires repair.
  3. [End of Sec. 3.1 (paragraph after Lem. 3.4)] The constant ν in Eq. (5) is not specified. The text says ν depends on dy, is upper bounded by sqrt(dy), and reports a numerical study with 1,000,000 random confusion matrices for dy = 10 giving a maximum of 1.16 and an average of 1.06. This simulation is not reproducible (no distribution over confusion matrices is given) and is inconsistent with the universal bound: a matrix with a single column of ones and all other entries zero is a valid confusion-matrix pattern with ||C||1/||C||2 = sqrt(dy − 1), which is ≈ 3 for dy = 10. Since ν multiplies the leading spectral-norm term, an unspecified ν makes the bound non-computable, and the claimed tightness is unsupported. The proposition should either set ν = sqrt(dy) explicitly or give a rigorous definition of ν with a corresponding bound, and the numerical study should either be properly specified or removed.
  4. [Sec. 4.1, Eqs. (9)–(11)] The regularizer's connection to the theory is not established. The bound controls ||C^fw_{S',γ}||2, but the algorithm minimizes the spectral norm of the KL surrogate L^fw_{S',γ} by assuming sign(∂C_ij/∂L_ij) ≈ 1. Because C_ij is a piecewise-constant ({0,1}) function of the weights, the derivative ∂C_ij/∂L_ij is not an ordinary derivative; interpreting it as a unit-scaled discrete difference discards the boundary terms that arise when the set S'_ij (which itself depends on fw) changes. The analogy to cross-entropy versus classification error is suggestive but not adequate, since cross-entropy is a calibrated surrogate for expected loss, here the surrogate is used entrywise on a hard indicator and no monotonicity is shown. This is load-bearing: if the surrogate descent direction does not reduce ||C^fw_{S',γ}||2, the method regularizes a different quantity from the one in Prop. 3.1. Please provide direct evidence, e.g., plot ||C^fw_{S',γ}||2 during training with and without the regularizer, or re-frame the method as an empirically motivated heuristic rather than a direct realization of the bound.
minor comments (5)
  1. [Eq. (1)] The condition in the definition of c_ij is written as max_{i≠i'} fw(x)[i'] — the subscript is ambiguous and appears to be a typo. It should be max_{i'≠i} fw(x)[i'].
  2. [Lem. 3.2] The notation DKL(w + u ‖ P) is nonstandard; the KL term is between the posterior distribution of w + u and the prior P. Writing DKL(Q ‖ P) or clarifying that Q is the shifted distribution would avoid confusion.
  3. [Fig. 2 caption] The caption mentions covariance and Kendall rank correlation but does not define the exact quantities plotted or the computation procedure; please add a precise definition in the caption or the main text.
  4. [Sec. 4.1, text before Eq. (12)] The word 'notional' in 'notional convenience' should be 'notational'.
  5. [Tables 1–5] The main results appear to be single-run evaluations; reporting the number of random seeds and the variance of the worst-class accuracy would materially strengthen the empirical claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the PAC-Bayes bound is assembled from external benchmarks, and the regularizer is a surrogate heuristic rather than a restatement of the bound.

full rationale

The paper's derivation chain is self-contained against external results. Proposition 3.1 bounds the worst-class robust error by the spectral norm of the empirical robust confusion matrix plus a weight-norm complexity term; the proof chains external PAC-Bayes bounds (Morvant et al. 2012; Neyshabur et al. 2017b; Xiao et al. 2023) through Lemmas 3.2-3.4 and the valid inequality ||C||_1 <= nu ||C||_2. No step defines the target quantity in terms of the regularizer, and the reported worst-class accuracies are measured, not fitted constants. The algorithm in Section 4.1, Eqs. (9)-(11), does not directly optimize ||C||_2; it substitutes a differentiable KL-surrogate L and assumes sign(dC_ij/dL_ij) ~= 1. This is an unproven surrogate assumption and a legitimate correctness risk, but it is not circular: the surrogate is not constructed from the bound's conclusion, and the experiments evaluate held-out worst-class accuracy rather than re-fitting the bound. Self-citations (Jin et al.; Huang et al.) are contextual and not load-bearing. Hence the circularity score is 0.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The central bound rests on standard PAC-Bayesian theory and perturbation bounds from prior work, plus the unproven sign alignment between the binary confusion matrix and the KL surrogate, and the unstated mmin > 8dy condition. The only fitted quantities in the method are the hyperparameters alpha and gamma.

free parameters (2)
  • alpha (regularization weight) = 0.3 (default; swept over {0.0, 0.1, 0.2, 0.3, 0.4})
    Balances the adversarial loss and the spectral regularizer in Eq. (12); chosen via sensitivity analysis in App. E.1, and the reported gains depend on this choice.
  • gamma (margin) = 0.0 and 0.1 (defaults; swept over {0.0, 0.1, 0.2, 0.3, 0.4})
    Appears in the margin confusion matrix and the KL surrogate in Eq. (10); sensitivity analysis in App. E.1 shows worst-class AA peaks near gamma=0.1.
assumptions (6)
  • standard math PAC-Bayesian generalization theory, specifically the McAllester/Morvant bound for Gibbs classifiers (Thm 2.1)
    Foundation of the chain derivation in Sec. 3; the paper does not re-derive it.
  • standard math Perron-Frobenius theorem implies d||C||_2/dC_ij >= 0 for nonnegative C
    Used in Proofs B.2-B.4 to compare spectral norms entrywise. This is a standard linear algebra result.
  • domain assumption Local perturbation bounds and sharpness limits from Neyshabur et al. (2017b) and Xiao et al. (2023) hold for the considered networks
    Assumed to derive Lems. C.2 and D.3-D.5; the bound's Phi' term inherits these.
  • domain assumption Perturbation u satisfies the sharpness condition P_u(max |fw+u-fw| < gamma/4) >= 1/2
    Standard PAC-Bayes assumption used in Lem. 3.2 to restrict the posterior; the KL bound in Lem. 3.3 relies on it.
  • ad hoc to paper sign(dC_ij/dL_ij) is approximately 1 for the surrogate matrix
    Unproved approximation in Sec. 4.1 (Eqs. 9-11) that justifies replacing the binary confusion matrix with the KL matrix; the entire regularizer's correctness rests on it.
  • domain assumption mmin > 8dy is assumed for the bound to be real
    The denominator (mmin - 8dy) under the square root in Prop. 3.1 requires this; the paper states the bound unconditionally, so this is an unflagged necessary condition.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Robust Fairness via Confusional Spectral Regularization." pith.science (2026). https://pith.science/paper/H5DZGZZ6

@misc{pith2026250113273,
  author       = {Pith},
  title        = {Pith review of: Enhancing Robust Fairness via Confusional Spectral Regularization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H5DZGZZ6}},
  note         = {Machine review of arXiv:2501.13273}
}
read the original abstract

Recent research has highlighted a critical issue known as ``robust fairness", where robust accuracy varies significantly across different classes, undermining the reliability of deep neural networks (DNNs). A common approach to address this has been to dynamically reweight classes during training, giving more weight to those with lower empirical robust performance. However, we find there is a divergence of class-wise robust performance between training set and testing set, which limits the effectiveness of these explicit reweighting methods, indicating the need for a principled alternative. In this work, we derive a robust generalization bound for the worst-class robust error within the PAC-Bayesian framework, accounting for unknown data distributions. Our analysis shows that the worst-class robust error is influenced by two main factors: the spectral norm of the empirical robust confusion matrix and the information embedded in the model and training set. While the latter has been extensively studied, we propose a novel regularization technique targeting the spectral norm of the robust confusion matrix to improve worst-class robust accuracy and enhance robust fairness. We validate our approach through comprehensive experiments on various datasets and models, demonstrating its effectiveness in enhancing robust fairness.

Figures

Figures reproduced from arXiv: 2501.13273 by the authors.

Figure 1
Figure 1. Illustration of the theoretical framework: worst-class robust generalization bound. Under [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. (right), our developed method maintains significantly lower training-test divergence (higher training-test correlation) compared to explicit reweighting approaches. Note that our method is not primarily designed to address the training-test divergence, which may be inherent to the dataset, but rather to avoid explicit reweighting that can exacerbate this divergence and limit effectiveness [PITH_FULL_IMAGE:figures/f… view at source ↗
Figure 3
Figure 3. In both confusion matrices, the hori￾zontal axis represents the true labels, while the vertical axis represents the predicted labels. The left figure shows the AA results of a WRN-34- 10 model trained using the TRADES method on CIFAR-10, whereas the right figure demonstrates the AA results of a WRN-34-10 model trained us￾ing our method with γ = 0.1. In this section, we discuss the fine-tuning per￾formance and the ad… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: We adversarially trained Preact-ResNet-18 models for standard TRADES (left) and our method with γ = 0.0 based on TRADES (right) for 200 epochs using SGD with a momentum of 0.9, batch size of 256, weight decay of 5 × 10−4 , and an initial learning rate of 0.1, which is …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

64 extracted references · 37 canonical work pages

  1. [1]

    A reductions approach to fair classification

    Alekh Agarwal, Alina Beygelzimer, Miroslav Dud \' k, John Langford, and Hanna Wallach. A reductions approach to fair classification. In International conference on machine learning, 2018

    work page 2018

  2. [2]

    Square attack: a query-efficient black-box adversarial attack via random search

    Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: a query-efficient black-box adversarial attack via random search. In ECCV, 2020

    work page 2020

  3. [3]

    Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples

    Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In ICML, 2018

    2018

  4. [4]

    The spectral norm of gaussian matrices with correlated entries

    Afonso S Bandeira and March T Boedihardjo. The spectral norm of gaussian matrices with correlated entries. arXiv preprint arXiv:2104.02662, 2021

    arXiv 2021

  5. [5]

    Spectrally-normalized margin bounds for neural networks

    Peter L Bartlett, Dylan J Foster, and Matus J Telgarsky. Spectrally-normalized margin bounds for neural networks. Advances in neural information processing systems, 30, 2017

    2017

  6. [6]

    Evasion attacks against machine learning at test time

    Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim S rndi \'c , Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In ECML PKDD, 2013

    work page 2013

  7. [7]

    Towards evaluating the robustness of neural networks

    Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pp.\ 39--57. IEEE, 2017

    work page 2017

  8. [8]

    Minimally distorted adversarial examples with a fast adaptive boundary attack

    Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with a fast adaptive boundary attack. In ICML, 2020 a

    work page 2020

Show all 64 references
  1. [9]

    Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks

    Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In ICML, 2020 b

    2020

  2. [10]

    Learnable boundary guided adversarial training

    Jiequan Cui, Shu Liu, Liwei Wang, and Jiaya Jia. Learnable boundary guided adversarial training. In ICCV, 2021

    2021

  3. [11]

    Classes are not equal: An empirical study on image recognition fairness

    Jiequan Cui, Beier Zhu, Xin Wen, Xiaojuan Qi, Bei Yu, and Hanwang Zhang. Classes are not equal: An empirical study on image recognition fairness. In CVPR, 2024

    2024

  4. [12]

    Class-balanced loss based on effective number of samples

    Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge Belongie. Class-balanced loss based on effective number of samples. In CVPR, 2019

    2019

  5. [13]

    Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data

    Gintare Karolina Dziugaite and Daniel M Roy. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data. arXiv preprint arXiv:1703.11008, 2017

    2017 arXiv

  6. [14]

    Evaluating and understanding the robustness of adversarial logit pairing

    Logan Engstrom, Andrew Ilyas, and Anish Athalye. Evaluating and understanding the robustness of adversarial logit pairing. arXiv preprint arXiv:1807.10272, 2018

    2018 arXiv

  7. [15]

    Generalizable adversarial training via spectral normalization

    Farzan Farnia, Jesse M Zhang, and David Tse. Generalizable adversarial training via spectral normalization. In ICLR, 2019

    2019

  8. [16]

    \"U ber matrizen aus nicht negativen elementen

    Georg Frobenius, Ferdinand Georg Frobenius, Ferdinand Georg Frobenius, Ferdinand Georg Frobenius, and Germany Mathematician. \"U ber matrizen aus nicht negativen elementen. 1912

    1912

  9. [17]

    Risk bounds for the majority vote: From a pac-bayesian analysis to a learning algorithm

    Pascal Germain, Alexandre Lacasse, Francois Laviolette, Mario Marchand, and Jean-Francis Roy. Risk bounds for the majority vote: From a pac-bayesian analysis to a learning algorithm. arXiv preprint arXiv:1503.08329, 2015

    2015 arXiv

  10. [18]

    Goodfellow, Jonathon Shlens, and Christian Szegedy

    Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015

    2015

  11. [19]

    Fairness without demographics in repeated loss minimization

    Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In ICML, 2018

    2018

  12. [20]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016

    2016

  13. [21]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 0 6840--6851, 2020

    2020

  14. [22]

    On generalization of graph autoencoders with adversarial training

    Tianjin Huang, Yulong Pei, Vlado Menkovski, and Mykola Pechenizkiy. On generalization of graph autoencoders with adversarial training. In ECML PKDD, 2021

    2021

  15. [23]

    Enhancing adversarial training via reweighting optimization trajectory

    Tianjin Huang, Shiwei Liu, Tianlong Chen, Meng Fang, Li Shen, Vlado Menkovski, Lu Yin, Yulong Pei, and Mykola Pechenizkiy. Enhancing adversarial training via reweighting optimization trajectory. In ECML, 2023

    2023

  16. [24]

    Fantastic generalization measures and where to find them

    Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantastic generalization measures and where to find them. International Conference on Learning Representations, 2020

    2020

  17. [25]

    How does weight correlation affect the generalisation ability of deep neural networks

    Gaojie Jin, Xinping Yi, Liang Zhang, Lijun Zhang, Sven Schewe, and Xiaowei Huang. How does weight correlation affect the generalisation ability of deep neural networks. NeurIPS, 2020

    2020

  18. [26]

    Enhancing adversarial training with second-order statistics of weights

    Gaojie Jin, Xinping Yi, Wei Huang, Sven Schewe, and Xiaowei Huang. Enhancing adversarial training with second-order statistics of weights. In CVPR, 2022 a

    2022

  19. [27]

    Weight expansion: A new perspective on dropout and generalization

    Gaojie Jin, Xinping Yi, Pengfei Yang, Lijun Zhang, Sven Schewe, and Xiaowei Huang. Weight expansion: A new perspective on dropout and generalization. arXiv preprint arXiv:2201.09209, 2022 b

    2022 arXiv

  20. [28]

    Adversarial logit pairing

    Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing. arXiv preprint arXiv:1803.06373, 2018

    2018 arXiv

  21. [29]

    Pac-bayes bounds for the risk of the majority vote and the variance of the gibbs classifier

    Alexandre Lacasse, Fran c ois Laviolette, Mario Marchand, Pascal Germain, and Nicolas Usunier. Pac-bayes bounds for the risk of the majority vote and the variance of the gibbs classifier. Advances in Neural information processing systems, 19, 2006

    2006

  22. [30]

    (not) bounding the true error

    John Langford and Rich Caruana. (not) bounding the true error. Advances in Neural Information Processing Systems, 2: 0 809--816, 2002

    2002

  23. [31]

    Pac-bayes risk bounds for sample-compressed gibbs classifiers

    Fran c ois Laviolette and Mario Marchand. Pac-bayes risk bounds for sample-compressed gibbs classifiers. In Proceedings of the 22nd international conference on Machine learning, 2005

    2005

  24. [32]

    Adversarial vertex mixup: Toward better adversarially robust generalization

    Saehyung Lee, Hyungyu Lee, and Sungroh Yoon. Adversarial vertex mixup: Toward better adversarially robust generalization. In CVPR, 2020

    2020

  25. [33]

    Wat: improve the worst-class robustness in adversarial training

    Boqi Li and Weiwei Liu. Wat: improve the worst-class robustness in adversarial training. In Proceedings of the AAAI Conference on Artificial Intelligence, 2023

    2023

  26. [34]

    Out-of-bounding-box triggers: A stealthy approach to cheat object detectors

    Tao Lin, Lijia Yu, Gaojie Jin, Renjue Li, Peng Wu, and Lijun Zhang. Out-of-bounding-box triggers: A stealthy approach to cheat object detectors. In ECCV, 2024

    2024

  27. [35]

    Just train twice: Improving group robustness without training group information

    Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning, pp.\ 6781--6792. PMLR, 2021

    2021

  28. [36]

    Large-scale long-tailed recognition in an open world

    Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, and Stella X Yu. Large-scale long-tailed recognition in an open world. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 2537--2546, 2019

    2019

  29. [37]

    On the tradeoff between robustness and fairness

    Xinsong Ma, Zekai Wang, and Weiwei Liu. On the tradeoff between robustness and fairness. Advances in Neural Information Processing Systems, 35: 0 26230--26241, 2022

    2022

  30. [38]

    Towards deep learning models resistant to adversarial attacks

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018

    2018

  31. [39]

    Simplified pac-bayesian margin bounds

    David McAllester. Simplified pac-bayesian margin bounds. In Learning Theory and Kernel Machines: 16th Annual Conference on Learning Theory and 7th Kernel Workshop, COLT/Kernel 2003, Washington, DC, USA, August 24-27, 2003. Proceedings, pp.\ 203--215. Springer, 2003

    2003

  32. [40]

    Pac-bayesian model averaging

    David A McAllester. Pac-bayesian model averaging. In Proceedings of the twelfth annual conference on Computational learning theory, pp.\ 164--170, 1999

    1999

  33. [41]

    Long-tail learning via logit adjustment

    Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar. Long-tail learning via logit adjustment. In ICLR, 2021

    2021

  34. [42]

    Pac-bayesian generalization bound on confusion matrix for multi-class classification

    Emilie Morvant, Sokol Ko c o, and Liva Ralaivola. Pac-bayesian generalization bound on confusion matrix for multi-class classification. In ICML, 2012

    2012

  35. [43]

    Learning from failure: De-biasing classifier from biased classifier

    Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure: De-biasing classifier from biased classifier. Advances in Neural Information Processing Systems, 33: 0 20673--20684, 2020

    2020

  36. [44]

    Exploring generalization in deep learning

    Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nathan Srebro. Exploring generalization in deep learning. arXiv preprint arXiv:1706.08947, 2017 a

    2017 arXiv

  37. [45]

    A pac-bayesian approach to spectrally-normalized margin bounds for neural networks

    Behnam Neyshabur, Srinadh Bhojanapalli, and Nathan Srebro. A pac-bayesian approach to spectrally-normalized margin bounds for neural networks. arXiv preprint arXiv:1707.09564, 2017 b

    2017 arXiv

  38. [46]

    Robustness and accuracy could be reconcilable by (proper) definition

    Tianyu Pang, Min Lin, Xiao Yang, Jun Zhu, and Shuicheng Yan. Robustness and accuracy could be reconcilable by (proper) definition. In International Conference on Machine Learning, 2022

    2022

  39. [47]

    Improving robust fariness via balance adversarial training

    Chunyu Sun, Chenye Xu, Chengyuan Yao, Siyuan Liang, Yichao Wu, Ding Liang, Xianglong Liu, and Aishan Liu. Improving robust fariness via balance adversarial training. In AAAI, 2023

    2023

  40. [48]

    Intriguing properties of neural networks

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013

    2013 arXiv

  41. [49]

    Better diffusion models further improve adversarial training

    Zekai Wang, Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, and Shuicheng Yan. Better diffusion models further improve adversarial training. In ICML, 2023

    2023

  42. [50]

    Cfa: Class-wise calibrated fair adversarial training

    Zeming Wei, Yifei Wang, Yiwen Guo, and Yisen Wang. Cfa: Class-wise calibrated fair adversarial training. In CVPR, 2023

    2023

  43. [51]

    Skip connections matter: On the transferability of adversarial examples generated with resnets

    Dongxian Wu, Yisen Wang, Shu-Tao Xia, James Bailey, and Xingjun Ma. Skip connections matter: On the transferability of adversarial examples generated with resnets. arXiv preprint arXiv:2002.05990, 2020 a

    2002 arXiv

  44. [52]

    Adversarial weight perturbation helps robust generalization

    Dongxian Wu, Shu - Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. In NeurIPS, 2020 b

    2020

  45. [53]

    Pac-bayesian spectrally-normalized bounds for adversarially robust generalization

    Jiancong Xiao, Ruoyu Sun, and Zhi-Quan Luo. Pac-bayesian spectrally-normalized bounds for adversarially robust generalization. Advances in Neural Information Processing Systems, 2023

    2023

  46. [54]

    To be robust or to be fair: Towards fairness in adversarial training

    Han Xu, Xiaorui Liu, Yaxin Li, Anil Jain, and Jiliang Tang. To be robust or to be fair: Towards fairness in adversarial training. In ICML, 2021

    2021

  47. [55]

    Spectral norm regularization for improving the generalizability of deep learning

    Yuichi Yoshida and Takeru Miyato. Spectral norm regularization for improving the generalizability of deep learning. arXiv preprint arXiv:1705.10941, 2017

    2017 arXiv

  48. [56]

    Wide residual networks

    Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC, 2016

    2016

  49. [57]

    Theoretically principled trade-off between robustness and accuracy

    Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML, 2019

    2019

  50. [58]

    Trajpac: Towards robustness verification of pedestrian trajectory prediction models

    Liang Zhang, Nathaniel Xu, Pengfei Yang, Gaojie Jin, Cheng-Chao Huang, and Lijun Zhang. Trajpac: Towards robustness verification of pedestrian trajectory prediction models. In ICCV, 2023

    2023

  51. [59]

    How does mixup help with robustness and generalization? arXiv preprint arXiv:2010.04819, 2020

    Linjun Zhang, Zhun Deng, Kenji Kawaguchi, Amirata Ghorbani, and James Zou. How does mixup help with robustness and generalization? arXiv preprint arXiv:2010.04819, 2020

    2010 arXiv

  52. [60]

    Towards fairness-aware adversarial learning

    Yanghao Zhang, Tianle Zhang, Ronghui Mu, Xiaowei Huang, and Wenjie Ruan. Towards fairness-aware adversarial learning. arXiv preprint arXiv:2402.17729, 2024

    2024 arXiv

  53. [61]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  54. [62]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  55. [63]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  56. [64]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.