Pith. sign in

REVIEW 3 major objections 5 minor 69 references

Enhancing Certified Robustness via Block Reflector Orthogonal Layers and Logit Annealing Loss

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A low-rank reflector layer makes Lipschitz networks more certifiably robust, cheaply.

desk verdict BRO is a clean, efficient orthogonal layer and the empirical story is strong, but the certificates assume an unverified 1-Lipschitz bound on the regularized stem, so the SOTA claims need a patch before they fully land. read the letter →

arxiv 2505.15174 v2 pith:VLADKYKG submitted 2025-05-21 cs.LG

Bo-Han Lai , Pin-Han Huang , Bo-Han Kung , Shang-Tse Chen This is my paper
classification cs.LG MSC 68T07
keywords certifiedrobustnessLipschitzneuralnetworksorthogonalconvolutionlow-rankparameterizationblockreflectorlogitannealinglossℓ2adversarialmargindistribution
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper seeks to establish that certified robustness against $\ell_2$ adversarial perturbations can be pushed substantially higher by combining an exact, cheap orthogonal layer with a training loss that respects the limited capacity of Lipschitz networks. It introduces the Block Reflector Orthogonal (BRO) layer, whose low-rank reflector parameterization yields a 1-Lipschitz convolution without iterative approximation, and the Logit Annealing (LA) loss, which progressively down-weights examples that already have large margins. The resulting BRONet reports certified accuracy above prior Lipschitz baselines on CIFAR-10/100, Tiny-ImageNet, and ImageNet, and the gains grow when training adds diffusion-generated synthetic data.

What carries the argument

The load-bearing object is the block-reflector parameterization $W = I - 2V(V^T V)^{-1}V^T$, an orthogonal matrix defined from an unconstrained low-rank matrix $V$; it is the Householder-style reflector generalized to a block of vectors. For convolutions, the layer computes the reflector on every frequency channel of the Fourier-transformed kernel, exploiting the 2D convolution theorem so that the whole operation is an exact orthogonal circular convolution. The supporting mechanism for training is the Logit Annealing loss $L_{LA}(z,y) = -T(1-p_t)^\beta \log(p_t)$ with $p = softmax((z - \xi y)/T)$, whose annealing factor reduces the gradient for already-correct high-margin points. Together they carry the argument: BRO gives an exact and efficient Lipschitz layer, and LA allocates the network's limited margin capacity to the points that still need it.

What would settle it

Take any trained BRO layer and compute the largest singular value of $W^T W - I$ for its Fourier-domain block matrices; values far above machine precision would falsify the exact-orthogonality claim. Re-certify BRONet using an independent and stricter upper bound on the true Lipschitz constant of the stem and early regularized layers; if the certified accuracies at radii 36/255, 72/255, and 108/255 drop below the reported numbers, the headline robustness is overstated.

Watch

Extended reading notes

Core claim

The central claim is that an orthogonal convolution can be built exactly and efficiently from a low-rank matrix $V$: the block-reflector form $W = I - 2V(V^T V)^{-1}V^T$ is orthogonal, and when applied frequency-by-frequency in the Fourier domain it defines a real orthogonal circular convolution. Because the construction needs no iterative orthogonalization, BRO layers are cheaper to train and store than the leading orthogonal layers while remaining provably 1-Lipschitz. On top of this layer, the paper argues that Lipschitz networks have limited Rademacher complexity, so aggressively maximizing every margin is wasteful; the Logit Annealing loss instead anneals large-margin points and yields a broader, less skewed margin distribution. Trained with these two ingredients, BRONet reports the best certified $\ell_2$ accuracies in its comparison table: on CIFAR-10, 70.6% at radius 36/255, and on ImageNet, 19.6% at radius 108/255.

Load-bearing premise

The certificates rely on the assumption that the Lipschitz constant estimated for the non-orthogonal early layers is a true upper bound; if the estimate comes out too low, the stated certified radii shrink.

Editorial extensions

If this is right

  • BRONet-L with the LA loss reports 81.6% clean and 70.6% certified accuracy on CIFAR-10 at radius 36/255, surpassing the cited Lipschitz baselines in the same setting.
  • On ImageNet, BRONet with LA reaches 19.6% certified accuracy at radius 108/255, showing that deterministic Lipschitz certification scales to large benchmarks.
  • Because BRO avoids iterative orthogonalization, deep LipConvNet-style and BRONet-style architectures train with lower memory and per-epoch time than SOC or LOT at comparable or better certified accuracy.
  • Adding diffusion-generated synthetic data compounds the gains: on ImageNet the full BRONet+LA recipe reports 21.6% certified accuracy at radius 108/255, up from 18.9% for the prior recipe.
  • The LA loss shifts the certified-radius distribution toward larger median, lower variance, and less positive skew, indicating that robustness improves for the typical point rather than a few outliers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Since the block-reflector identity is purely algebraic, the same low-rank orthogonal construction could be dropped into other norm-preserving modules, including dense layers and residual connections, wherever an exact 1-Lipschitz map is needed at reduced cost.
  • The annealing mechanism is a generic answer to capacity-limited margin learning, so the LA loss should transfer beyond BRO to any Lipschitz or margin-based training pipeline; a direct test would be running CE, CR, and LA under identical compute on a fixed architecture.
  • If the BRO certificates survive an independent Lipschitz audit, exact orthogonal parameterization removes a common source of looseness in Lipschitz certificates, potentially making deterministic certification a more practical alternative to randomized smoothing at scale.
  • The paper's margin-distribution analysis implies a concrete, testable prediction: for any fixed architecture and budget, a loss that anneals high-margin points should lower the variance of certified radii; measuring that on Tiny-ImageNet or ImageNet would test the capacity story beyond CIFAR.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes two ingredients for certified ℓ2 robustness in Lipschitz networks: (i) the Block Reflector Orthogonal (BRO) layer, a low-rank orthogonal parameterization based on the block-reflector formula W = I − 2V(VᵀV)⁻¹Vᵀ, implemented for convolutions through the Fourier domain; and (ii) the Logit Annealing (LA) loss, a softmax-based loss with a focal-style annealing term motivated by a Rademacher-complexity argument about the limited capacity of Lipschitz models. The authors combine BRO backbones with a last-layer-normalized head into BRONet and report state-of-the-art certified accuracy on CIFAR-10/100, Tiny-ImageNet, and ImageNet, both with and without diffusion-generated training data.

Significance. If the certificates are valid, the paper is a useful contribution: BRO is simple, exact, and avoids the iterative orthogonalization used by SOC and LOT; the paper includes careful efficiency comparisons and backbone ablations on standard benchmarks, and the implementation is publicly available. The LA loss gives consistent improvements at moderate radii across several datasets and appears to be a practical training recipe. A clear strength is that the core BRO orthogonality argument is algebraic and self-contained, and the ablation structure cleanly separates the effects of the BRO layer, the LA loss, and the architectural change. However, the validity of every reported certified number depends on strict 1-Lipschitzness of all layers, including layers that are only Lipschitz-regularized during training, and the formal motivation for the LA loss is quantitatively vacuous as stated. Both issues are load-bearing for the headline claims and need to be addressed before the empirical results can be taken at face value.

major comments (3)
  1. [§2.1 / App. B.2 / App. B.5] The certified-accuracy tables assume the entire network is 1-Lipschitz, but the stem layer (and six layers on ImageNet) are described as unconstrained and only Lipschitz-regularized during training. EMMA, as used in Appendix B.5, is a loss-level margin-maximizing regularizer and does not by itself provide a certified upper bound on the stem's spectral norm; no post-training upper bound on Lip(stem) is reported. Since the radius formula is ε = margin/(√2 L) (or the analogous LLN bound), any true stem Lipschitz constant above 1 makes every reported certified accuracy in Tables 1, 2, and 9 an overestimate. Please supply a verified upper bound for the regularized layers and incorporate it into the certification, or replace those layers with exactly 1-Lipschitz constructions and retrain.
  2. [§5 / Eq. (7)] Proposition 3 applies the Ledoux-Talagrand contraction to the ramp loss alone, but the ramp loss is composed with the margin operator M(f(x), y) = f_y(x) − max_{k≠y} f_k(x), which is 2-Lipschitz as a function of f(x). The correct Lipschitz constant for the composed function is 2/τ, not 1/τ, so Eq. (7) is missing a factor of 2. Even with this correction, the bound is (2L/(τn)) Σᵢ ||xᵢ||, which is of order 10² for normalized CIFAR/ImageNet inputs and makes the lower bound in Eq. (6) vacuous. Thus the paper's formal argument that Lipschitz models cannot enlarge margins indefinitely is not quantitatively established; the LA loss should be presented as an empirical contribution unless a sharper complexity analysis is supplied.
  3. [Appendix A.2 / Prop. 2] The proof of Lemma 3 switches between Vᵀ and V* in the same derivation, and the claim that BRO(C) remains real because C is real is asserted without the necessary Hermitian-symmetry argument: for a real convolution, the Fourier-domain multiplier must satisfy W̃_p = conjugate(W̃_−p), and this must be proved for the BRO construction. Please rewrite the lemma using conjugate transposes throughout and add the explicit conjugation-equivariance argument that guarantees the output is real. As written, the proof of the central orthogonality-and-reality claim is incomplete, although the underlying identity appears correct.
minor comments (5)
  1. [Algorithm 1] Line 11 returns Y[:, k:-k, k:-k], but the padding size is defined as k′ = ⌊k/2⌋, so the crop should be Y[:, k′:-k′]; the output spatial size w is also never defined in the pseudocode.
  2. [Appendix D.5] The hyperparameter grid is written as T ∈ {0.25, 0.5, 0, 75, 1.0}, which should be {0.25, 0.5, 0.75, 1.0}; in addition, the statement that the LipConvNet offset is ξ = 2√2 "due to an oversight in the implementation" needs clarification about which reported experiments use which offset.
  3. [Appendix B.5] The main results are reported as averages of three runs, but no standard deviations or error bars are given; since the headline gains are modest (e.g., 70.6 vs. 69.8 on CIFAR-10 at ε = 36/255), reporting variance would materially strengthen the comparison.
  4. [Appendix D.6] Table 15's header says "Time (seconds/epoch)" while the text says time is calculated in minutes per training epoch; please make the units consistent.
  5. [Tables 1 and 2] The LA loss sometimes degrades certified accuracy at the largest radius (e.g., CIFAR-10 at 108/255 in Table 1, and CIFAR-100 at 108/255 in Table 2 relative to LiResNet+LA); the abstract and introduction should scope the claim to moderate radii or acknowledge this nonuniformity more prominently.

Circularity Check

0 steps flagged · score 1.0 of 10

No meaningful circularity: the BRO orthogonality proof is self-contained, the LA loss is heuristically motivated rather than derived from the reported accuracies, and the only self-citation is non-load-bearing; the main caveat is an EMMA-regularized stem, which is a soundness gap rather than circularity.

full rationale

The central derivation is the BRO parameterization W = I - 2V(V^T V)^-1 V^T, whose orthogonality and convolution extension are proved in Appendices A.1 and A.2 using only linear algebra and the Fourier convolution theorem; these proofs do not import the benchmark results, so the orthogonality/efficiency claim is self-contained. The Logit Annealing loss is motivated by Theorem 1 and Proposition 3, but the loss itself is explicitly assembled from the Focal Loss annealing factor (1-p_t)^beta and the temperature/offset of Prach & Lampert (2022), so the Rademacher bound is a post-hoc motivation rather than a derivation of the loss; no equation equates the loss to the later certified-accuracy numbers. LA hyperparameters were tuned on LipConvNet/CIFAR-100 (Appendices B.4 and D.5), and Appendix E explicitly acknowledges this limitation ('the parameters were chosen based on LipConvNets trained on CIFAR-100 without diffusion-synthetic augmentation... may not fully align with different models and datasets'); this is standard hyperparameter selection, not a fitted input renamed as a prediction, and the headline gains on CIFAR-10, Tiny-ImageNet, and ImageNet are not directly tuned on those test sets. The only self-citation in the reference list (Das et al., 2018, which includes author S.-T. Chen) appears in a list of preprocessing defenses and is not load-bearing. The most serious concern is not circularity: Appendices B.2 and B.5 state that the stem is 'unconstrained ... Lipschitz-regularized during training' with an EMMA target budget rather than an exactly orthogonal layer, and no certified upper bound on the stem's true Lipschitz constant is supplied; if the EMMA estimate is too low, the reported radii would overstate true robustness. That is a soundness/correctness risk, not an equivalence between the paper's inputs and outputs, so it does not raise the circularity score beyond the minor-caveat range.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central claims rest on standard linear algebra (block reflectors, FFT convolution theorem), Rademacher complexity bounds, and the inherited validity of EMMA-based Lipschitz regularization. The LA loss and BRO rank ratios introduce tuned hyperparameters. No new physical or mathematical entities are postulated.

free parameters (5)
  • LA loss temperature T = 0.75
    Selected by ablation on LipConvNet-10-32 on CIFAR-100 (Appendix D.5) and then applied to all datasets and architectures.
  • LA loss offset xi = 2.0 (2*sqrt(2) for LipConvNet)
    Selected by ablation on CIFAR-100 LipConvNet; the sqrt(2) variant is noted as an implementation oversight.
  • LA loss annealing exponent beta = 5.0
    Selected from {1,3,5,7} on CIFAR-100 LipConvNet; controls the focal-loss-style down-weighting of easy examples.
  • BRO rank ratio n/m = m/4 (BRONet-M CIFAR), m/8 (Tiny-ImageNet), m/2 (BRONet-L, ImageNet)
    Rank of the unconstrained matrix V chosen via ablation (Appendix D.3); controls expressiveness and computational cost.
  • EMMA target budget epsilon = 108/255 (CIFAR-10/100), 72/255 (ImageNet)
    Lipschitz regularization target for the stem, chosen following prior work settings (Hu et al., 2024).
assumptions (6)
  • standard math 2D convolution theorem and block-diagonalization of circular convolution via FFT
    Used in the proof of Proposition 2 (Appendix A.2), building on Trockman and Kolter (2021).
  • standard math Block reflector property: W = I - 2V(V^T V)^(-1)V^T is orthogonal for full-rank V
    Proved in Appendix A.1; the core of the BRO layer.
  • standard math Ledoux-Talagrand contraction and Rademacher margin bounds
    Used in Theorem 1 and Proposition 3 to motivate the LA loss (Section 5 and Appendix C).
  • domain assumption The Lipschitz-regularized stem layers (EMMA) yield a valid certified Lipschitz upper bound
    Sections B.2 and B.5 use Lipschitz-regularized layers in the stem; the paper does not prove that the estimator gives a certified upper bound.
  • standard math MaxMin activation is 1-Lipschitz and gradient-norm preserving
    Used in all architectures (Sections 2.2 and B.2).
  • domain assumption Zero-padding followed by cropping of the BRO convolution output remains 1-Lipschitz
    Appendix A.4 argues the norm drop does not expand the norm; the layer is then not exactly orthogonal but remains contractive.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Certified Robustness via Block Reflector Orthogonal Layers and Logit Annealing Loss." pith.science (2026). https://pith.science/paper/VLADKYKG

@misc{pith2026250515174,
  author       = {Pith},
  title        = {Pith review of: Enhancing Certified Robustness via Block Reflector Orthogonal Layers and Logit Annealing Loss},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VLADKYKG}},
  note         = {Machine review of arXiv:2505.15174}
}
read the original abstract

Lipschitz neural networks are well-known for providing certified robustness in deep learning. In this paper, we present a novel, efficient Block Reflector Orthogonal (BRO) layer that enhances the capability of orthogonal layers on constructing more expressive Lipschitz neural architectures. In addition, by theoretically analyzing the nature of Lipschitz neural networks, we introduce a new loss function that employs an annealing mechanism to increase margin for most data points. This enables Lipschitz models to provide better certified robustness. By employing our BRO layer and loss function, we design BRONet - a simple yet effective Lipschitz neural network that achieves state-of-the-art certified robustness. Extensive experiments and empirical analysis on CIFAR-10/100, Tiny-ImageNet, and ImageNet validate that our method outperforms existing baselines. The implementation is available at https://github.com/ntuaislab/BRONet.

Figures

Figures reproduced from arXiv: 2505.15174 by the authors.

Figure 1
Figure 1. Visualization of model performance on CIFAR-10. The circle size denotes model size. in practice, these approaches cannot provide robustness guarantees and may fail against more sophisticated attacks. Certified defenses, unlike empirical ones, provide provable robustness by ensuring no adversarial examples exist within an ℓp-norm ball of radius ε centered on the prediction point. Certified defenses against adversaria… view at source ↗
Figure 2
Figure 2. Comparison of runtime and memory usage among SOC, LOT, and the proposed BRO. with an observation of a minor norm drop resulting from the removal of output padding, is provided in Appendix A.4. For layers where the input dimension differs from the output dimension, we enforce the 1-Lipschitz constraint via semi￾orthogonal matrices. For details about the semi-orthogonal layers, please refer to Appendix A.3. 4.2. Prope… view at source ↗
Figure 3
Figure 3. Certified accuracy with respect to radius. The LA loss helps learn appropriate margin. To demonstrate that the LA loss enables learning an appro￾priate margin for most data points, we further investigate the certified radius distribution. Following Cohen et al. (2019), we plot the certified accuracy with respect to the radius on CIFAR-100 to visualize the margin distribution in [PITH_FULL_IMAGE:figures/full_fig_p00… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Visualization of BRO convolution for different cin and cout. A.4. The Implementation and Effect of Zero-padding Following Xu et al. (2022), we apply zero-padding on images X ∈ R c×s×s , creating Xpad ∈ R c×(s+2k ′ )×(s+2k ′ ) , before performing the 2D FFT. After apply…
Figure 5
Figure 5. Figure 5: Demonstration of the runtime and memory consumption under different settings with LipConvNet architecture. The notation s denotes the input size, init denote the initial channel of the the entire model, and k denotes the kernel size. The batch sizes are fixed at 512 fo…
Figure 6
Figure 6. Figure 6: Demonstration of the runtime and memory consumption under different settings with LipConvNet architecture. For each line, we keep the input size s fixed while varying the initial channel. The batch sizes are fixed at 512 for all plots, and each value is the average ove…
Figure 7
Figure 7. Figure 7: The proposed Block Reflector Orthogonal (BRO) convolution kernel, which is an orthogonal matrix, employs Fourier transfor￾mation to simulate the convolution operation. This convolution is inherently orthogonal and thus 1-Lipschitz, providing guarantees for adversarial …
Figure 8
Figure 8. Figure 8: Following Trockman & Kolter (2021); Singla & Feizi (2021b); Xu et al. (2022), we use the proposed orthogonal convolution layer to construct the LipConvnet. This figure illustrates the LipConvnet-5, which cascades five BRO convolution layers. The activation function use…
Figure 9
Figure 9. Figure 9: Following the LiResNet architecture (Leino et al., 2021; Hu et al., 2023), we utilized the BRO layer to construct BRONet. The parameters L, W, and D can be adjusted to control the model size. differs from the orthogonal convolution described in BRO convolutional layer,…
Figure 10
Figure 10. Figure 10: Comparison of three loss functions. The x-axis is pt. This figure displays curves representing the behavior of the proposed LA loss, contrasted with cross-entropy loss and the Certificate Regularization (CR) term. We observe the discontinuous gradient of the CR term. …
Figure 11
Figure 11. Figure 11: CR Loss Landscape Analysis. This figure illustrates the loss landscape to investigate the effects of the CR term. Notably, the CR term can suddenly become “activated” or “deactivated,” which is vividly depicted in the landscape transitions. These abrupt changes contri…
Figure 12
Figure 12. Figure 12: Histogram of margin distribution. The left histogram represents margin distribution obtained from the training set, while the right histogram shows margin distribution from the test set. The x-axis represents the margin values. These visualizations demonstrate that th…
Figure 13
Figure 13. Figure 13: Plots of condition number of parameterized matrix in Fourier domain. The left plot shows the condition number with randomly initialized parameters, whereas the right plot shows the condition number with trained parameters. D.6. LipConvNet Ablation Experiments More det…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

69 extracted references · 58 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    M., Malgouyres, F., and Mamalet, F

    Achour, E. M., Malgouyres, F., and Mamalet, F. Existence, stability and scalability of orthogonal convolutional neural networks. Journal of Machine Learning Research (JMLR), 2022

    work page 2022

  3. [3]

    Sorting out L ipschitz function approximation

    Anil, C., Lucas, J., and Grosse, R. Sorting out L ipschitz function approximation. In International Conference on Machine Learning (ICML), 2019

    work page 2019

  4. [4]

    J., Delattre, B., Allauzen, A., and Hu, B

    Araujo, A., Havens, A. J., Delattre, B., Allauzen, A., and Hu, B. A unified algebraic perspective on Lipschitz neural networks. In International Conference on Learning Representations (ICLR), 2023

    work page 2023

  5. [5]

    L., Foster, D

    Bartlett, P. L., Foster, D. J., and Telgarsky, M. J. Spectrally-normalized margin bounds for neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2017

    work page 2017

  6. [6]

    Pay attention to your loss: understanding misconceptions about Lipschitz neural networks

    B \'e thune, L., Boissin, T., Serrurier, M., Mamalet, F., Friedrich, C., and Gonzalez Sanz, A. Pay attention to your loss: understanding misconceptions about Lipschitz neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2022

    work page 2022

  7. [7]

    D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. In Advances in Neural Information Processing Systems (NeurIPS), 2020

    work page 2020

  8. [8]

    and Wagner, D

    Carlini, N. and Wagner, D. Audio adversarial examples: Targeted attacks on speech-to-text. In IEEE security and privacy workshops (SPW), 2018

    work page 2018

Show all 69 references
  1. [9]

    Robust overfitting may be mitigated by properly learned smoothening

    Chen, T., Zhang, Z., Liu, S., Chang, S., and Wang, Z. Robust overfitting may be mitigated by properly learned smoothening. In International Conference on Learning Representations (ICLR), 2021

    2021

  2. [10]

    and Nowicki, D

    Chernodub, A. and Nowicki, D. Norm-preserving orthogonal permutation linear unit activation functions (OPLU) . arXiv preprint arXiv:1604.02313, 2016

    2016 arXiv

  3. [11]

    Certified adversarial robustness via randomized smoothing

    Cohen, J., Rosenfeld, E., and Kolter, Z. Certified adversarial robustness via randomized smoothing. In International Conference on Machine Learning (ICML), 2019

    2019

  4. [12]

    and Hein, M

    Croce, F. and Hein, M. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International Conference on Machine Learning (ICML), 2020

    2020

  5. [13]

    E., and Chau, D

    Das, N., Shanbhogue, M., Chen, S.-T., Hohman, F., Li, S., Chen, L., Kounavis, M. E., and Chau, D. H. Shield: Fast, practical defense and vaccination for deep learning using jpeg compression. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery ...

    2018

  6. [14]

    A new formulation of the hypermatrix householder- QR decomposition

    Dietrich, G. A new formulation of the hypermatrix householder- QR decomposition. Computer Methods in Applied Mechanics and Engineering, 9 0 (3): 0 273--280, 1976

    1976

  7. [15]

    Incorporating Nesterov momentum into Adam

    Dozat, T. Incorporating Nesterov momentum into Adam . International Conference on Learning Representations workshop (ICLR workshop), 2016

    2016

  8. [16]

    Formal verification of piece-wise linear feed-forward neural networks

    Ehlers, R. Formal verification of piece-wise linear feed-forward neural networks. In International Symposium on Automated Technology for Verification and Analysis (ATVA), 2017

    2017

  9. [17]

    Evaluating and understanding the robustness of adversarial logit pairing

    Engstrom, L., Ilyas, A., and Athalye, A. Evaluating and understanding the robustness of adversarial logit pairing. arXiv preprint arXiv:1807.10272, 2018

    2018 arXiv

  10. [18]

    Generalizable adversarial training via spectral normalization

    Farnia, F., Zhang, J., and Tse, D. Generalizable adversarial training via spectral normalization. In International Conference on Learning Representations (ICLR), 2019

    2019

  11. [19]

    Efficient and accurate estimation of Lipschitz constants for deep neural networks

    Fazlyab, M., Robey, A., Hassani, H., Morari, M., and Pappas, G. Efficient and accurate estimation of Lipschitz constants for deep neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2019

    2019

  12. [20]

    J., Shlens, J., and Szegedy, C

    Goodfellow, I. J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014

    2014 arXiv

  13. [21]

    On the effectiveness of interval bound propagation for training verifiably robust models

    Gowal, S., Dvijotham, K., Stanforth, R., Bunel, R., Qin, C., Uesato, J., Arandjelovic, R., Mann, T., and Kohli, P. On the effectiveness of interval bound propagation for training verifiably robust models. arXiv preprint arXiv:1810.12715, 2018

    arXiv

  14. [22]

    Delving deep into rectifiers: Surpassing human-level performance on imagenet classification

    He, K., Zhang, X., Ren, S., and Sun, J. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2015

    2015

  15. [23]

    J., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al

    Hu, E. J., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR), 2022

    2022

  16. [24]

    Unlocking deterministic robustness certification on Imagenet

    Hu, K., Zou, A., Wang, Z., Leino, K., and Fredrikson, M. Unlocking deterministic robustness certification on Imagenet . In Advances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  17. [25]

    A recipe for improved certifiable robustness

    Hu, K., Leino, K., Wang, Z., and Fredrikson, M. A recipe for improved certifiable robustness. In The Twelfth International Conference on Learning Representations (ICLR), 2024

    2024

  18. [26]

    Jain, A. K. Fundamentals of digital image processing. Prentice-Hall, Inc., 1989

    1989

  19. [27]

    Elucidating the design space of diffusion-based generative models

    Karras, T., Aittala, M., Aila, T., and Laine, S. Elucidating the design space of diffusion-based generative models. In Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  20. [28]

    Guiding a diffusion model with a bad version of itself

    Karras, T., Aittala, M., Kynk\"a\"anniemi, T., Lehtinen, J., Aila, T., and Laine, S. Guiding a diffusion model with a bad version of itself. In Advances in Neural Information Processing Systems (NeurIPS), 2024 a

    2024

  21. [29]

    Analyzing and improving the training dynamics of diffusion models

    Karras, T., Aittala, M., Lehtinen, J., Hellsten, J., Aila, T., and Laine, S. Analyzing and improving the training dynamics of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024 b

    2024

  22. [30]

    Certified robustness to adversarial examples with differential privacy

    Lecuyer, M., Atlidakis, V., Geambasu, R., Hsu, D., and Jana, S. Certified robustness to adversarial examples with differential privacy. In 2019 IEEE Symposium on Security and Privacy (SP), 2019

    2019

  23. [31]

    and Talagrand, M

    Ledoux, M. and Talagrand, M. Probability in Banach Spaces: isoperimetry and processes. Springer Science & Business Media, 2013

    2013

  24. [32]

    and Kim, D

    Lee, M. and Kim, D. Robust evaluation of diffusion-based adversarial purification. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023

    2023

  25. [33]

    Globally-robust neural networks

    Leino, K., Wang, Z., and Fredrikson, M. Globally-robust neural networks. In International Conference on Machine Learning (ICML), 2021

    2021

  26. [34]

    Sok: Certified robustness for deep neural networks

    Li, L., Xie, T., and Li, B. Sok: Certified robustness for deep neural networks. In 2023 IEEE symposium on security and privacy (SP), 2023

    2023

  27. [35]

    B., and Jacobsen, J.-H

    Li, Q., Haque, S., Anil, C., Lucas, J., Grosse, R. B., and Jacobsen, J.-H. Preventing gradient attenuation in Lipschitz constrained convolutional networks. In Advances in Neural Information Processing Systems (NeurIPS), 2019

    2019

  28. [36]

    Focal loss for dense object detection

    Lin, T.-Y., Goyal, P., Girshick, R., He, K., and Dollar, P. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017

    2017

  29. [37]

    Towards deep learning models resistant to adversarial attacks

    Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR), 2018

    2018

  30. [38]

    Connecting certified and adversarial training

    Mao, Y., M\" u ller, M., Fischer, M., and Vechev, M. Connecting certified and adversarial training. In Advances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  31. [39]

    J., Araujo, A., and Allauzen, A

    Meunier, L., Delattre, B. J., Araujo, A., and Allauzen, A. A dynamical system perspective for Lipschitz neural networks. In International Conference on Machine Learning (ICML), 2022

    2022

  32. [40]

    Foundations of machine learning

    Mohri, M., Rostamizadeh, A., and Talwalkar, A. Foundations of machine learning. MIT press, 2018

    2018

  33. [41]

    N., Eckert, F., Fischer, M., and Vechev, M

    Mueller, M. N., Eckert, F., Fischer, M., and Vechev, M. Certified training: Small boxes are all you need. In The Eleventh International Conference on Learning Representations (ICLR), 2023

    2023

  34. [42]

    Orthogonal wasserstein gans

    M \"u ller, J., Klein, R., and Weinmann, M. Orthogonal wasserstein gans. arXiv preprint arXiv:1911.13060, 2019

    1911 arXiv

  35. [43]

    Pytorch: An imperative style, high-performance deep learning library

    Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems (NeurIPS), 2019

    2019

  36. [44]

    and Lampert, C

    Prach, B. and Lampert, C. H. Almost-orthogonal layers for efficient general-purpose Lipschitz networks. In European Conference on Computer Vision (ECCV), 2022

    2022

  37. [45]

    Prach, B., Brau, F., Buttazzo, G., and Lampert, C. H. 1-Lipschitz layers compared: Memory, speed, and certifiable robustness. arXiv preprint arXiv:2311.16833, 2023

    2023 arXiv

  38. [46]

    Deep isometric learning for visual recognition

    Qi, H., You, C., Wang, X., Ma, Y., and Malik, J. Deep isometric learning for visual recognition. In International Conference on Machine Learning (ICML), 2020

    2020

  39. [47]

    Defense-gan: Protecting classifiers against adversarial attacks using generative models

    Samangouei, P., Kabkab, M., and Chellappa, R. Defense-gan: Protecting classifiers against adversarial attacks using generative models. In International Conference on Learning Representations (ICLR), 2018

    2018

  40. [48]

    Sch \"o nemann, P. H. A generalized solution of the orthogonal procrustes problem. Psychometrika, 31 0 (1): 0 1--10, 1966

    1966

  41. [49]

    and Parlett, B

    Schreiber, R. and Parlett, B. Block reflectors: Theory and computation. SIAM Journal on Numerical Analysis, 25 0 (1): 0 189--205, 1988

    1988

  42. [50]

    A., Xu, Z., Dickerson, J., Studer, C., Davis, L

    Shafahi, A., Najibi, M., Ghiasi, M. A., Xu, Z., Dickerson, J., Studer, C., Davis, L. S., Taylor, G., and Goldstein, T. Adversarial training for free! In Advances in Neural Information Processing Systems (NeurIPS), 2019

    2019

  43. [51]

    Z., and Hsieh, C.-J

    Shi, Z., Wang, Y., Zhang, H., Kolter, J. Z., and Hsieh, C.-J. Efficiently computing local Lipschitz constants of neural networks via bound propagation. In Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  44. [52]

    and Feizi, S

    Singla, S. and Feizi, S. Fantastic four: Differentiable bounds on singular values of convolution layers. In International Conference on Learning Representations (ICLR), 2021 a

    2021

  45. [53]

    and Feizi, S

    Singla, S. and Feizi, S. Skew orthogonal convolutions. In International Conference on Machine Learning (ICML), 2021 b

    2021

  46. [54]

    and Feizi, S

    Singla, S. and Feizi, S. Improved techniques for deterministic l2 robustness. In Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  47. [55]

    Improved deterministic l2 robustness on CIFAR-10 and CIFAR-100

    Singla, S., Singla, S., and Feizi, S. Improved deterministic l2 robustness on CIFAR-10 and CIFAR-100 . In International Conference on Learning Representations (ICLR), 2022

    2022

  48. [56]

    Intriguing properties of neural networks

    Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. Intriguing properties of neural networks. In International Conference on Learning Representations (ICLR), 2014

    2014

  49. [57]

    and Kolter, J

    Trockman, A. and Kolter, J. Z. Orthogonalizing convolutional layers with the Cayley transform. In International Conference on Learning Representations (ICLR), 2021

    2021

  50. [58]

    Lipschitz -margin training: Scalable certification of perturbation invariance for deep neural networks

    Tsuzuku, Y., Sato, I., and Sugiyama, M. Lipschitz -margin training: Scalable certification of perturbation invariance for deep neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2018

    2018

  51. [59]

    and Scaman, K

    Virmaux, A. and Scaman, K. Lipschitz regularity of deep neural networks: analysis and efficient estimation. In Advances in Neural Information Processing Systems (NeurIPS), 2018

    2018

  52. [60]

    Wang, C.-Y., Bochkovskiy, A., and Liao, H.-Y. M. Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv preprint arXiv:2207.02696, 2022

    2022 arXiv

  53. [61]

    and Manchester, I

    Wang, R. and Manchester, I. Direct parameterization of Lipschitz -bounded deep networks. In International Conference on Machine Learning (ICML), 2023

    2023

  54. [62]

    Wang, S., Zhang, H., Xu, K., Lin, X., Jana, S., Hsieh, C.-J., and Kolter, J. Z. Beta-crown: Efficient bound propagation with per-neuron split constraints for neural network robustness verification. In Advances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  55. [63]

    Better diffusion models further improve adversarial training

    Wang, Z., Pang, T., Du, C., Lin, M., Liu, W., and Yan, S. Better diffusion models further improve adversarial training. In International Conference on Machine Learning (ICML), 2023

    2023

  56. [64]

    Dynamical isometry and a mean field theory of cnns: How to train 10,000-layer vanilla convolutional neural networks

    Xiao, L., Bahri, Y., Sohl-Dickstein, J., Schoenholz, S., and Pennington, J. Dynamical isometry and a mean field theory of cnns: How to train 10,000-layer vanilla convolutional neural networks. In International Conference on Machine Learning (ICML), 2018

    2018

  57. [65]

    LOT : Layer-wise orthogonal training on improving l2 certified robustness

    Xu, X., Li, L., and Li, B. LOT : Layer-wise orthogonal training on improving l2 certified robustness. In Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  58. [66]

    E., Salman, H., Razenshteyn, I., and Li, J

    Yang, G., Duan, T., Hu, J. E., Salman, H., Razenshteyn, I., and Li, J. Randomized smoothing of all shapes and sizes. In International Conference on Machine Learning (ICML), 2020

    2020

  59. [67]

    Constructing orthogonal convolutions in an explicit manner

    Yu, T., Li, J., Cai, Y., and Li, P. Constructing orthogonal convolutions in an explicit manner. In International Conference on Learning Representations (ICLR), 2022

    2022

  60. [68]

    Zhang, H., Wang, S., Xu, K., Li, L., Li, B., Jana, S., Hsieh, C.-J., and Kolter, J. Z. General cutting planes for bound-propagation-based neural network verification. In Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  61. [69]

    Zhang, M., Lucas, J., Ba, J., and Hinton, G. E. Lookahead optimizer: k steps forward, 1 step back. In Advances in Neural Information Processing Systems (NeurIPS), 2019

    2019

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.