Pith. sign in

REVIEW 3 major objections 4 minor 91 references

Towards Understanding The Calibration Benefits of Sharpness-Aware Minimization

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Sharpness-aware minimization keeps neural networks well-calibrated because its perturbed loss implicitly maximizes the entropy of the predictive distribution, and a simple variant, CSAM, pushes the effect further.

desk verdict The empirical case that SAM improves calibration is solid, but the central theorem is unsupported: Lemma 1's proof drops the gradient norm and the stated bound fails. read the letter →

arxiv 2505.23866 v2 pith:3WH35FDE submitted 2025-05-29 cs.LG cs.AI

Chengli Tan , Yubo Zhou , Haishan Ye , Guang Dai , Junmin Liu , Zengjie Song , Jiangshe Zhang , Zixiang Zhao
show 2 more authors
Yunda Hao Yong Xu
This is my paper
classification cs.LGcs.AI
keywords sharpness-awareminimizationmodelcalibrationexpectederrorentropyregularizationoverconfidencedistributionshiftCSAMimplicit
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that sharpness-aware minimization (SAM) does not just find flat minima; it also keeps neural networks honest about their uncertainty. Minimizing the SAM perturbed loss, the authors show, is equivalent in effect to adding an entropy-regularization term that penalizes overconfident predictions. The formal result is an inequality bounding the SAM loss from below by the original cross-entropy loss minus a scaled binary entropy of the true-class probability, with a coefficient larger than one, so overconfidence is pushed back. The paper's experiments report that SAM-trained models have substantially lower expected calibration error (ECE), a measure of how far average confidence strays from accuracy, than SGD-trained ones, that the benefit persists under distribution shift, and that a proposed variant, CSAM, lowers calibration error further without hurting accuracy.

What carries the argument

The load-bearing object is the SAM perturbed weight $\tilde\theta = \theta + \rho \nabla \ell_\theta(z)/\|\nabla \ell_\theta(z)\|_2$, combined with a second-order Taylor expansion of the cross-entropy loss. The argument assumes the smallest Hessian eigenvalue stays above $-1/\rho$, so the quadratic term in the expansion cannot cancel the linear term, and the perturbed true-class probability satisfies $\tilde p_y \le e^{-\rho/2}p_y$. That exponential decay is what makes the coefficient $\lambda=(1-\tilde p_y)/(1-p_y)$ exceed one, turning the SAM objective into a lower bound that includes an entropy bonus. In CSAM, the per-example loss in the outer loop is redefined so that predictions with $\tilde p_y>1/2$ are explicitly de-emphasized, strengthening the entropy penalty.

What would settle it

Pick any training trajectory and any example, and record the true-class probability before and after a SAM perturbation together with the gradient norm and the smallest Hessian eigenvalue. A single step where the curvature condition holds but the gradient norm is below 1 and $\tilde p_y > e^{-\rho/2}p_y$ would show that the stated assumptions do not support Lemma 1.

Watch

Extended reading notes

Core claim

The central discovery is that SAM implicitly maximizes the entropy of the predictive distribution. For a single example with cross-entropy loss, Theorem 1 establishes that $$\tilde{\ell}_\$\theta$(z) \ge \ell_\$\theta$(z) - \$\lambda$ H(p_y) + H(\tilde p_y),$$ where $H$ is the binary entropy, $p_y$ is the probability assigned to the true label, $\tilde p_y$ is that probability after the SAM perturbation, and $\lambda = (1-\tilde p_y)/(1-p_y)>1$. Because $\lambda>1$, minimizing the SAM loss puts more weight on raising $H(p_y)$ than on raising $H(\tilde p_y)$, which forces the predictive distribution away from the extremes of overconfidence. The authors extend the same inequality to mini-batch SAM using geometric means of the per-example probabilities, and they use the mechanism to design CSAM, a variant that down-weights overconfident examples during the outer SAM step.

Load-bearing premise

The proof's chain of inequalities requires the gradient norm to be at least 1 at each sampled example, but the paper only states and checks a curvature condition; if the gradient is shorter than that, the claimed exponential decay of the true-class probability, and with it the entropy-regularization bound, does not follow from the stated assumptions.

Editorial extensions

If this is right

  • SAM-trained models need less post-hoc calibration: in the paper's CIFAR experiments, uncalibrated SAM's ECE is generally below the temperature-scaled or isotonic-regression-calibrated SGD's ECE.
  • The calibration benefit carries over to out-of-distribution data: on ImageNet-C and CIFAR-10/100-C, SAM keeps lower ECE than SGD, and ensembling amplifies the advantage.
  • The entropy mechanism strengthens at the end of training: the penalty coefficient grows with $\tilde p_y$, so SAM's calibration effect is strongest exactly when networks tend to become overconfident.
  • Switching from SGD to SAM only for the late stage of training is enough to obtain most of the calibration benefit, which suggests cheaper training schedules.
  • CSAM consistently yields the lowest calibration error among the compared calibration methods while matching or beating SAM's accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the entropy-regularization mechanism is causal, the calibration benefit should be predictable from a model's confidence distribution: architectures and checkpoints that are more overconfident should show larger ECE drops under SAM, matching the paper's observation about capacity.
  • The same lower-bound argument could extend to any strictly proper scoring loss whose Taylor expansion shares the curvature condition, which would let one test whether SAM's calibration effect is specific to cross-entropy or generic.
  • CSAM's down-weighting of overconfident examples suggests a family of schedules in which the suppression strength $\gamma$ is annealed or made adaptive per example; the paper only tests fixed $\gamma$.
  • A concrete testable extension is to log gradient norms and Hessian spectra during SAM training; if the inequality $\tilde p_y \le e^{-\rho/2}p_y$ holds even where the stated assumptions are loose, the entropy mechanism is more robust than the proof suggests.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper argues that sharpness-aware minimization (SAM) improves the calibration of deep neural networks by implicitly regularizing the negative entropy of the predictive distribution, and it introduces CSAM, a variant that further reduces expected calibration error. The theoretical part (Section 4.1 and Appendix A) claims that, under a Hessian eigenvalue lower bound, the SAM-perturbed loss is bounded below by the original loss plus an entropy-regularization term, with a coefficient λ > 1. The empirical part reports lower ECE for SAM relative to SGD across CIFAR-10/100, ImageNet-1K, and distribution-shift benchmarks, and shows that CSAM achieves lower ECE than SAM and a range of calibration baselines.

Significance. If the theoretical claim were correct, it would provide a new explanation of SAM's empirical calibration benefits and justify a simple algorithmic modification. The experimental study is broad and the reported ECE improvements are consistent across several architectures, datasets, and distribution shifts, which is a genuine strength. The paper also engages with a substantial set of calibration baselines. However, the central theoretical result is not sound as written, and the experiments cannot compensate for the failure of the proof that motivates both the theoretical contribution and the proposed CSAM variant.

major comments (3)
  1. [Appendix A, Proof of Lemma 1] The first-order Taylor term is written as ρ, but the exact SAM perturbation gives (θ̃ - θ)ᵀ∇ℓ_θ(z) = ρ‖∇ℓ_θ(z)‖. The chain of inequalities therefore requires ‖∇ℓ_θ(z)‖ ≥ 1, which is neither stated nor proved and is false for cross-entropy losses near confidently correct predictions. A concrete counterexample is the scalar logistic loss ℓ(w) = -log σ(w) with w = log 99, ρ = 0.1: the Hessian is p(1-p) = 0.0099 > -1/ρ, so the assumption holds, but the SAM ascent step gives p̃ = σ(w-ρ) ≈ 0.989, which violates p̃ ≤ e^{-ρ/2}p ≈ 0.942. Since Lemma 1 is used in Lemma 2 and Theorems 1-3, the paper's central claim that SAM implicitly maximizes predictive entropy is unsupported.
  2. [Appendix A, Proof of Lemma 1 and Lemma 2] The quadratic term in Taylor's theorem is evaluated at an intermediate point θ′ on the segment between θ and θ̃, but the stated Hessian assumption applies to θ (or to each example at the current θ). The lower bound κ_min(∇²ℓ_θ(z)) > -1/ρ does not imply the same bound at θ′, so the second-order inequality is not justified. The same issue is inherited by Lemma 2, whose proof states that a 'similar argument' concludes the proof.
  3. [Section 4.1, Theorem 1] The proof of Theorem 1 relies on Lemma 1 to ensure λ = (1-p̃_y)/(1-p_y) > 1 and to justify the entropy inequality. Once Lemma 1 is invalid, the claimed lower bound ℓ̃_θ(z) ≥ ℓ_θ(z) - λH(p_y) + H(p̃_y) with λ > 1 does not follow, and the subsequent interpretation that SAM pushes p_y away from overconfidence is not established by the given arguments. Theorems 2 and 3 inherit this issue because they use the same Lemma.
minor comments (4)
  1. [Appendix C, Table S8] The Adaptive ECE for SAM on WRN-28-10 CIFAR-100 is reported as 4.67 in Table S8, while the corresponding value in Table 4 is 2.17; this inconsistency should be resolved.
  2. [Algorithm 1, Step 3] Step 3 says 'Compute cross-entropy loss' but uses the notation ℓ̃_θ(zi), which is the modified loss defined in Equation (1); the pseudocode should distinguish the loss used for the ascent step from the loss used for the descent step.
  3. [Section 4.1, Figure 2] The text states that λ is a monotone-increasing function of ρ and p̃_y 'when the equality holds,' but the monotonicity is not proved and the figure shows λ only as a function of p̃_y for fixed ρ values; the statement should be made precise or qualified.
  4. [Conclusion] The conclusion acknowledges that the study is limited to cross-entropy loss, but the theoretical analysis also silently assumes conditions on the gradient norm and on the Hessian at intermediate Taylor points; these assumptions should be stated explicitly if a revised version is prepared.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the entropy-regularization claim is a derived theorem, and the empirical validation does not reuse the claim as evidence.

full rationale

The paper's central theoretical claim is that SAM implicitly maximizes the entropy of the predictive distribution. This is presented as Theorem 1, an inequality derived from Lemma 1, which is in turn attempted from an assumed Hessian lower bound. The entropy bound is an algebraic consequence of the intended lemma, not an input to the argument. No expected calibration error (ECE) value or calibration metric is fitted into the perturbation radius ρ, the CSAM coefficient γ, or the theorem statements, so the 'predictions' are not statistically forced. The proposed CSAM method is validated empirically against SAM and other calibration baselines on held-out test data, which does not presuppose the theoretical claim being tested. The paper explicitly limits its theory to the cross-entropy loss, which is a scope limitation rather than a circular step. The self-citations (Tan et al., 2024a,b) appear in the related-work discussion and are not load-bearing for the calibration theorem. The noteworthy weakness is a possible missing gradient-norm condition in the proof of Lemma 1, but that is a soundness gap, not a circularity: the claimed result does not reduce by construction to its own assumptions.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central proof depends on two assumptions beyond standard calculus: the Hessian lower bound (stated) and a hidden gradient-norm lower bound (unstated). CSAM introduces one tuned hyperparameter gamma. No new physical entities are introduced.

free parameters (2)
  • perturbation radius rho = 0.05 (CIFAR-10, ResNet/ViT), 0.2 (CIFAR-100 and SAM/CSAM comparisons)
    Hyperparameter of SAM that controls the magnitude of the implicit entropy regularization; it appears directly in Lemma 1's exponential bound and is tuned per dataset.
  • CSAM regularization coefficient gamma = Grid {0.5, 1.0, 2.0}; best value chosen per dataset
    Introduced in Eq. (1) to amplify the penalty on overconfident examples; selected by validation performance, not derived from first principles.
assumptions (4)
  • domain assumption The smallest Hessian eigenvalue of the loss is lower-bounded by -1/rho along the optimization trajectory.
    Stated in Lemma 1 and justified by a citation (Zhou et al., 2021, Section 6.2); used to control the second-order term in the Taylor expansion.
  • ad hoc to paper The gradient norm of the cross-entropy loss is at least 1, so the linear Taylor term rho * ||gradient|| can be treated as rho.
    Unstated in Lemma 1; without it the proof's central inequality fails. This is the load-bearing gap identified in the weakest_assumption.
  • standard math Taylor's theorem with second-order remainder applies to the empirical loss.
    Used in the proofs of Lemmas 1 and 2 to expand the loss at perturbed weights.
  • domain assumption The data distribution and mini-batch sampling are i.i.d.
    Standard premise for the empirical loss and expected calibration error; stated in Section 3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Understanding The Calibration Benefits of Sharpness-Aware Minimization." pith.science (2026). https://pith.science/paper/3WH35FDE

@misc{pith2026250523866,
  author       = {Pith},
  title        = {Pith review of: Towards Understanding The Calibration Benefits of Sharpness-Aware Minimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3WH35FDE}},
  note         = {Machine review of arXiv:2505.23866}
}
read the original abstract

Deep neural networks have been increasingly used in safety-critical applications such as medical diagnosis and autonomous driving. However, many studies suggest that they are prone to being poorly calibrated and have a propensity for overconfidence, which may have disastrous consequences. In this paper, unlike standard training such as stochastic gradient descent, we show that the recently proposed sharpness-aware minimization (SAM) counteracts this tendency towards overconfidence. The theoretical analysis suggests that SAM allows us to learn models that are already well-calibrated by implicitly maximizing the entropy of the predictive distribution. Inspired by this finding, we further propose a variant of SAM, coined as CSAM, to ameliorate model calibration. Extensive experiments on various datasets, including ImageNet-1K, demonstrate the benefits of SAM in reducing calibration error. Meanwhile, CSAM performs even better than SAM and consistently achieves lower calibration error than other approaches

Figures

Figures reproduced from arXiv: 2505.23866 by the authors.

Figure 1
Figure 1. Confidence and reliability histograms for a PyramidNet (Han et al., 2017) trained on CIFAR-100 (Krizhevsky [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Value of the coefficient λ as a monotone function of the perturbation radius ρ and the predicted probability associated with the true label p˜y. compute in each bin Bi the average confidence conf(Bi) = 1/|Bi | P zj∈Bi max fθ(xj ) and the average accuracy acc(Bi) = 1/|Bi |I[yj = arg maxzj∈Bi fθ(xj )], where I[·] is the indicator function. Then, we can obtain an estimator by averaging over the bins ECE = [ X M i=1 |Bi… view at source ↗
Figure 3
Figure 3. ECE of SAM before post-hoc processing and SGD after various post-hoc processing. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

91 extracted references · 73 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    and Flammarion, N

    Andriushchenko, M. and Flammarion, N. Towards understanding sharpness-aware minimization. In ICML, pp.\ 639--668, 2022

    2022

  3. [3]

    A modern look at the relationship between sharpness and generalization

    Andriushchenko, M., Croce, F., M \"u ller, M., Hein, M., and Flammarion, N. A modern look at the relationship between sharpness and generalization. In ICML, pp.\ 840--902, 2023

    2023

  4. [4]

    Sharpness-aware minimization improves language model generalization

    Bahri, D., Mobahi, H., and Tay, Y. Sharpness-aware minimization improves language model generalization. In ACL, pp.\ 7360--7371, 2022

    2022

  5. [5]

    L., Long, P

    Bartlett, P. L., Long, P. M., and Bousquet, O. The dynamics of sharpness-aware minimization: Bouncing across ravines and drifting towards wide minima. Journal of Machine Learning Research, 24 0 (316): 0 1--36, 2023

    2023

  6. [6]

    Low-pass filtering SGD for recovering flat optima in the deep learning optimization landscape

    Bisla, D., Wang, J., and Choromanska, A. Low-pass filtering SGD for recovering flat optima in the deep learning optimization landscape. In AISTATS, pp.\ 8299--8339, 2022

    2022

  7. [7]

    and Elisseeff, A

    Bousquet, O. and Elisseeff, A. Stability and generalization. Journal of Machine Learning Research, 2: 0 499--526, 2002

    2002

  8. [8]

    Reliability, sufficiency, and the decomposition of proper scores

    Br \"o cker, J. Reliability, sufficiency, and the decomposition of proper scores. Quarterly Journal of the Royal Meteorological Society, 135 0 (643): 0 1512--1519, 2009

    2009

Show all 91 references
  1. [9]

    On mixup regularization

    Carratino, L., Ciss \'e , M., Jenatton, R., and Vert, J.-P. On mixup regularization. Journal of Machine Learning Research, 23 0 (325): 0 1--31, 2022

    2022

  2. [10]

    Entropy- SGD : Biasing gradient descent into wide valleys

    Chaudhari, P., Choromanska, A., Soatto, S., LeCun, Y., Baldassi, C., Borgs, C., Chayes, J., Sagun, L., and Zecchina, R. Entropy- SGD : Biasing gradient descent into wide valleys. Journal of Statistical Mechanics: Theory and Experiment, 2019 0 (12): 0 124018, 2019

    2019

  3. [11]

    When vision T ransformers outperform R esnets without pretraining or strong data augmentations

    Chen, X., Hsieh, C.-J., and Gong, B. When vision T ransformers outperform R esnets without pretraining or strong data augmentations. In ICLR, pp.\ 1--20, 2022

    2022

  4. [12]

    Chib, P. S. and Singh, P. Recent advancements in end-to-end autonomous driving using deep learning: A survey. IEEE Transactions on Intelligent Vehicles, pp.\ 103--118, 2023

    2023

  5. [13]

    M., Biggio, L., Orvieto, A., Proske, F

    Compagnoni, E. M., Biggio, L., Orvieto, A., Proske, F. N., Kersting, H., and Lucchi, A. An SDE for modeling SAM : Theory and insights. In ICML, pp.\ 25209--25253, 2023

    2023

  6. [14]

    Image N et: A large-scale hierarchical image database

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Image N et: A large-scale hierarchical image database. In CVPR, pp.\ 248--255, 2009

    2009

  7. [15]

    Sharp minima can generalize for deep nets

    Dinh, L., Pascanu, R., Bengio, S., and Bengio, Y. Sharp minima can generalize for deep nets. In ICML, pp.\ 1019--1028, 2017

    2017

  8. [16]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, pp.\ 1--21, 2021

    2021

  9. [17]

    T., Zhen, L., Goh, R

    Du, J., Yan, H., Feng, J., Zhou, J. T., Zhen, L., Goh, R. S. M., and Tan, V. Efficient sharpness-aware minimization for improved training of neural networks. In ICLR, pp.\ 1--18, 2022

    2022

  10. [18]

    Efficient and scalable B ayesian neural nets with rank-1 factors

    Dusenberry, M., Jerfel, G., Wen, Y., Ma, Y., Snoek, J., Heller, K., Lakshminarayanan, B., and Tran, D. Efficient and scalable B ayesian neural nets with rank-1 factors. In ICML, pp.\ 2782--2792, 2020

    2020

  11. [19]

    Sharpness-aware minimization for efficiently improving generalization

    Foret, P., Kleiner, A., Mobahi, H., and Neyshabur, B. Sharpness-aware minimization for efficiently improving generalization. In ICLR, pp.\ 1--20, 2021

    2021

  12. [20]

    and Fujisawa, M

    Futami, F. and Fujisawa, M. Information-theoretic generalization analysis for expected calibration error. In NeurIPS, pp.\ 1--52, 2024

    2024

  13. [21]

    and Ghahramani, Z

    Gal, Y. and Ghahramani, Z. Dropout as a B ayesian approximation: Representing model uncertainty in deep learning. In ICML, pp.\ 1050--1059, 2016

    2016

  14. [22]

    Adafocal: Calibration-aware adaptive focal loss

    Ghosh, A., Schaaf, T., and Gormley, M. Adafocal: Calibration-aware adaptive focal loss. In NeurIPS, pp.\ 1583--1595, 2022

    2022

  15. [23]

    Gneiting, T., Balabdaoui, F., and Raftery, A. E. Probabilistic forecasts, calibration and sharpness. Journal of the Royal Statistical Society Series B: Statistical Methodology, 69 0 (2): 0 243--268, 2007

    2007

  16. [24]

    Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q. On calibration of modern neural networks. In ICML, pp.\ 1321--1330, 2017

    2017

  17. [25]

    Deep pyramidal residual networks

    Han, D., Kim, J., and Kim, J. Deep pyramidal residual networks. In CVPR, pp.\ 5927--5935, 2017

    2017

  18. [26]

    Train faster, generalize better: Stability of stochastic gradient descent

    Hardt, M., Recht, B., and Singer, Y. Train faster, generalize better: Stability of stochastic gradient descent. In ICML, pp.\ 1225--1234, 2016

    2016

  19. [27]

    Z., Snoek, J., Lakshminarayanan, B., Dai, A

    Havasi, M., Jenatton, R., Fort, S., Liu, J. Z., Snoek, J., Lakshminarayanan, B., Dai, A. M., and Tran, D. Training independent subnetworks for robust prediction. In ICLR, pp.\ 1--13, 2021

    2021

  20. [28]

    Deep residual learning for image recognition

    He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In CVPR, pp.\ 770--778, 2016

    2016

  21. [29]

    and Dietterich, T

    Hendrycks, D. and Dietterich, T. Benchmarking neural network robustness to common corruptions and perturbations. In ICLR, pp.\ 1--16, 2019

    2019

  22. [30]

    Hinton, G. E. and van Camp, D. Keeping neural networks simple. In IJCNN, pp.\ 11--18, 1993

    1993

  23. [31]

    E., and Weinberger, K

    Huang, G., Li, Y., Pleiss, G., Liu, Z., Hopcroft, J. E., and Weinberger, K. Q. Snapshot ensembles: Train 1, get m for free. In ICLR, pp.\ 1--14, 2017 a

    2017

  24. [32]

    Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K. Q. Densely connected convolutional networks. In CVPR, pp.\ 4700--4708, 2017 b

    2017

  25. [33]

    Izmailov, P., Podoprikhin, D., Garipov, T., Vetrov, D., and Wilson, A. G. Averaging weights leads to wider optima and better generalization. In UAI, pp.\ 1--10, 2018

    2018

  26. [34]

    An adaptive policy to employ sharpness-aware minimization

    Jiang, W., Yang, H., Zhang, Y., and Kwok, J. An adaptive policy to employ sharpness-aware minimization. In ICLR, pp.\ 1--19, 2023

    2023

  27. [35]

    Calibrating predictive model estimates to support personalized medicine

    Jiang, X., Osl, M., Kim, J., and Ohno-Machado, L. Calibrating predictive model estimates to support personalized medicine. Journal of the American Medical Informatics Association, 19 0 (2): 0 263--274, 2012

    2012

  28. [36]

    Kaddour, J., Liu, L., Silva, R., and Kusner, M. J. When do flat minima optimizers work? In NeurIPS, pp.\ 16577--16595, 2022

    2022

  29. [37]

    S., Mudigere, D., Nocedal, J., Smelyanskiy, M., and Tang, P

    Keskar, N. S., Mudigere, D., Nocedal, J., Smelyanskiy, M., and Tang, P. T. P. On large-batch training for deep learning: Generalization gap and sharp minima. In ICLR, pp.\ 1--16, 2017

    2017

  30. [38]

    Stability analysis of sharpness-aware minimization

    Kim, H., Park, J., Choi, Y., and Lee, J. Stability analysis of sharpness-aware minimization. arXiv preprint arXiv:2301.06308, 2023

    2023 arXiv

  31. [39]

    X., and Hospedales, T

    Kim, M., Li, D., Hu, S. X., and Hospedales, T. Fisher SAM : Information geometry and sharpness aware minimisation. In ICML, pp.\ 11148--11161, 2022

    2022

  32. [40]

    Learning multiple layers of features from tiny images

    Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009

    2009

  33. [41]

    Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration

    Kull, M., Perello Nieto, M., K \"a ngsepp, M., Silva Filho, T., Song, H., and Flach, P. Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration. In NeurIPS, pp.\ 1--11, 2019

    2019

  34. [42]

    Trainable calibration measures for neural networks from kernel mean embeddings

    Kumar, A., Sarawagi, S., and Jain, U. Trainable calibration measures for neural networks from kernel mean embeddings. In ICML, pp.\ 2805--2814, 2018

    2018

  35. [43]

    Kuzucu, S., Oksuz, K., Sadeghi, J., and Dokania, P. K. On calibration of object detectors: Pitfalls, evaluation and baselines. In ECCV, pp.\ 185--204, 2025

    2025

  36. [44]

    Kwon, J., Kim, J., Park, H., and Choi, I. K. A SAM : Adaptive sharpness-aware minimization for scale-invariant learning of deep neural networks. In ICML, pp.\ 5905--5914, 2021

    2021

  37. [45]

    Simple and scalable predictive uncertainty estimation using deep ensembles

    Lakshminarayanan, B., Pritzel, A., and Blundell, C. Simple and scalable predictive uncertainty estimation using deep ensembles. In NeurIPS, pp.\ 1--12, 2017

    2017

  38. [46]

    and Giannakis, G

    Li, B. and Giannakis, G. B. Enhancing sharpness-aware optimization through variance suppression. In NeurIPS, pp.\ 1--19, 2023

    2023

  39. [47]

    Towards efficient and scalable sharpness-aware minimization

    Liu, Y., Mai, S., Chen, X., Hsieh, C.-J., and You, Y. Towards efficient and scalable sharpness-aware minimization. In CVPR, pp.\ 12360--12370, 2022

    2022

  40. [48]

    and Hutter, F

    Loshchilov, I. and Hutter, F. S GDR : Stochastic gradient descent with warm restarts. In ICLR, pp.\ 1--16, 2017

    2017

  41. [49]

    and Hutter, F

    Loshchilov, I. and Hutter, F. Decoupled weight decay regularization. In ICLR, pp.\ 1--18, 2019

    2019

  42. [50]

    Make sharpness-aware minimization stronger: A sparsified perturbation approach

    Mi, P., Shen, L., Ren, T., Zhou, Y., Sun, X., Ji, R., and Tao, D. Make sharpness-aware minimization stronger: A sparsified perturbation approach. In NeurIPS, pp.\ 30950--30962, 2022

    2022

  43. [51]

    Revisiting the calibration of modern neural networks

    Minderer, M., Djolonga, J., Romijnders, R., Hubis, F., Zhai, X., Houlsby, N., Tran, D., and Lucic, M. Revisiting the calibration of modern neural networks. In NeurIPS, pp.\ 15682--15694, 2021

    2021

  44. [52]

    and Khan, M

    M \"o llenhoff, T. and Khan, M. E. SAM as an optimal relaxation of B ayes. In ICLR, pp.\ 1--25, 2023

    2023

  45. [53]

    Calibrating deep neural networks using focal loss

    Mukhoti, J., Kulharia, V., Sanyal, A., Golodetz, S., Torr, P., and Dokania, P. Calibrating deep neural networks using focal loss. In NeurIPS, pp.\ 15288--15299, 2020

    2020

  46. [54]

    M \"u ller, R., Kornblith, S., and Hinton, G. E. When does label smoothing help? In NeurIPS, pp.\ 1--10, 2019

    2019

  47. [55]

    P., Cooper, G., and Hauskrecht, M

    Naeini, M. P., Cooper, G., and Hauskrecht, M. Obtaining well calibrated probabilities using B ayesian binning. In AAAI, pp.\ 1--7, 2015

    2015

  48. [56]

    Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., and Ng, A. Y. Reading digits in natural images with unsupervised feature learning. In NeurIPS Workshop, pp.\ 1--9, 2011

    2011

  49. [57]

    Deep neural networks are easily fooled: High confidence predictions for unrecognizable images

    Nguyen, A., Yosinski, J., and Clune, J. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In CVPR, pp.\ 427--436, 2015

    2015

  50. [58]

    G., and Goldstein, T

    Ni, R., Chiang, P.-y., Geiping, J., Goldblum, M., Wilson, A. G., and Goldstein, T. K- SAM : Sharpness-aware minimization at the speed of SGD . arXiv preprint arXiv:2210.12864, 2022

    2022 arXiv

  51. [59]

    and Caruana, R

    Niculescu-Mizil, A. and Caruana, R. Predicting good probabilities with supervised learning. In ICML, pp.\ 625--632, 2005

    2005

  52. [60]

    Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift

    Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J., Lakshminarayanan, B., and Snoek, J. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In NeurIPS, pp.\ 1--12, 2019

    2019

  53. [61]

    Regularizing neural networks by penalizing confident output distributions

    Pereyra, G., Tucker, G., Chorowski, J., Kaiser, L., and Hinton, G. Regularizing neural networks by penalizing confident output distributions. In ICLR Workshop, pp.\ 1--11, 2017

    2017

  54. [62]

    Platt, J. et al. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in Large Margin Classifiers, 10 0 (3): 0 61--74, 1999

    1999

  55. [63]

    E., Farazi, M., Saratchandran, H., and Lucey, S

    Ramasinghe, S., MacDonald, L. E., Farazi, M., Saratchandran, H., and Lucey, S. How much does initialization affect generalization? In ICML, pp.\ 28637--28655, 2023

    2023

  56. [64]

    On deep neural network calibration by regularization and its impact on refinement

    Singh, A. On deep neural network calibration by regularization and its impact on refinement. In ICLR, pp.\ 1--15, 2021

    2021

  57. [65]

    Sharpness-aware L ookahead for accelerating convergence and improving generalization

    Tan, C., Zhang, J., Liu, J., and Gong, Y. Sharpness-aware L ookahead for accelerating convergence and improving generalization. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp.\ 1--14, 2024 a

    2024

  58. [66]

    Stabilizing sharpness-aware minimization through a simple renormalization strategy

    Tan, C., Zhang, J., Liu, J., Wang, Y., and Hao, Y. Stabilizing sharpness-aware minimization through a simple renormalization strategy. arXiv preprint arXiv:2401.07250, 2024 b

    2024 arXiv

  59. [67]

    Dual focal loss for calibration

    Tao, L., Dong, M., and Xu, C. Dual focal loss for calibration. In ICML, pp.\ 33833--33849, 2023

    2023

  60. [68]

    A., Bhattacharya, T., and Michalak, S

    Thulasidasan, S., Chennupati, G., Bilmes, J. A., Bhattacharya, T., and Michalak, S. On mixup training: Improved calibration and predictive uncertainty for deep neural networks. In NeurIPS, pp.\ 1--12, 2019

    2019

  61. [69]

    O., Houlsby, N., Kolesnikov, A., Beyer, L., Zhai, X., Unterthiner, T., Yung, J., Steiner, A., Keysers, D., Uszkoreit, J., et al

    Tolstikhin, I. O., Houlsby, N., Kolesnikov, A., Beyer, L., Zhai, X., Unterthiner, T., Yung, J., Steiner, A., Keysers, D., Uszkoreit, J., et al. Mlp-mixer: An all-mlp architecture for vision. In NeurIPS, pp.\ 24261--24272, 2021

    2021

  62. [70]

    Calibration in deep learning: A survey of the state-of-the-art

    Wang, C. Calibration in deep learning: A survey of the state-of-the-art. arXiv preprint arXiv:2308.01222, 2023

    2023 arXiv

  63. [71]

    Rethinking calibration of deep neural networks: Do not be afraid of overconfidence

    Wang, D.-B., Feng, L., and Zhang, M.-L. Rethinking calibration of deep neural networks: Do not be afraid of overconfidence. In NeurIPS, pp.\ 11809--11820, 2021 a

    2021

  64. [72]

    On the pitfall of mixup for uncertainty calibration

    Wang, D.-B., Li, L., Zhao, P., Heng, P.-A., and Zhang, M.-L. On the pitfall of mixup for uncertainty calibration. In CVPR, pp.\ 7609--7618, 2023

    2023

  65. [73]

    Eliminating sharp minima from SGD with truncated heavy-tailed noise

    Wang, X., Oh, S., and Rhee, C.-H. Eliminating sharp minima from SGD with truncated heavy-tailed noise. In ICLR, pp.\ 1--157, 2021 b

    2021

  66. [74]

    How does sharpness-aware minimization minimizes sharpness? In NeurIPS Workshop, pp.\ 1--94, 2022

    Wen, K., Ma, T., and Li, Z. How does sharpness-aware minimization minimizes sharpness? In NeurIPS Workshop, pp.\ 1--94, 2022

    2022

  67. [75]

    Sharpness minimization algorithms do not only minimize sharpness to achieve better generalization

    Wen, K., Li, Z., and Ma, T. Sharpness minimization algorithms do not only minimize sharpness to achieve better generalization. In NeurIPS, pp.\ 1--12, 2024

    2024

  68. [76]

    Batch E nsemble: an alternative approach to efficient ensemble and lifelong learning

    Wen, Y., Tran, D., and Ba, J. Batch E nsemble: an alternative approach to efficient ensemble and lifelong learning. In ICLR, pp.\ 1--19, 2020

    2020

  69. [77]

    Perfecting imperfect physical neural networks with transferable robustness using sharpness-aware training

    Xu, T., Luo, Z., Liu, S., Fan, L., Xiao, Q., Wang, B., Wang, D., and Huang, C. Perfecting imperfect physical neural networks with transferable robustness using sharpness-aware training. arXiv preprint arXiv:2411.12352, 2024

    2024 arXiv

  70. [78]

    Sharpness-aware minimization revisited: Weighted sharpness as a regularization term

    Yue, Y., Jiang, J., Ye, Z., Gao, N., Liu, Y., and Zhang, K. Sharpness-aware minimization revisited: Weighted sharpness as a regularization term. In KDD, pp.\ 1--10, 2023

    2023

  71. [79]

    and Elkan, C

    Zadrozny, B. and Elkan, C. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In ICML, pp.\ 609--616, 2001

    2001

  72. [80]

    and Elkan, C

    Zadrozny, B. and Elkan, C. Transforming classifier scores into accurate multiclass probability estimates. In KDD, pp.\ 694--699, 2002

    2002

  73. [81]

    and Komodakis, N

    Zagoruyko, S. and Komodakis, N. Wide residual networks. arXiv preprint arXiv:1605.07146, 2016

    2016 arXiv

  74. [82]

    Mixup: Beyond empirical risk minimization

    Zhang, H. Mixup: Beyond empirical risk minimization. In ICLR, pp.\ 1--13, 2018

    2018

  75. [83]

    Zhang, M., Lucas, J., Ba, J., and Hinton, G. E. Lookahead optimizer: k steps forward, 1 step back. In NeurIPS, pp.\ 1--19, 2019

    2019

  76. [84]

    G A-SAM : Gradient-strength based adaptive sharpness-aware minimization for improved generalization

    Zhang, Z., Luo, R., Su, Q., and Sun, X. G A-SAM : Gradient-strength based adaptive sharpness-aware minimization for improved generalization. In EMNLP, pp.\ 3888--3903, 2022

    2022

  77. [85]

    Penalizing gradient norm for efficiently improving generalization in deep learning

    Zhao, Y., Zhang, H., and Hu, X. Penalizing gradient norm for efficiently improving generalization in deep learning. In ICML, pp.\ 26982--26992, 2022 a

    2022

  78. [86]

    Randomized sharpness-aware training for boosting computational efficiency in deep learning

    Zhao, Y., Zhang, H., and Hu, X. Randomized sharpness-aware training for boosting computational efficiency in deep learning. arXiv preprint arXiv:2203.09962, 2022 b

    2022 arXiv

  79. [87]

    Regularizing neural networks via adversarial model perturbation

    Zheng, Y., Zhang, R., and Mao, Y. Regularizing neural networks via adversarial model perturbation. In CVPR, pp.\ 8156--8165, 2021

    2021

  80. [88]

    Towards understanding why L ookahead generalizes better than SGD and beyond

    Zhou, P., Yan, H., Yuan, X., Feng, J., and Yan, S. Towards understanding why L ookahead generalizes better than SGD and beyond. In NeurIPS, pp.\ 27290--27304, 2021

    2021

  81. [89]

    Imb SAM : A closer look at sharpness-aware minimization in class-imbalanced recognition

    Zhou, Y., Qu, Y., Xu, X., and Shen, H. Imb SAM : A closer look at sharpness-aware minimization in class-imbalanced recognition. In ICCV, pp.\ 11345--11355, 2023

    2023

  82. [90]

    Sharpness-aware minimization efficiently selects flatter minima late in training

    Zhou, Z., Wang, M., Mao, Y., Li, B., and Yan, J. Sharpness-aware minimization efficiently selects flatter minima late in training. In ICLR, pp.\ 1--16, 2025

    2025

  83. [91]

    Rethinking data distillation: Do not overlook calibration

    Zhu, D., Lei, B., Zhang, J., Fang, Y., Xie, Y., Zhang, R., and Xu, D. Rethinking data distillation: Do not overlook calibration. In ICCV, pp.\ 4935--4945, 2023

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.