Pith. sign in

REVIEW 5 major objections 5 minor 113 references

Mini-batch SAM provably favors flat minima: the sharpest stable curvature is capped by a cube-root bound in batch size, learning rate, and perturbation radius.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

SAM's largest Hessian eigenvalue is bounded by the cube root of bGamma/(2*rho*eta^2), so larger radius, smaller batch, or larger learning rate restrict linearly stable minima to flatter regions.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Theorem 1 looks like a promising extension of SGD linear stability to SAM, but the proof is deferred to a missing supplement and the empirical test doesn't isolate the bound, so the paper is not yet ready. the 5 major comments →

arxiv 2608.03197 v1 pith:C4LZXTKJ submitted 2026-08-04 cs.LG

On the Implicit Flatness Bias of Sharpness-Aware Minimization: A Linear Stability Analysis with Quantitative Hyperparameter Bounds

Jiaxin Deng , Junbiao Pang This is my paper
classification cs.LG
keywords sharpness-aware minimizationimplicit biasflat minimalinear stabilityHessian eigenvaluehyperparameter boundsgeneralizationSAM perturbation radius
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to explain quantitatively why Sharpness-Aware Minimization (SAM) lands in flat minima, and how the perturbation radius $\rho$ combines with batch size and learning rate rather than acting alone. It proves that a linearly stable interpolating minimum of mini-batch SAM must have largest Hessian eigenvalue $\lambda_{\max}$ no larger than $\sqrt[3]{b\Gamma/(2\rho\eta^2)}$, where $b$ is batch size, $\eta$ is learning rate, and $\Gamma$ bounds the gradient norm. The bound says the flatness bias is governed by the ratio $b/(\rho\eta^2)$: smaller batches, larger learning rates, and larger (but still local) radii all restrict SAM to flatter minima. A controlled study of 900 CIFAR-100 models finds the predicted negative association between $\rho$ and the largest Hessian eigenvalue, and a theory-derived variant, TLC-SAM, adapts $\rho$ by monitoring Taylor-approximation error and further lowers the top eigenvalue.

Core claim

The central claim is a theorem: for mini-batch SAM near an interpolating global minimum, under local linearization, squared loss, and a gradient-noise alignment assumption, every linearly stable solution obeys $\lambda_{\max} \le \sqrt[3]{b\,\mathbb{E}[\|\nabla L(w_t)\|]/(2\rho\eta^2)} \le \sqrt[3]{b\Gamma/(2\rho\eta^2)}$. The middle term is the sharpened form, using the expected gradient norm; the right-hand side is the clean bound using the uniform gradient bound $\Gamma$. The paper reads this as a quantitative mechanism: SAM's adversarial perturbation injects $\rho$-dependent terms into the stochastic-gradient noise covariance, and linear stability then forbids directions whose curvature

What carries the argument

The key object is Lemma 1, an approximate formula for the covariance of SAM's mini-batch gradient noise near an interpolating minimum, $\Sigma(w_t)\approx 2L(w_t)\big(G(w_t)+2\rho\,G(w_t)H(w_t)^\top/\|\nabla L(w_t)\|+\rho^2 H(w_t)G(w_t)H(w_t)/\|\nabla L(w_t)\|^2\big)$. Inserting this covariance into the linear-stability condition for the linearized squared loss produces the cubic bound; the SAM perturbation $\hat{\varepsilon}=\rho\nabla L/\|\nabla L\|$ enters only through the $\rho$ and $\rho^2$ coefficients, which is why the bound has $\rho$ in the denominator.

Load-bearing premise

The load-bearing premise is that near an interpolating minimum the per-sample losses and gradients are nearly decoupled, so the SAM noise covariance really takes the algebraic form in Lemma 1; if that decoupling fails, the $\rho$-dependent terms change and the cubic bound does not follow.

What would settle it

Train a genuinely over-parameterized network (for example ResNet-18 on CIFAR-100) with fixed batch size and learning rate, measure $\Gamma$ and the largest Hessian eigenvalue at the final minimum over a sweep of $\rho$, and check whether $\lambda_{\max}$ ever exceeds $\sqrt[3]{b\Gamma/(2\rho\eta^2)}$ while the Taylor-locality error stays small. A single violation within the stable local regime refutes the theorem. Alternatively, estimate the SAM gradient-noise covariance empirically and test whether the $\rho$-dependent cross term is present at the predicted magnitude.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Flatness of SAM solutions is set by the ratio $b/(\rho\eta^2)$: users can trade batch size and learning rate against perturbation radius while keeping the same sharpness cap.
  • Increasing $\rho$ flattens the minimum only while the first-order Taylor model remains valid; beyond that, linear stability and safe training are lost.
  • SAM variants such as ASAM and F-SAM can be seen as ways to enlarge the effective radius or to reduce descent interference, thereby tightening the same bound.
  • A radius controller that expands $\rho$ when the Taylor approximation is accurate can lower the largest Hessian eigenvalue relative to fixed-radius SAM, as TLC-SAM demonstrates.
  • The largest Hessian eigenvalue is not the whole spectrum: SAM and TLC-SAM also redistribute spectral mass, lowering the dominant curvature while keeping a more dispersed spectrum than pure sharpness targeting would suggest.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper reports a pooled rank correlation with $\sqrt[3]{b/(2\rho\eta^2)}$, but does not test whether contours of measured $\lambda_{\max}$ collapse onto that product; a direct grid experiment varying $b,\eta,\rho$ and plotting $\lambda_{\max}$ against $b/(\rho\eta^2)$ would test whether the bound is tight, not just order-preserving.
  • If the $\rho$-dependent noise covariance is the actual mechanism, full-batch SAM (where the mini-batch noise term vanishes) should show much weaker flatness bias; this ablation is not in the paper.
  • The Taylor-locality error $E_t$ could serve as a per-point confidence measure for trusting the local bound, and the same controller could be applied to other SAM variants beyond radius adaptation.
  • The bound's scaling suggests $\rho$ should grow roughly as $\eta^{-2}$ to hold flatness fixed across a learning-rate schedule; a schedule that grows $\rho$ late in training could preserve flatness as $\eta$ decays, a direction the theory points to but does not study.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a linear-stability analysis of mini-batch Sharpness-Aware Minimization (SAM) near interpolating global minima. Its central claim is Theorem 1: every linearly stable minimum satisfies λ_max ≤ (b E[||∇L(w_t)||]/(2ρη^2))^{1/3} ≤ (bΓ/(2ρη^2))^{1/3}, where b is batch size, η is learning rate, ρ is the perturbation radius, and Γ bounds the gradient norm. The paper also reports a large empirical study (900 models on CIFAR-100 with ResNet-18 and VGG-19) claiming to validate the predicted negative association between ρ and the largest Hessian eigenvalue, and introduces TLC-SAM, a radius-adaptation method based on Taylor-locality error.

Significance. If Theorem 1 were proved and its assumptions verified, the result would provide a quantitative, testable link between SAM's hyperparameters and the curvature of its stable minima, potentially guiding hyperparameter selection and the design of SAM variants. The paper also contains a substantial controlled empirical study and a practical adaptive method, TLC-SAM, which are strengths. However, the central theorem is stated without a proof, the key noise-covariance lemma rests on an informal and unverified 'nearly decoupled' assumption, and the empirical evidence is mostly correlational rather than a direct test of the bound. The current contribution is therefore a promising conjecture plus supporting trends, not an established quantitative theory.

major comments (5)
  1. [Theoretical Evidence, Theorem 1 (Eq. 10)] The central result is stated without proof. After Lemma 1, the text simply says 'we have' and displays the cubic bound, with no derivation connecting Definition 1's linear-stability condition or Lemma 1's covariance to inequality (10). The proof is deferred to a supplementary material that is not included. Since the cubic dependence on ρ, η, and b and the use of E||∇L(w_t)|| are non-obvious, this missing proof makes the main theoretical claim unsupported as presented.
  2. [Assumption 1 and Theorem 1 (Eq. 10)] Assumption 1 states E[||∇L(w_t)||^2] ≤ Γ, but Theorem 1's second inequality uses E[||∇L(w_t)||] ≤ Γ. By Jensen, E||∇L|| ≤ sqrt(E||∇L||^2) ≤ sqrt(Γ). The displayed upper bound should therefore contain sqrt(Γ), not Γ, unless the assumption is restated as a bound on the first moment. This is a load-bearing inconsistency in the main inequality.
  3. [Lemma 1 and Assumptions 2–3] Lemma 1's covariance formula (9) is derived under an informal 'nearly decoupled' condition on ∇f(x_i; w_t) and L_i(w_t), which is not stated among Assumptions 1–3 and is not verified for the CIFAR-scale networks used in the experiments. Assumption 2 says the full-batch gradient is negligible compared with per-sample gradients, which near an interpolating minimum is plausible only if per-sample gradients remain nonzero; the exact noise model is crucial because the ρ-dependent terms in (9) are what make Theorem 1 SAM-specific. Without a precise and justified noise model, the theorem's derivation is incomplete.
  4. [Empirical Evidence, Table 1] The empirical validation reports a negative Spearman correlation between ρ and λ_max. However, Theorem 1 is an upper bound on λ_max under linear stability assumptions, not a guarantee that increasing ρ strictly decreases λ_max for arbitrary trained models. The bound also depends on E||∇L(w_t)|| (or Γ), which is not measured or controlled. The reported correlations are consistent with the theorem but do not test its quantitative form, so they cannot substitute for the missing derivation.
  5. [Empirical Evidence, Table 2] Table 2 correlates test accuracy with (b/(2ρη^2))^{1/3} after pooling over all hyperparameter settings. But Theorem 1 makes no direct prediction about test accuracy; it predicts an upper bound on λ_max for linearly stable minima, and Γ is not fixed across runs. This pooled correlation is therefore only a loose heuristic check and does not validate the bound.
minor comments (5)
  1. [Eq. (9)] Notation: H(w_t) is symmetric, so H(w_t)^⊤ is redundant; also the expression mixes matrices and norms without dimensional clarity. Please clarify the exact meaning of the terms.
  2. [Assumption 1] The expectation E[·] is not explicitly over the mini-batch randomness; please specify the probability space.
  3. [Figure 1] The figure caption says 'Independent x/y axes' and many panels appear to lack numerical axis labels; this makes the visual trend hard to assess. Please provide full axis labels or larger panels.
  4. [Table 3] The table mixes results from different papers with different training epochs (e.g., 300-epoch PyramidNet entries). The footnote cautions against strict ranking, but the table layout invites direct comparison; consider separating the authors' own runs from literature values more clearly.
  5. [References] Some references are incomplete, e.g., 'Wu, L.; Ma, C.; et al. 2018' should list all authors. Also, the abstract and body contain typesetting artifacts such as '3 p' before formulas; please correct them.

Circularity Check

0 steps flagged

No circularity: the central bound is derived from stated assumptions and an external linear-stability framework, not from the data it predicts.

full rationale

The paper's central claim is Theorem 1, a quantitative upper bound on the largest Hessian eigenvalue λ_max for linearly stable SAM minima. The derivation chain is: (1) approximate the SAM update (Eq. 7-8); (2) characterize the gradient-noise covariance under local linearization and near-interpolating assumptions (Lemma 1); (3) invoke the external linear-stability framework of (Wu, Wang, and Su 2022) together with Definition 1 to obtain the cubic bound. The bound is not defined in terms of λ_max; it is a nontrivial inequality relating λ_max to b, η, ρ, and a gradient-norm bound Γ. No parameter is fitted to the empirical data and then renamed a prediction. The empirical study (CIFAR-100, 900 models) tests a qualitative monotonic implication of the bound (larger ρ associated with smaller λ_max), but the bound itself is not derived from that data; the correlation is reported as evidence, not as input. TLC-SAM is designed from the theory's stability–locality principle and evaluated separately; its empirical behavior is not a logical consequence assumed in the proof of Theorem 1. There are no self-citations to the present authors' prior work. The only notable gap is that the proof of Theorem 1 is deferred to a supplementary material not included, so the algebraic steps connecting Lemma 1 to the cubic bound cannot be inspected. However, a missing proof is a rigor/completeness concern, not circularity: the paper does not redefine the bound as the stability condition, nor does it silently assume the inequality it claims to prove. Under the standard for this review, the derivation is self-contained with respect to circularity.

Axiom & Free-Parameter Ledger

1 free parameters · 5 axioms · 0 invented entities

The core bound rests on five domain assumptions about interpolation, gradient bounds, gradient-noise alignment, local linearization, and covariance decoupling. None are verified in the provided text. Gamma is an unestimated free constant. TLC-SAM is a proposed algorithm, not an invented physical entity; its design choices are heuristics rather than axioms.

free parameters (1)
  • Gamma (gradient-norm bound) = unspecified
    Assumption 1 introduces Gamma bounding E[||nabla L(w_t)||^2]; the theorem's bound uses Gamma but no value is given. The empirical validation drops Gamma entirely, treating it as constant across settings, which is untested.
axioms (5)
  • domain assumption Over-parameterized interpolating minimum: min_w L(w)=0 and G(w*)=H(w*).
    Assumed in Setup; the analysis only applies at global minima of squared loss where training loss is zero.
  • domain assumption Assumption 1: exists Gamma with E[||nabla L(w_t)||^2] <= Gamma for all t.
    This bounds gradient norm but is never verified; the empirical study does not estimate Gamma.
  • domain assumption Assumption 2: full-batch gradient is negligible compared with per-sample gradients.
    Needed for gradient-noise alignment; likely false outside a small neighborhood of an interpolating minimum.
  • domain assumption Assumption 3: local linearization of model around w* holds and w*=0.
    The whole stability analysis is local and linear; real networks trained on CIFAR are not well described by a linear model near a global min.
  • domain assumption Lemma 1: nabla f(x_i; w_t) and L_i(w_t) are nearly decoupled; G(w_t) approximates H(w*).
    These decoupling and local-relation assumptions underpin the SAM gradient-noise covariance; no empirical check is provided.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of On the Implicit Flatness Bias of Sharpness-Aware Minimization: A Linear Stability Analysis with Quantitative Hyperparameter Bounds." pith.science (2026). https://pith.science/paper/C4LZXTKJ

@misc{pith2026260803197,
  author       = {Pith},
  title        = {Pith review of: On the Implicit Flatness Bias of Sharpness-Aware Minimization: A Linear Stability Analysis with Quantitative Hyperparameter Bounds},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C4LZXTKJ}},
  note         = {Machine review of arXiv:2608.03197}
}
Share X Bluesky LinkedIn Reddit HN
abstract

Sharpness-Aware Minimization (SAM) improves generalization by seeking parameters whose loss is robust to local adversarial perturbations, but the quantitative mechanism underlying its implicit bias toward flat minima remains unclear. In particular, the perturbation radius $\rho$ is typically treated as an isolated tuning parameter, despite defining the neighborhood in which SAM measures sharpness. We analyze mini-batch SAM near an interpolating minimum through linear stability. Under local linearization and gradient-noise alignment assumptions, we prove that every linearly stable minimum satisfies $\lambda_{\max}\leq\sqrt[3]{b\Gamma/(2\rho\eta^2)}$, where $\lambda_{\max}$ is the largest Hessian eigenvalue, $b$ is the batch size, $\eta$ is the learning rate, and $\Gamma$ bounds the gradient norm. The bound quantitatively characterizes SAM's implicit flatness bias: holding the other quantities fixed, a smaller batch size, a larger learning rate, or a larger radius restricts linearly stable SAM to flatter minima. It also exposes a necessary trade-off: $\rho$ should be large enough to promote flatness, yet remain local enough to preserve the approximation and stable training. We validate this prediction in a controlled study of 900 models on CIFAR-100 with ResNet-18 and VGG-19, where increasing $\rho$ is consistently associated with a smaller largest Hessian eigenvalue across batch-size and learning-rate settings. Finally, we instantiate the analysis in Taylor-Locality Controlled SAM (TLC-SAM), which adjusts $\rho$ using the observed Taylor-approximation error and further reduces the top Hessian eigenvalue relative to fixed-radius SAM. Our results provide quantitative hyperparameter bounds and a stability--locality perspective for analyzing and designing SAM variants.

Figures

Figures reproduced from arXiv: 2608.03197 by Jiaxin Deng, Junbiao Pang.

Figure 1
Figure 1. Figure 1: Effect of ρ on the largest Hessian eigenvalue (left) and test accuracy (right) for ResNet-18 on CIFAR-100. Rows and columns index five batch sizes and five learning rates, respectively; each panel contains results from 18 separately trained models. Full-size panels are provided in the supple￾mentary material. in which sharpness is measured and the strength with which SAM biases optimization toward flat reg… view at source ↗
Figure 2
Figure 2. Figure 2: Hessian spectral densities of ResNet-18 trained on CIFAR-100 with SGD, SAM, and TLC-SAM. The dashed vertical [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

113 extracted references · 53 canonical work pages · 2 internal anchors

  1. [1]

    Advances in Neural Information Processing Systems , volume=

    Explicit eigenvalue regularization improves sharpness-aware minimization , author=. Advances in Neural Information Processing Systems , volume=

  2. [2]

    Advances in Neural Information Processing Systems , volume=

    Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach , author=. Advances in Neural Information Processing Systems , volume=

  3. [3]

    Advances in Neural Information Processing Systems , volume=

    On linear stability of sgd and input-smoothness of neural networks , author=. Advances in Neural Information Processing Systems , volume=

  4. [4]

    Advances in Neural Information Processing Systems , volume=

    How sgd selects the global minima in over-parameterized learning: A dynamical stability perspective , author=. Advances in Neural Information Processing Systems , volume=

  5. [5]

    Logarithmic landscape and power-law escape rate of SGD , author=

  6. [6]

    Journal of Machine Learning Research , volume=

    Sharpness-aware minimization and the edge of stability , author=. Journal of Machine Learning Research , volume=

  7. [7]

    Advances in Neural Information Processing Systems , volume=

    The crucial role of normalization in sharpness-aware minimization , author=. Advances in Neural Information Processing Systems , volume=

  8. [8]

    Advances in Neural Information Processing Systems , volume=

    Sharpness minimization algorithms do not only minimize sharpness to achieve better generalization , author=. Advances in Neural Information Processing Systems , volume=

  9. [9]

    Journal of Machine Learning Research , volume=

    The dynamics of sharpness-aware minimization: Bouncing across ravines and drifting towards wide minima , author=. Journal of Machine Learning Research , volume=

  10. [10]

    arXiv preprint arXiv:2302.09693 , year=

    msam: Micro-batch-averaged sharpness-aware minimization , author=. arXiv preprint arXiv:2302.09693 , year=

    Pith/arXiv arXiv

  11. [11]

    Stability Analysis of Sharpness-Aware Minimization

    Stability analysis of sharpness-aware minimization , author=. arXiv preprint arXiv:2301.06308 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  12. [12]

    International Conference on Machine Learning , pages=

    An sde for modeling sam: Theory and insights , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  13. [13]

    The effects of overparameterization on sharpness-aware minimization: An empirical and theoretical analysis , author=

  14. [14]

    Advances in Neural Information Processing Systems , volume=

    Practical sharpness-aware minimization cannot converge all the way to optima , author=. Advances in Neural Information Processing Systems , volume=

  15. [15]

    The Eleventh International Conference on Learning Representations , year=

    The Implicit Bias of Minima Stability in Multivariate Shallow ReLU Networks , author=. The Eleventh International Conference on Learning Representations , year=

  16. [16]

    Advances in Neural Information Processing Systems , volume=

    The implicit bias of minima stability: A view from function space , author=. Advances in Neural Information Processing Systems , volume=

  17. [17]

    The Eleventh International Conference on Learning Representations , year=

    How sharpness-aware minimization minimizes sharpness? , author=. The Eleventh International Conference on Learning Representations , year=

  18. [18]

    Advances in Neural Information Processing Systems , volume=

    Label noise sgd provably prefers flat global minimizers , author=. Advances in Neural Information Processing Systems , volume=

  19. [19]

    The Thirteenth International Conference on Learning Representations , year=

    Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late In Training , author=. The Thirteenth International Conference on Learning Representations , year=

  20. [20]

    Advances in Neural Information Processing Systems , volume=

    Gradient descent on two-layer nets: Margin maximization and simplicity bias , author=. Advances in Neural Information Processing Systems , volume=

  21. [21]

    arXiv preprint arXiv:2403.05529 , year=

    Computational-statistical gaps in gaussian single-index models , author=. arXiv preprint arXiv:2403.05529 , year=

    Pith/arXiv arXiv

  22. [22]

    Advances in Neural Information Processing Systems , volume=

    The alignment property of SGD noise and how it helps select flat minima: A stability analysis , author=. Advances in Neural Information Processing Systems , volume=

  23. [23]

    International Conference on Learning Representations , year=

    Gradient Descent Provably Optimizes Over-parameterized Neural Networks , author=. International Conference on Learning Representations , year=

  24. [24]

    Advances in neural information processing systems , volume=

    Neural tangent kernel: Convergence and generalization in neural networks , author=. Advances in neural information processing systems , volume=

  25. [25]

    Advances in neural information processing systems , volume=

    Implicit regularization in matrix factorization , author=. Advances in neural information processing systems , volume=

  26. [26]

    2009 IEEE conference on computer vision and pattern recognition , pages=

    Imagenet: A large-scale hierarchical image database , author=. 2009 IEEE conference on computer vision and pattern recognition , pages=. 2009 , organization=

    2009

  27. [27]

    IEEE Transactions on Computational Social Systems , year=

    Adaptive Sharpness-Aware Minimization for Adversarial Domain Generalization , author=. IEEE Transactions on Computational Social Systems , year=

  28. [28]

    Remote Sensing , volume=

    Implicit Sharpness-Aware Minimization for Domain Generalization , author=. Remote Sensing , volume=. 2024 , publisher=

    2024

  29. [29]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Imbsam: A closer look at sharpness-aware minimization in class-imbalanced recognition , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  30. [30]

    IEEE transactions on neural networks and learning systems , volume=

    Why resnet works? residuals generalize , author=. IEEE transactions on neural networks and learning systems , volume=. 2020 , publisher=

    2020

  31. [31]

    The Thirteenth International Conference on Learning Representations , year=

    Sharpness-Aware Minimization: General Analysis and Improved Rates , author=. The Thirteenth International Conference on Learning Representations , year=

  32. [32]

    Advances in Neural Information Processing Systems , volume=

    Enhancing sharpness-aware optimization through variance suppression , author=. Advances in Neural Information Processing Systems , volume=

  33. [33]

    Advances in Neural Information Processing Systems , volume=

    Surprising instabilities in training deep networks and a theoretical analysis , author=. Advances in Neural Information Processing Systems , volume=

  34. [34]

    Proceedings of the European conference on computer vision (ECCV) , pages=

    Encoder-decoder with atrous separable convolution for semantic image segmentation , author=. Proceedings of the European conference on computer vision (ECCV) , pages=

  35. [35]

    , author=

    Yolov5 release v7.0. , author=

  36. [36]

    2011 international conference on computer vision , pages=

    Semantic contours from inverse detectors , author=. 2011 international conference on computer vision , pages=. 2011 , organization=

    2011

  37. [37]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Pyramid scene parsing network , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

  38. [38]

    International journal of computer vision , volume=

    The pascal visual object classes (voc) challenge , author=. International journal of computer vision , volume=. 2010 , publisher=

    2010

  39. [39]

    Proceedings of the Conference on computer Vision and Pattern Recognition (CVPR) , pages=

    Cats and dogs , author=. Proceedings of the Conference on computer Vision and Pattern Recognition (CVPR) , pages=. 2012 , organization=

    work page 2012

  40. [40]

    Proceedings of the IEEE international conference on computer vision workshops , pages=

    3d object representations for fine-grained categorization , author=. Proceedings of the IEEE international conference on computer vision workshops , pages=

  41. [41]

    2008 Sixth Indian conference on computer vision, graphics & image processing , pages=

    Automated flower classification over a large number of classes , author=. 2008 Sixth Indian conference on computer vision, graphics & image processing , pages=. 2008 , organization=

    2008

  42. [42]

    arXiv preprint arXiv:2002.03495 , year=

    A diffusion theory for deep learning dynamics: Stochastic gradient descent exponentially favors flat minima , author=. arXiv preprint arXiv:2002.03495 , year=

    Pith/arXiv arXiv 2002

  43. [43]

    Advances in neural information processing systems , volume=

    The marginal value of adaptive gradient methods in machine learning , author=. Advances in neural information processing systems , volume=

  44. [44]

    Proceedings of the Conference on computer Vision and Pattern Recognition (CVPR) , pages=

    2d human pose estimation: New benchmark and state of the art analysis , author=. Proceedings of the Conference on computer Vision and Pattern Recognition (CVPR) , pages=

    work page

  45. [45]

    Proceedings of the IEEE/CVF international conference on computer vision (ICCV) , pages=

    O2u-net: A simple noisy label detection approach for deep neural networks , author=. Proceedings of the IEEE/CVF international conference on computer vision (ICCV) , pages=

    work page

  46. [46]

    2020 IEEE international conference on big data (Big data) , pages=

    Pyhessian: Neural networks through the lens of the hessian , author=. 2020 IEEE international conference on big data (Big data) , pages=. 2020 , organization=

    2020

  47. [47]

    I Can't Believe It's Not Better! - Understanding Deep Learning Through Empirical Falsification

    On the maximum hessian eigenvalue and generalization , author=. Proceedings on "I Can't Believe It's Not Better! - Understanding Deep Learning Through Empirical Falsification" at NeurIPS 2022 Workshops , pages=. 2023 , organization=

    work page 2022

  48. [48]

    Proceedings of the International Conference on Machine Learning (ICML) , pages=

    Fisher sam: Information geometry and sharpness aware minimisation , author=. Proceedings of the International Conference on Machine Learning (ICML) , pages=. 2022 , organization=

    work page 2022

  49. [49]

    arXiv preprint arXiv:2401.12033 , year=

    Momentum-SAM: Sharpness Aware Minimization without Computational Overhead , author=. arXiv preprint arXiv:2401.12033 , year=

    work page arXiv

  50. [50]

    arXiv preprint arXiv:2110.08529 , year=

    Sharpness-aware minimization improves language model generalization , author=. arXiv preprint arXiv:2110.08529 , year=

    Pith/arXiv arXiv

  51. [51]

    arXiv preprint arXiv:2203.09962 , year=

    Randomized sharpness-aware training for boosting computational efficiency in deep learning , author=. arXiv preprint arXiv:2203.09962 , year=

    Pith/arXiv arXiv

  52. [52]

    arXiv preprint arXiv:2210.12864 , year=

    K-sam: Sharpness-aware minimization at the speed of sgd , author=. arXiv preprint arXiv:2210.12864 , year=

    Pith/arXiv arXiv

  53. [53]

    Advances in Neural Information Processing Systems , volume=

    Deep learning on a data diet: Finding important examples early in training , author=. Advances in Neural Information Processing Systems , volume=

    work page

  54. [54]

    Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR) , pages=

    Friendly sharpness-aware minimization , author=. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR) , pages=

    work page

  55. [55]

    FedGAMMA: Federated Learning With Global Sharpness-Aware Minimization , year=

    Dai, Rong and Yang, Xun and Sun, Yan and Shen, Li and Tian, Xinmei and Wang, Meng and Zhang, Yongdong , journal=. FedGAMMA: Federated Learning With Global Sharpness-Aware Minimization , year=

    work page

  56. [56]

    SALR: Sharpness-Aware Learning Rate Scheduler for Improved Generalization , year=

    Yue, Xubo and Nouiehed, Maher and Kontar, Raed Al , journal=. SALR: Sharpness-Aware Learning Rate Scheduler for Improved Generalization , year=

    work page

  57. [57]

    OpenMMLab Pose Estimation Toolbox and Benchmark , author=

  58. [58]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Aggregated residual transformations for deep neural networks , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

  59. [59]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Human pose regression with residual log-likelihood estimation , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    work page

  60. [60]

    arXiv preprint arXiv:1812.05159 , year=

    An empirical study of example forgetting during deep neural network learning , author=. arXiv preprint arXiv:1812.05159 , year=

    Pith/arXiv arXiv

  61. [61]

    Proceedings of the Conference on European Conference on Computer Vision (ECCV) , pages=

    Simcc: A simple coordinate classification perspective for human pose estimation , author=. Proceedings of the Conference on European Conference on Computer Vision (ECCV) , pages=. 2022 , organization=

    work page 2022

  62. [62]

    arXiv preprint arXiv:1707.08819 , year=

    A downsampled variant of imagenet as an alternative to the cifar datasets , author=. arXiv preprint arXiv:1707.08819 , year=

    Pith/arXiv arXiv

  63. [63]

    arXiv preprint arXiv:2111.12621 , year=

    Accelerating deep learning with dynamic data pruning , author=. arXiv preprint arXiv:2111.12621 , year=

    Pith/arXiv arXiv

  64. [64]

    Effective Gradient Sample Size via Variation Estimation for Accelerating Sharpness aware Minimization

    Effective Gradient Sample Size via Variation Estimation for Accelerating Sharpness aware Minimization , author=. arXiv preprint arXiv:2403.08821 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  65. [65]

    Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR) , pages=

    Quantization and training of neural networks for efficient integer-arithmetic-only inference , author=. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR) , pages=

    work page

  66. [66]

    Proceedings of the International Conference on Learning Representations (ICLR) , year=

    InfoBatch: Lossless Training Speed Up by Unbiased Dynamic Data Pruning , author=. Proceedings of the International Conference on Learning Representations (ICLR) , year=

    work page

  67. [67]

    CS 231N , volume=

    Tiny imagenet visual recognition challenge , author=. CS 231N , volume=

  68. [68]

    Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR) , pages=

    Gradient norm aware minimization seeks first-order flatness and improves generalization , author=. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR) , pages=

    work page

  69. [69]

    Proceedings of the International Conference on Computer Vision (ICCV) , pages=

    Flatness-Aware Minimization for Domain Generalization , author=. Proceedings of the International Conference on Computer Vision (ICCV) , pages=

    work page

  70. [70]

    arXiv preprint arXiv:1708.04552 , year=

    Improved regularization of convolutional neural networks with cutout , author=. arXiv preprint arXiv:1708.04552 , year=

    Pith/arXiv arXiv

  71. [71]

    Proceedings of the International Conference on Learning Representations (ICLR) , year=

    An Adaptive Policy to Employ Sharpness-Aware Minimization , author=. Proceedings of the International Conference on Learning Representations (ICLR) , year=

    work page

  72. [72]

    Advances in Neural Information Processing Systems (NeurIPS) , volume=

    Sharpness-aware training for free , author=. Advances in Neural Information Processing Systems (NeurIPS) , volume=

    work page

  73. [73]

    Proceedings of the International Conference on Machine Learning (ICML) , pages=

    Unique properties of flat minima in deep networks , author=. Proceedings of the International Conference on Machine Learning (ICML) , pages=. 2020 , organization=

    work page 2020

  74. [74]

    Proceedings of the International Conference on Machine Learning (ICML) , pages=

    Asam: Adaptive sharpness-aware minimization for scale-invariant learning of deep neural networks , author=. Proceedings of the International Conference on Machine Learning (ICML) , pages=. 2021 , organization=

    work page 2021

  75. [75]

    Advances in Neural Information Processing Systems (NeurIPS) , volume=

    Swad: Domain generalization by seeking flat minima , author=. Advances in Neural Information Processing Systems (NeurIPS) , volume=

    work page

  76. [76]

    Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR) , pages=

    Towards efficient and scalable sharpness-aware minimization , author=. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR) , pages=

    work page

  77. [77]

    Proceedings of the International Conference on Learning Representations (ICLR) , year=

    Efficient sharpness-aware minimization for improved training of neural networks , author=. Proceedings of the International Conference on Learning Representations (ICLR) , year=

    work page

  78. [78]

    Advances in Neural Information Processing Systems (NeurIPS) , volume=

    Lookahead optimizer: k steps forward, 1 step back , author=. Advances in Neural Information Processing Systems (NeurIPS) , volume=

    work page

  79. [79]

    Advances in Neural Information Processing Systems (NeurIPS) , volume=

    Exploring generalization in deep learning , author=. Advances in Neural Information Processing Systems (NeurIPS) , volume=

    work page

  80. [80]

    Proceedings of the International Conference on Learning Representations (ICLR) , year=

    Sharpness-aware minimization for efficiently improving generalization , author=. Proceedings of the International Conference on Learning Representations (ICLR) , year=

    work page

Showing first 80 references.

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.