Pith. sign in

REVIEW 5 cited by

K-SAM: Sharpness-Aware Minimization at the Speed of SGD

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2210.12864 v1 pith:5OWRRVZY submitted 2022-10-23 cs.LG cs.CV

Renkun Ni , Ping-yeh Chiang , Jonas Geiping , Micah Goldblum , Andrew Gordon Wilson , Tom Goldstein This is my paper · ORCID
classification cs.LGcs.CV
keywords challengecomputationalcostk-samminimizationsharpness-awarevanillaaccuracy
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Sharpness-Aware Minimization (SAM) has recently emerged as a robust technique for improving the accuracy of deep neural networks. However, SAM incurs a high computational cost in practice, requiring up to twice as much computation as vanilla SGD. The computational challenge posed by SAM arises because each iteration requires both ascent and descent steps and thus double the gradient computations. To address this challenge, we propose to compute gradients in both stages of SAM on only the top-k samples with highest loss. K-SAM is simple and extremely easy-to-implement while providing significant generalization boosts over vanilla SGD at little to no additional cost.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. On the Implicit Flatness Bias of Sharpness-Aware Minimization: A Linear Stability Analysis with Quantitative Hyperparameter Bounds

    cs.LG 2026-08 reject novelty 6.0 of 10

    SAM's largest Hessian eigenvalue is bounded by the cube root of bGamma/(2*rho*eta^2), so larger radius, smaller batch, or larger learning rate restrict linearly stable minima to flatter regions.

  2. Towards Understanding The Calibration Benefits of Sharpness-Aware Minimization

    cs.LG 2025-05 reject novelty 6.0 of 10

    SAM's calibration benefit is attributed to implicit entropy maximization, but the proof rests on an unstated gradient-norm assumption; CSAM shows further ECE reductions.

  3. Gradient-Energy Guided Block-Wise Perturbations for Sharpness-Aware Minimization

    cs.LG 2026-07 conditional novelty 5.0 of 10

    GEAR-SAM re-allocates SAM's fixed perturbation radius across network blocks in proportion to an EMA of squared block-gradient norms, improving generalization on CIFAR, transfer, and label-noise benchmarks.

  4. Towards Understanding the Role of Sharpness-Aware Minimization Algorithms for Out-of-Distribution Generalization

    cs.LG 2024-12 conditional novelty 5.0 of 10

    SAM and several variants improve average OOD and gradual domain adaptation accuracy over Adam on four small benchmarks, but the new theoretical bounds match rather than improve on prior rates.

  5. Domain-Generalization to Improve Learning in Meta-Learning Algorithms

    cs.LG 2025-08 reject novelty 4.0 of 10

    DGS-MAML layers gradient matching onto SharpMAML and claims O(1/T) convergence and tighter PAC-Bayes bounds, but the displayed theorems give O(1/sqrt T) under the paper's own parameter choices.

Pith tools