SAMPLe adds dual gradient constraints (ERM alignment plus full-batch orthogonality) to SAM-style prompt learning and raises harmonic-mean base-to-new accuracy across CoOp, CoCoOp, MaPLe, TCP and CoPrompt.
Sharpness-aware mini- mization improves language model generalization.arXiv preprint arXiv:2110.08529
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
abstract
The allure of superhuman-level capabilities has led to considerable interest in language models like GPT-3 and T5, wherein the research has, by and large, revolved around new model architectures, training tasks, and loss objectives, along with substantial engineering efforts to scale up model capacity and dataset size. Comparatively little work has been done to improve the generalization of these models through better optimization. In this work, we show that Sharpness-Aware Minimization (SAM), a recently proposed optimization procedure that encourages convergence to flatter minima, can substantially improve the generalization of language models without much computational overhead. We show that SAM is able to boost performance on SuperGLUE, GLUE, Web Questions, Natural Questions, Trivia QA, and TyDiQA, with particularly large gains when training data for these tasks is limited.
years
2026 2representative citing papers
ShaPO improves LLM safety robustness over standard preference optimization by enforcing worst-case objectives via selective geometry control at token and reward levels.
citing papers explorer
-
SAMPLe: SAM-based Optimizer for Prompt Learning in VLMs
SAMPLe adds dual gradient constraints (ERM alignment plus full-batch orthogonality) to SAM-style prompt learning and raises harmonic-mean base-to-new accuracy across CoOp, CoCoOp, MaPLe, TCP and CoPrompt.
-
Revisiting Robustness for LLM Safety Alignment via Selective Geometry Control
ShaPO improves LLM safety robustness over standard preference optimization by enforcing worst-case objectives via selective geometry control at token and reward levels.