WinQ accelerates quantization-aware training up to 4x and improves sub-4-bit accuracy up to 8.8% by weight interpolation resets and noise-regularized gradients that increase Hessian eigenvalue magnitudes around saddle points.
Modulora: finetuning 2-bit llms on consumer gpus by integrating with modular quantizers
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
WinQ: Accelerating Quantization-Aware Training of Language Models Around Saddle Points
WinQ accelerates quantization-aware training up to 4x and improves sub-4-bit accuracy up to 8.8% by weight interpolation resets and noise-regularized gradients that increase Hessian eigenvalue magnitudes around saddle points.