ResQ quantizes LLMs to 4-bit with a low-rank 8-bit residual subspace chosen by PCA, beating prior 4-bit methods on perplexity and accuracy.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residuals
ResQ quantizes LLMs to 4-bit with a low-rank 8-bit residual subspace chosen by PCA, beating prior 4-bit methods on perplexity and accuracy.