Power-of-two weight quantization applied post-training to a 124M GPT-2 model degrades cross-entropy from 3.17 to about 4.1-4.5 at 4-6 bits, while promising memory and bit-shift savings.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
eess.SP 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Power-of-Two (PoT) Weights in Large Language Models (LLMs)
Power-of-two weight quantization applied post-training to a 124M GPT-2 model degrades cross-entropy from 3.17 to about 4.1-4.5 at 4-6 bits, while promising memory and bit-shift savings.