Quantization usually causes modest information loss and reduced factual knowledge recall in LLMs, especially smaller ones, but BitSandBytes preserves performance best and occasional gains occur.
InFindings of the Associa- tion for Computational Linguistics: ACL 2024, pages 12186–12215, Bangkok, Thailand
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Through a Compressed Lens: Investigating The Impact of Quantization on Factual Knowledge Recall
Quantization usually causes modest information loss and reduced factual knowledge recall in LLMs, especially smaller ones, but BitSandBytes preserves performance best and occasional gains occur.