Quantization usually causes modest information loss and reduced factual knowledge recall in LLMs, especially smaller ones, but BitSandBytes preserves performance best and occasional gains occur.
InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 28494–28513, Suzhou, China
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Through a Compressed Lens: Investigating The Impact of Quantization on Factual Knowledge Recall
Quantization usually causes modest information loss and reduced factual knowledge recall in LLMs, especially smaller ones, but BitSandBytes preserves performance best and occasional gains occur.