3-bit quantization induces new stereotypical biases in 6-21% of previously unbiased BBQ items across three LLMs, undetected by perplexity increases under 3%, with models declining in 'unknown' responses by 17.4%.
Winning big with small models: Knowledge distil- lation vs. self-training for reducing hallucination in QA agents
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
years
2026 2representative citing papers
KD outperforms SFT for LLM post-training in low-data regimes but the advantage fades with abundant data unless the teacher is stronger; a two-stage strategy aids domain-specific low-resource cases.
citing papers explorer
-
Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels
3-bit quantization induces new stereotypical biases in 6-21% of previously unbiased BBQ items across three LLMs, undetected by perplexity increases under 3%, with models declining in 'unknown' responses by 17.4%.
-
Understanding Knowledge Distillation in Post-Training: When It Helps and When It Fails
KD outperforms SFT for LLM post-training in low-data regimes but the advantage fades with abundant data unless the teacher is stronger; a two-stage strategy aids domain-specific low-resource cases.