Pith. sign in

REVIEW 2 major objections 3 minor 1 cited by

Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels

T0 review · 2 major / 3 minor · reviewed 2026-05-19 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Quantization at 3 bits causes 6-21% of unbiased LLM items to develop new stereotypes while perplexity barely changes.

desk verdict Low-bit quantization introduces new stereotypical responses on BBQ items that perplexity misses, but the drop in 'unknown' answers could reflect general degradation more than targeted bias. read the letter →

arxiv 2605.15208 v1 pith:ZT36ZURY submitted 2026-05-02 cs.LG cs.AI

Plawan Kumar Rath , Rahul Maliakkal This is my paper · ORCID
classification cs.LGcs.AI
keywords quantizationbiasemergencelargelanguagemodelsmodelcompressionfairnessevaluationBBQbenchmarkprecisionlevels
checked against Cost.FunctionalEquationFoundation.ArithmeticFromLogic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tests whether post-training quantization preserves model behavior beyond standard quality measures. It tracks responses item by item on the BBQ benchmark as precision drops from BF16 to 3 bits in three instruction-tuned models. Results show a steady rise in new stereotypical answers that follows a dose-response curve, confirmed statistically, even though aggregate metrics like perplexity remain almost flat. The work demonstrates that fairness failures can appear at precision levels still considered safe by current evaluation standards. This gap matters because quantized models are widely deployed on edge devices where bias can affect real decisions.

What carries the argument

Item-level bias tracking on 12,148 BBQ items across five precision levels and three models, analyzed with logistic regression for dose-response.

What would settle it

Repeating the controlled runs and finding no statistically significant rise in stereotypical answers at 3-bit or 4-bit precision relative to BF16.

Watch

Extended reading notes

Core claim

3-bit quantization causes 6-21% of previously unbiased BBQ items to develop new stereotypical behaviors following a clear dose-response pattern, with models' willingness to select unknown answers declining by 17.4%; these item-level shifts remain invisible to perplexity, which rises less than 0.5% at 8-bit and under 3% at 4-bit across models.

Load-bearing premise

Changes in responses to BBQ items at lower bit widths reflect genuine emergence of stereotypical bias rather than random variation, model degradation, or evaluation noise.

Editorial extensions

If this is right

  • Even 4-bit models already show new bias in 2.5-5.6% of items despite minimal perplexity shift.
  • Models become less willing to answer unknown, reducing by 17.4% at lowest precision.
  • Aggregate quality metrics systematically miss fairness degradation during compression.
  • Deployment of quantized models requires explicit bias testing before release.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Compression pipelines for production use may need separate bias audits at each precision step.
  • Other compression methods such as pruning could produce similar hidden fairness shifts.
  • Post-quantization calibration focused on uncertainty and neutrality might reduce the observed bias increase.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The paper claims that post-training quantization of instruction-tuned LLMs (Qwen2.5-7B, Mistral-7B, Phi-3.5-mini) at precisions from BF16 to 3-bit induces new stereotypical biases on the BBQ benchmark. Across 12,148 items and 5 seeds (911,100 inferences), 6-21% of previously unbiased items develop stereotypical responses at 3-bit following a logistic-regression-confirmed dose-response, accompanied by a 17.4% drop in 'unknown' selections; these shifts are invisible to perplexity, which rises <0.5% at 8-bit and <3% at 4-bit.

Significance. If the central interpretation holds, the work is significant for demonstrating that standard aggregate metrics miss fairness-critical degradation under compression. Strengths include the multi-model, multi-precision design, large item count with statistical controls, explicit dose-response modeling via logistic regression, and the scale of controlled comparisons that allow distinguishing gradual from threshold effects.

major comments (2)
  1. [§4.2] §4.2 (item-level shift analysis): The reported 6-21% emergence of new stereotypical behaviors is not accompanied by a breakdown showing whether anti-stereotypical answers increase at a comparable rate. Given the 17.4% decline in 'unknown' selections, a non-preferential default to one of the two named options would be expected to raise both stereotypical and anti-stereotypical counts roughly equally; without this disaggregation or a control for uniform abstention failure, the attribution to specific bias emergence (rather than general instruction-following degradation) is not yet established and directly affects the 'fairness-critical failures' conclusion.
  2. [Methods §2.1] Methods §2.1 and Appendix A: The quantization implementation details (library, calibration dataset, group size, and any post-quantization fine-tuning) are described at a high level only. Because the central claim concerns threshold-dependent safety failures at 4-bit and 3-bit, the absence of these parameters prevents independent verification that the observed item-level changes are reproducible rather than artifacts of a particular quantization recipe.
minor comments (3)
  1. [Abstract] Abstract: The 6-21% range should be disaggregated by model or reported with per-model confidence intervals to allow readers to assess consistency.
  2. [Figure 2] Figure 2 (dose-response plots): Add error bars or shaded regions reflecting the 5 random seeds so that the logistic regression fit can be visually assessed for robustness.
  3. [Table 1] Table 1: Clarify whether the 'previously unbiased' baseline is computed per model or pooled across models, as this affects the denominator for the 6-21% statistic.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive and detailed feedback. The comments raise important points about distinguishing bias emergence from general degradation and ensuring reproducibility. We address each major comment below and indicate the revisions planned for the next version of the manuscript.

read point-by-point responses

  1. Referee: [§4.2] §4.2 (item-level shift analysis): The reported 6-21% emergence of new stereotypical behaviors is not accompanied by a breakdown showing whether anti-stereotypical answers increase at a comparable rate. Given the 17.4% decline in 'unknown' selections, a non-preferential default to one of the two named options would be expected to raise both stereotypical and anti-stereotypical counts roughly equally; without this disaggregation or a control for uniform abstention failure, the attribution to specific bias emergence (rather than general instruction-following degradation) is not yet established and directly affects the 'fairness-critical failures' conclusion.

    Authors: We agree that a disaggregation of anti-stereotypical responses is needed to strengthen the attribution to specific bias rather than uniform degradation in instruction following or abstention. Our logistic regression analysis models the dose-response specifically for stereotypical outputs, but we acknowledge the current presentation does not explicitly compare rates of increase across categories. In the revised manuscript we will add this breakdown to §4.2, reporting the relative changes in stereotypical, anti-stereotypical, and unknown selections across precision levels together with statistical tests for differential effects. This addition directly addresses the concern and supports the fairness-critical interpretation. revision: yes

  2. Referee: [Methods §2.1] Methods §2.1 and Appendix A: The quantization implementation details (library, calibration dataset, group size, and any post-quantization fine-tuning) are described at a high level only. Because the central claim concerns threshold-dependent safety failures at 4-bit and 3-bit, the absence of these parameters prevents independent verification that the observed item-level changes are reproducible rather than artifacts of a particular quantization recipe.

    Authors: The referee is correct that the current description is insufficient for full reproducibility. In the revised manuscript we will expand §2.1 and Appendix A to specify the quantization library and version, the exact calibration dataset, the group size parameter, and an explicit statement that no post-quantization fine-tuning was applied. These details will allow independent replication of the observed threshold effects at 4-bit and 3-bit. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: direct empirical measurements on external benchmark

full rationale

This paper reports controlled experiments evaluating three instruction-tuned LLMs at five precision levels on 12,148 BBQ items, directly counting item-level shifts in stereotypical responses and 'unknown' selections across random seeds. No derivations, first-principles results, equations, or predictions are claimed. Logistic regression serves only to statistically confirm the observed dose-response in the measured data rather than generating or substituting for primary results. No self-citations are load-bearing, no parameters are fitted to a subset and then presented as predictions, and all outcomes derive from comparisons against the external BBQ benchmark. The study is self-contained with independent empirical content and no reduction of claims to inputs by construction.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

The central claim rests on empirical observations and standard statistical practices rather than new free parameters, axioms, or invented entities.

assumptions (1)
  • domain assumption Standard assumptions of logistic regression and averaging across random seeds apply to the bias measurements and dose-response analysis.
    Invoked to establish the clear dose-response pattern across precision levels.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels." pith.science (2026). https://pith.science/paper/ZT36ZURY

@misc{pith2026260515208,
  author       = {Pith},
  title        = {Pith review of: Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZT36ZURY}},
  note         = {Machine review of arXiv:2605.15208}
}
read the original abstract

Large Language Models are routinely compressed via post-training quantization to reduce inference costs and memory footprint for cloud and edge deployment, yet the impact of this compression on model quality remains poorly understood. Existing studies typically compare only two conditions (full-precision vs. a single quantized variant), rely on aggregate bias metrics, and evaluate a single model family, making it impossible to distinguish gradual degradation from threshold-dependent safety failures. We conduct a controlled empirical study of three instruction-tuned models (Qwen2.5-7B, Mistral-7B, Phi-3.5-mini) at five precision levels (BF16 through 3-bit) on 12,148 BBQ bias benchmark items across 5 random seeds, totaling 911,100 inference records. Our results reveal that 3-bit quantization causes 6-21% of previously unbiased items to develop new stereotypical behaviors, following a clear dose-response pattern confirmed via logistic regression, while models' willingness to select "unknown" answers declines by 17.4%. Crucially, these item-level changes are invisible to standard quality metrics: perplexity increases by less than 0.5% at 8-bit and under 3% at 4-bit across all three models, yet 2.5-5.6% of items already develop new biases at 4-bit. These findings demonstrate that aggregate evaluation metrics systematically miss fairness-critical degradation, underscoring the need for quality-aware compression protocols that explicitly test for bias emergence before deployment.

Figures

Figures reproduced from arXiv: 2605.15208 by the authors.

Figure 2
Figure 2. Relative error in per-item SRS by bit-width. The monotonic increase [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Unknown Selection Rate (USR) decline under compression. All [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Comparison of effect sizes (Cohen’s h) between all items and latent-bias items (SRS ≥ 0.2 at BF16). The latent-bias subset shows medium-to-large effects, confirming that population-level metrics understate compression’s true impact. analysis pipeline [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. QuantiBias: Benchmarking Quantization-Induced Bias in LLMs

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Quantization leaves refusal and multiple-choice bias checks flat while open-ended stereotype endorsement remains high (~24–27% under an independent judge), a gap standard safety evaluations miss.

Reference graph

Works this paper leans on

25 extracted references · 25 canonical work pages · cited by 1 Pith paper

  1. [1]

    Large Language Models: A Survey

    S. Minaee, T. Mikolov, N. Nikzad, M. Chenaghlu, R. Socher, X. Am- atriain, and J. Gao, “Large language models: A survey,”arXiv preprint arXiv:2402.06196, 2024

    work page Pith review arXiv 2024

  2. [2]

    A survey of post-training scaling in large language models,

    Y . Xuet al., “A survey of post-training scaling in large language models,” inProc. 63rd Annu. Meeting Assoc. Comput. Linguistics (ACL), 2025

    work page 2025

  3. [3]

    LLMCBench: Benchmarking large language model com- pression for efficient deployment,

    J. Xuet al., “LLMCBench: Benchmarking large language model com- pression for efficient deployment,” inAdvances in Neural Information Processing Systems 37 (NeurIPS), 2024

    work page 2024

  4. [4]

    A survey of model compression techniques: Past, present, and future,

    Z. Liaoet al., “A survey of model compression techniques: Past, present, and future,”Frontiers in Robotics and AI, vol. 12, p. 1518965, 2025

    work page 2025

  5. [5]

    A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions

    L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, and T. Liu, “A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions,” arXiv preprint arXiv:2311.05232, 2023

    work page Pith review arXiv 2023

  6. [6]

    A survey on hallucination in large language and foundation models,

    V . Rawte, A. Sheth, and A. Das, “A survey on hallucination in large language and foundation models,”Preprints, 202504.1236, 2025

    work page arXiv 2025

  7. [7]

    Winning Big with Small Models: Knowledge Distillation vs

    A. Thakuret al., “Winning big with small models: Knowledge distil- lation vs. self-training for reducing hallucination in QA agents,”arXiv preprint arXiv:2502.19545, 2025

    work page arXiv 2025

  8. [8]

    Bias and fairness in large language models: A survey,

    I. O. Gallegos, R. A. Rossi, J. Barber, M. M. Tanjim, S. Kim, F. Der- noncourt, T. Yu, R. Zhang, and N. K. Ahmed, “Bias and fairness in large language models: A survey,”Computational Linguistics, vol. 50, no. 3, pp. 1097–1179, 2024

    work page 2024

Show all 25 references
  1. [9]

    Beyond perplexity: Multi-dimensional safety evaluation of LLM compression,

    Y . Renet al., “Beyond perplexity: Multi-dimensional safety evaluation of LLM compression,” inFindings of the Assoc. Comput. Linguistics: EMNLP, 2024

    2024

  2. [10]

    A survey on out-of-distribution evalu- ation of neural NLP models,

    Z. Yuan, Y . Chen, and M. Xia, “A survey on out-of-distribution evalu- ation of neural NLP models,”arXiv preprint arXiv:2306.15261, 2023

    2023

  3. [11]

    A survey on large language model benchmarks,

    Q. Liuet al., “A survey on large language model benchmarks,”arXiv preprint arXiv:2508.15361, 2025

    2025

  4. [12]

    Ro- bust lottery tickets for pre-trained language models,

    H. Zheng, Q. Peng, Y . Yang, Z. Chen, Z. Zhu, and S. Poria, “Ro- bust lottery tickets for pre-trained language models,”arXiv preprint arXiv:2211.03013, 2022

    2022

  5. [13]

    Understanding and over- coming the challenges of efficient transformer quantization,

    Y . Bondarenko, M. Nagel, and T. Blankevoort, “Understanding and over- coming the challenges of efficient transformer quantization,” inProc. Conf. Empirical Methods in Natural Language Processing (EMNLP), 2021, pp. 7947–7969

    2021

  6. [14]

    The compression techniques applied on deep learning model,

    F. Ong, “The compression techniques applied on deep learning model,” Highlights in Science, Engineering and Technology, vol. 17, pp. 208– 213, 2022

    2022

  7. [15]

    The dawn after the dark: An empirical study on factuality hallucination in large language models,

    J. Li, J. Chen, R. Ren, X. Cheng, W. X. Zhao, J.-Y . Nie, and J.-R. Wen, “The dawn after the dark: An empirical study on factuality hallucination in large language models,”arXiv preprint arXiv:2401.03205, 2024

    2024

  8. [16]

    Understanding the effect of model compression on social bias in large language models,

    G. Gonc ¸alves and E. Strubell, “Understanding the effect of model compression on social bias in large language models,” inProc. Conf. Empirical Methods in Natural Language Processing (EMNLP), 2023

    2023

  9. [17]

    How does quantization affect multilingual LLMs?

    K. Marchisioet al., “How does quantization affect multilingual LLMs?” arXiv preprint arXiv:2407.03211, 2024

    2024

  10. [18]

    Investigating the impact of quantization methods on the safety and reliability of large language models,

    Y . Linet al., “Investigating the impact of quantization methods on the safety and reliability of large language models,”arXiv preprint arXiv:2502.15799, 2025

    2025

  11. [19]

    What do compressed deep neural networks forget?

    S. Hooker, A. Courville, G. Clark, Y . Dauphin, and A. Frome, “What do compressed deep neural networks forget?”arXiv preprint arXiv:1911.05248, 2019

    1911

  12. [20]

    BBQ: A hand-built bias benchmark for question answering,

    A. Parrish, A. Chen, N. Nangia, V . Padmakumar, J. Phang, J. Thompson, P. M. Htut, and S. R. Bowman, “BBQ: A hand-built bias benchmark for question answering,” inFindings of the Assoc. Comput. Linguistics: ACL, 2022, pp. 2086–2105

    2022

  13. [21]

    MLX: An array framework for Apple Silicon,

    Apple Inc., “MLX: An array framework for Apple Silicon,” GitHub Repository, 2023. [Online]. Available: https://github.com/ml-explore/ mlx

    2023

  14. [22]

    Cohen,Statistical Power Analysis for the Behavioral Sciences, 2nd ed

    J. Cohen,Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Lawrence Erlbaum Associates, 1988

    1988

  15. [23]

    Uncertainty drives social bias changes in quantized large language models,

    S. Z. Hua, S. Lotfi, and I. Y . Chen, “Uncertainty drives social bias changes in quantized large language models,”arXiv preprint arXiv:2602.06181, 2026

    2026

  16. [24]

    Alignment-aware quantization for LLM safety,

    S. Wee, S. Kim, H. Kim, K. Hwang, and N. Kwak, “Alignment-aware quantization for LLM safety,”arXiv preprint arXiv:2511.07842, 2025

    2025

  17. [25]

    Accuracy is not all you need,

    S. Dutta, A. Pandey, S. Chattopadhyay, T. Sinha, and S. Chakraborty, “Accuracy is not all you need,”arXiv preprint arXiv:2407.09141, 2024

    2024

Pith tools

Reviewed May 19, 2026 · model on record in the stance chip above.