Pith. sign in

REVIEW 4 major objections 6 minor 58 references

Fair-GPTQ: Bias-Aware Quantization for Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fair-GPTQ claims that adding a bias penalty to GPTQ's quantization objective produces 4-bit LLMs with lower stereotype scores than the FP16 models they are compressed from, while preserving at least 90% of zero-shot accuracy and retaining 4

desk verdict A promising fairness-aware quantization objective, but the paper's own pseudocode doesn't match the derived update, so the empirical claims are not yet anchored to the theory. read the letter →

arxiv 2509.15206 v3 pith:TSTDXU6C submitted 2025-09-18 cs.CL

Irina Proskurina , Guillaume Metzler , Julien Velcin This is my paper
classification cs.CL
keywords quantizationlargelanguagemodelsfairnessstereotypebiasGPTQpost-trainingmitigationgroup
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Fair-GPTQ claims that the bias amplification observed in quantized language models is not a fixed cost of compression: by adding a bias penalty to the quantization objective, the rounding of weights can be steered to reduce stereotype generation. The method modifies GPTQ's optimization so that quantized weights actively minimize the gap in model behavior between stereotypical and anti-stereotypical sentence pairs, and it derives a closed-form update from the optimal brain surgeon framework. On OPT and Mistral models quantized to 4 bits, it reports lower CrowS-Pairs and StereoSet bias scores than both the half-precision models and GPTQ baselines, while preserving at least 90% of zero-shot accuracy and retaining 4-bit memory and speed benefits. This matters because debiasing is folded into compression rather than added as an expensive post-hoc step, making fairness-aware deployment of large models more practical.

What carries the argument

The bias-aware Hessian H = 2(X0X0ᵀ + X1X1ᵀ + α(X0−X1)(X0−X1)ᵀ) and the non-zero gradient J = 2αW(X0−X1)(X0−X1)ᵀ; the OBS solution yields a closed-form debiasing update W ← W − (H⁻¹H_biasWᵀ)ᵀ that is applied before the standard GPTQ quantization loop. Because the Hessian is block-diagonal across weight rows, the update is computed once per column and applied across all rows, keeping the per-layer cost the same as GPTQ.

What would settle it

Quantize a model with Fair-GPTQ using the same StereoSet pairs, then evaluate on a benchmark of newly authored minimal pairs that share no templates with StereoSet. If stereotype scores do not improve, the debiasing effect is calibration-overlap rather than a general property. Alternatively, probe layer-by-layer bias scores after applying the update only to lower layers to check whether unmodified upper layers re-amplify the difference.

Watch

Extended reading notes

Core claim

The central claim is that a fairness constraint can be embedded directly into the quantization objective. The paper starts from GPTQ's layer-wise weight-reconstruction loss and adds the term α ||W′(X0−X1)||², where X0 and X1 are paired texts that differ only in a protected-attribute token. Keeping the gradient of this term non-zero at the pretrained weights, it solves the resulting constrained OBS problem and obtains a weight update that separates into a debiasing correction applied before the GPTQ rounding loop. Experiments on OPT and Mistral show that this correction lowers stereotype likelihood on CrowS-Pairs, StereoSet, and BBQ relative to FP16 and GPTQ, with zero-shot accuracy staying w

Load-bearing premise

The method assumes that lower bias on the paired calibration sentences (StereoSet development pairs) transfers to lower stereotype rates on unrelated benchmarks like CrowS-Pairs and BBQ; if that transfer fails, the reported debiasing is an artifact of calibration-data overlap.

Editorial extensions

If this is right

  • Debiasing can be incorporated into a single quantization pass, increasing runtime by only about 20% compared to GPTQ, rather than requiring separate fine-tuning or inference-time interventions.
  • 4-bit quantized models can have lower stereotype scores than the half-precision models they are compressed from, showing that compression need not amplify bias if the rounding is fairness-aware.
  • The attention output projection and the MLP output (down-projection) matrices carry the largest bias-correction signal, so targeting these matrices is sufficient for most of the debiasing effect.
  • Applying the fairness update to the lower layers yields the largest stereotype-score reductions, giving a cheap strategy when full-layer updates are infeasible.
  • The method matches or exceeds iterative null-space projection debiasing on racial stereotypes, while also delivering 4-bit memory and speed benefits.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The same paired-difference penalty could be ported to other weight-rounding schemes (e.g., round-to-nearest with error compensation), potentially generalizing fairness-aware compression beyond GPTQ-family methods.
  • Inference: The lower-layer effect hints that stereotype-relevant computation is concentrated in early layers in these transformers; if true, debiasing could be targeted even more cheaply by adjusting only the bottom few layers.
  • Inference: A direct stress test would be to calibrate on StereoSet but evaluate on a newly constructed minimal-pair benchmark; if the bias reduction disappears, the method's gains are tied to calibration-set overlap rather than a general mechanism.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Fair-GPTQ, a post-training quantization method that augments the GPTQ layer-wise reconstruction objective with a bias penalty α||W'(X0−X1)||², where X0 and X1 are paired stereotypical/anti-stereotypical inputs. The authors derive an OBS-style closed-form update (Proposition 1, Eq. 6) that accounts for a non-zero gradient at the pretrained weights, and they present Algorithm 1 to perform debiasing and quantization. Experiments on OPT and Mistral models at 4 bits report lower CrowS-Pairs, StereoSet, and BBQ stereotype scores than FP16 and GPTQ baselines, with some degradation in zero-shot accuracy and perplexity, and claim at least 90% of baseline zero-shot accuracy is preserved. The method is also compared with INLP, Self-Debias, and SentenceDebias, and a sanity check with unrelated calibration pairs is included.

Significance. If the claimed results hold, Fair-GPTQ would be a useful contribution: it is, to my knowledge, the first quantization scheme that explicitly builds a group-fairness term into the GPTQ objective, and the algebraic derivation of the OBS-like update with a non-zero gradient is clean and clearly presented. The paper also includes a meaningful sanity check (Appendix D.1), evaluates on external holdout benchmarks (CrowS-Pairs test, BBQ, SoFA), and reports scaling across OPT model sizes. These are strengths. However, the current manuscript has a load-bearing mismatch between the stated theoretical solution and Algorithm 1, and no code is provided, so the link between the derivation and the reported empirical gains is not established.

major comments (4)
  1. [§3.2, Algorithm 1, Eq. (4)–(6)] Algorithm 1 does not implement the optimization problem solved in Proposition 1. Eq. (4) defines H = X0X0^T + X1X1^T + αΔXΔX^T, and Eq. (6) requires H_w^{-1} = (H⊗I)^{-1} in both the debiasing correction and the quantization-compensation term. However, Algorithm 1 line 6 sets C ← Cholesky(H_acc^{-1})^T with H_acc = X0X0^T + X1X1^T, and lines 10–11 use this C for error compensation. The bias term αΔXΔX^T is therefore omitted from the compensation step. As written, the algorithm minimizes a different objective and does not solve Eq. (5). If the experiments actually use the full H in the loop, the pseudocode misdescribes the implementation; if they follow the pseudocode, the claim that experiments 'validate our theoretical solution' is unsupported. Without released code, this cannot be checked.
  2. [§3.2, Algorithm 1, lines 5–9; Eq. (6)] A second discrepancy: Eq. (6) is derived for the original weights w and their quantized values quant(w_q). In Algorithm 1, line 5 first applies the debiasing shift W ← W − (H^{-1}H_bias W^T)^T, and only then line 9 quantizes W:,j. The quantized values are therefore quant(w_q + δ_q), not quant(w_q), while the compensation term in Eq. (6) is expressed in terms of (w_q − quant(w_q)). Unless the derivation is reworked to account for the pre-shifted weights, the update used in the algorithm is not the one derived in Proposition 1.
  3. [Abstract and §5.1, Table 1] The claim that Fair-GPTQ 'preserves at least 90% of baseline accuracy on zero-shot benchmarks' is not supported by the reported results for all configurations. In Table 1, Mistral-7B with Fair-GPTQ_l achieves Cloze 62.87 vs FP16 78.29 (≈80.3%) and HSwag 47.11 vs 60.93 (≈77.3%). Since the lower-layer strategy is one of the paper's main findings, the abstract's unqualified 90% assertion is an overstatement. The claim should be restricted to the configurations where it holds, or revised.
  4. [§5.1, α selection; §4.2 calibration data] The fairness penalty in Eq. (1) is a proxy for the stereotype benchmarks, and α is selected on the development split of the same benchmark family used for calibration (StereoSet dev). The paper reports reductions on StereoSet test and CrowS-Pairs, which reduces but does not eliminate the concern of selection on the evaluation axis. I would like to see either a sensitivity analysis across α values on holdout benchmarks or a confirmation that the reported qualitative conclusions are stable for a range of α rather than only for the tuned value.
minor comments (6)
  1. [§3.1, Eqs. (2)–(4)] The derivation of H and J is correct, but the notation J for both the matrix gradient and the Jacobian could be confusing. Also, Eq. (4) writes H = 2(X0X0^T + X1X1^T + αΔXΔX^T), while the text later refers to H_acc = X0X0^T + X1X1^T and H_bias = 2αΔXΔX^T; the factor 2 placement is inconsistent between the main text and Algorithm 1. Please align the notation.
  2. [Appendix A, proof of Proposition 1] In the expression for L(λ), the term λ[−e_q^T H^{-1}(λe_q + H_w) + ...] appears to contain a typo: H_w should be J_w. The final result is correct, but the intermediate expression should be fixed.
  3. [§4.2 and Table 1] The α grid is described as 'between 0.1 and 1 in steps of 0.5', which is ambiguous (0.1, 0.6, 1.1? or 0.1, 0.5, 1.0?). Please specify the exact values tested.
  4. [Appendix D.4, Table 9] On SoFA, Fair-GPTQ does not uniformly reduce bias: for OPT-6.7B, Fair-GPTQ_ALL increases race bias (0.101 vs GPTQ-SS 0.092) and disability bias (0.048 vs 0.044), and Fair-GPTQ_l increases disability bias to 0.057. The paper should comment on this rather than only highlighting improvements on other benchmarks.
  5. [§5.1, Figure 4] The y-axis label 'ΔW / W' and the logarithmic scale are not fully defined. Please specify whether ΔW is the Frobenius norm of the debiasing update, and how the relative magnitudes are averaged over layers.
  6. [General] The paper states that code will be released upon acceptance. Given the discrepancy between Algorithm 1 and Eq. (6), providing the implementation or pseudocode that exactly matches the experiments is essential for reproducibility.

Circularity Check

1 steps flagged · score 2.0 of 10

Bias penalty is a fitted proxy for the reported stereotype benchmarks, so the benchmark reductions are partly a check of the calibration objective on nearby metrics; the derivation itself is not circular.

  1. fitted input called prediction [Section 3.1, Eq. (1); Section 4.2, Calibration Data]
    "To make the quantization step sensitive to potential stereotypes, we introduce a bias penalty that measures how much the quantized model changes the representation gap between the stereotyped (X0) and anti-stereotyped (X1) inputs. Formally, this can be restated as: ... We use the development subset of STEREO SET as calibration data, because its human-annotated sentence pairs align with our framework, which requires paired inputs (X in Eq. (1))."

    The added penalty α||W′(X0−X1)||² in Eq. (1) is a smooth surrogate for the stereotyped/anti-stereotyped likelihood comparisons that CrowS-Pairs and StereoSet score, and α is selected on the StereoSet development split. Reporting CP/SS reductions afterward therefore partly confirms that the calibration objective transfers to closely related metrics on the same benchmark family, rather than independently testing the OBS derivation. Because the evaluation also uses external test splits and additional benchmarks (BBQ, SoFA), the prediction is not statistically forced; this is partial/evaluative circularity, not definitional.

full rationale

The derivation chain is internally consistent: Eq. (1) defines a new quantization objective, Proposition 1 solves the resulting constrained quadratic problem, and the proof in Appendix A is a standard Lagrangian derivation. No load-bearing step imports the reported benchmarks into the derivation, and no reliance on self-citations or author-specific uniqueness theorems appears. The only circularity-adjacent issue is that the fairness penalty is a calibrated proxy for the stereotype-pair metrics: calibration pairs come from StereoSet dev, α is tuned on that dev split, and the reported CP/SS improvements are measured on closely related minimal-pair benchmarks. This is a legitimate but partially expected result, not a forced equivalence. The availability of holdout and external evaluations (CrowS-Pairs test, BBQ, SoFA, zero-shot accuracy) provides independent empirical content, so the central claim is measured rather than constructed. Separately, I note a potential implementation inconsistency: Algorithm 1 builds the quantization compensation from H_acc^{-1} (line 6) while Eq. (6) requires H^{-1} including the bias term, and line 9 quantizes weights after the debiasing shift. This is a correctness/reproducibility risk, not circularity, and is not reflected in the score.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central method rests on one tuned hyperparameter (alpha) and several domain assumptions about how pair-level output differences connect to benchmark stereotype scores. No new entities are introduced.

free parameters (1)
  • alpha = 0.1 for all-layer (ALL), 0.5 for lower/upper/UL layer subsets
    Strength of the fairness regularizer in Eq. (1); chosen in Section 5.1 by balancing debiasing effect with perplexity, making it a tuned hyperparameter rather than a fixed constant.
assumptions (5)
  • domain assumption Second-order Taylor approximation of Eq. (1) around the pretrained weights is accurate for the OBS weight update, despite a nonzero gradient.
    Invoked in Section 3.1, Eqs. (2)-(4); if the quadratic model is poor, the debiasing update W <- W - (H^{-1}H_bias W^T)^T may not minimize the stated objective.
  • standard math The Gram matrix H = 2(X0X0^T + X1X1^T + alpha * DeltaX DeltaX^T) is invertible, or safely regularized, so H^{-1} exists.
    Used in Eq. (4) and Algorithm 1 line 5; no explicit regularization is described.
  • domain assumption Bias can be operationalized as the difference in layer outputs for paired stereotypical/anti-stereotypical inputs, and minimizing this difference reduces downstream stereotyped generation.
    This is the core modeling choice in Eq. (1); it also determines the paired calibration format and the sanity-check experiment.
  • domain assumption StereoSet development pairs are representative of bias measured by CrowS-Pairs, StereoSet test, BBQ, and SoFA.
    Calibration uses 4,212 StereoSet dev pairs (Section 4.2); transfer to other benchmarks is assumed.
  • ad hoc to paper Debiasing only attention output projections and output FC/down-projection matrices is sufficient.
    Chosen in Section 4.2 based on residual-stream reasoning and prior work; supported in ablation (Table 10) but not derived.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fair-GPTQ: Bias-Aware Quantization for Large Language Models." pith.science (2026). https://pith.science/paper/TSTDXU6C

@misc{pith2026250915206,
  author       = {Pith},
  title        = {Pith review of: Fair-GPTQ: Bias-Aware Quantization for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TSTDXU6C}},
  note         = {Machine review of arXiv:2509.15206}
}
read the original abstract

The high memory demands of generative language models have drawn attention to quantization, which reduces memory usage by mapping model weights to lower-precision integers. However, recent empirical studies show that, while efficient, quantization can increase the likelihood of generating biased outputs and degrade performance on fairness benchmarks. In this work, we draw new links between quantization and model fairness by adding explicit group-fairness constraints to the quantization objective and introduce Fair-GPTQ, the first quantization method explicitly designed to reduce unfairness in large language models. The added constraints guide the learning of the rounding operation toward less-biased text generation for protected groups. Specifically, we focus on stereotype generation involving occupational bias and discriminatory language spanning gender, race, and religion. Fair-GPTQ has minimal impact on performance, preserving at least 90% of baseline accuracy on zero-shot benchmarks, reduces unfairness relative to a half-precision model, and retains the memory and speed benefits of 4-bit quantization.

Figures

Figures reproduced from arXiv: 2509.15206 by the authors.

Figure 1
Figure 1. Illustration of the proposed method Fair-GPTQ. Fair-GPTQ quantizes models using paired [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Bias scores across 6 categories for the quantized OPT-6.7B model evaluated on the BBQ [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. CrowS stereotype scores and perplexity for Fair-GPTQ in the [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Relative weight updates obtained using the proposed Fair-GPTQ method, compared with [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

58 extracted references · 2 canonical work pages

  1. [1]

    Pi QA : Reasoning about physical commonsense in natural language

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Pi QA : Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432--7439, 2020. URL https://api.semanticscholar.org/CorpusID:208290939

    2020

  2. [2]

    Man is to computer programmer as woman is to homemaker? debiasing word embeddings

    Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29, 2016

    2016

  3. [3]

    Shikha Bordia and Samuel R. Bowman. Identifying and reducing gender bias in word-level language models. In Sudipta Kar, Farah Nadeem, Laura Burdick, Greg Durrett, and Na-Rae Han, editors, Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Student Research Workshop , pages 7--15, Minneapolis,...

    doi:10.18653/v1/n19-3002 2019

  4. [4]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 0 1877--1901, 2020

    1901

  5. [5]

    Semantics derived automatically from language corpora contain human-like biases

    Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. Semantics derived automatically from language corpora contain human-like biases. Science, 356 0 (6334): 0 183--186, 2017

    2017

  6. [6]

    Quip: 2-bit quantization of large language models with guarantees

    Jerry Chee, Yaohui Cai, Volodymyr Kuleshov, and Christopher M De Sa. Quip: 2-bit quantization of large language models with guarantees. Advances in Neural Information Processing Systems, 36: 0 4396--4429, 2023

    2023

  7. [7]

    Think you have solved question answering? T ry ARC , the AI2 reasoning challenge

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? T ry ARC , the AI2 reasoning challenge. CoRR, abs/1803.05457, 2018. URL http://arxiv.org/abs/1803.05457

    arXiv 2018

  8. [8]

    Racial bias in hate speech and abusive language detection datasets

    Thomas Davidson, Debasmita Bhattacharya, and Ingmar Weber. Racial bias in hate speech and abusive language detection datasets. In Sarah T. Roberts, Joel Tetreault, Vinodkumar Prabhakaran, and Zeerak Waseem, editors, Proceedings of the Third Workshop on Abusive Language Online, pages 25--35, Florence, Italy, August 2019. Association for Computational Lingu...

    doi:10.18653/v1/w19-3504 2019

Show all 58 references
  1. [9]

    Gpt3.int8(): 8-bit matrix multiplication for transformers at scale

    Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. Gpt3.int8(): 8-bit matrix multiplication for transformers at scale. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, page...

    2022

  2. [10]

    Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale. Advances in neural information processing systems, 35: 0 30318--30332, 2022 b

    2022

  3. [11]

    A mathematical framework for transformer circuits

    Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dari...

    2021

  4. [12]

    From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair NLP models

    Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair NLP models. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Proceedings of th...

    2023 doi

  5. [13]

    Optimal brain compression: A framework for accurate post-training quantization and pruning

    Elias Frantar and Dan Alistarh. Optimal brain compression: A framework for accurate post-training quantization and pruning. Advances in Neural Information Processing Systems, 35: 0 4475--4488, 2022

    2022

  6. [14]

    Gptq: Accurate post-training quantization for generative pre-trained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323, 2022

    2022 arXiv

  7. [15]

    Marlin: Mixed-precision auto-regressive parallel inference on large language models

    Elias Frantar, Roberto L Castro, Jiale Chen, Torsten Hoefler, and Dan Alistarh. Marlin: Mixed-precision auto-regressive parallel inference on large language models. In Proceedings of the 30th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming, page...

    2025

  8. [16]

    Transformer feed-forward layers are key-value memories

    Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language P...

    2021 doi

  9. [17]

    Understanding the effect of model compression on social bias in large language models

    Gustavo Gon c alves and Emma Strubell. Understanding the effect of model compression on social bias in large language models. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 26...

    2023 doi

  10. [18]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  11. [19]

    Accelerate: Training and inference at scale made simple, efficient and adaptable

    Sylvain Gugger, Lysandre Debut, Thomas Wolf, Philipp Schmid, Zachary Mueller, Sourab Mangrulkar, Marc Sun, and Benjamin Bossan. Accelerate: Training and inference at scale made simple, efficient and adaptable. https://github.com/huggingface/accelerate, 2022

    2022

  12. [20]

    Optimal brain surgeon and general network pruning

    Babak Hassibi, David G Stork, and Gregory J Wolff. Optimal brain surgeon and general network pruning. In IEEE international conference on neural networks, pages 293--299. IEEE, 1993

    1993

  13. [21]

    Rethinking channel dimensions to isolate outliers for low-bit weight quantization of large language models

    Jung Hwan Heo, Jeonghoon Kim, Beomseok Kwon, Byeongwook Kim, Se Jung Kwon, and Dongsoo Lee. Rethinking channel dimensions to isolate outliers for low-bit weight quantization of large language models. arXiv preprint arXiv:2309.15531, 2023

    2023 arXiv

  14. [22]

    Accurate post training quantization with small calibration sets

    Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, and Daniel Soudry. Accurate post training quantization with small calibration sets. In International conference on machine learning, pages 4466--4475. PMLR, 2021

    2021

  15. [23]

    Compressing llms: The truth is rarely pure and never simple

    Ajay Jaiswal, Zhe Gan, Xianzhi Du, Bowen Zhang, Zhangyang Wang, and Yinfei Yang. Compressing llms: The truth is rarely pure and never simple. arXiv preprint arXiv:2310.01382, 2023

    2023 arXiv

  16. [24]

    Perplexity—a measure of the difficulty of speech recognition tasks

    Fred Jelinek, Robert L Mercer, Lalit R Bahl, and James K Baker. Perplexity—a measure of the difficulty of speech recognition tasks. The Journal of the Acoustical Society of America, 62 0 (S1): 0 S63--S63, 1977

    1977

  17. [25]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023 arXiv

  18. [26]

    Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeff Wu, and Dario Amodei

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeff Wu, and Dario Amodei. Scaling laws for neural language models. ArXiv, abs/2001.08361, 2020. URL https://api.semanticscholar.org/CorpusID:210861095

    2001 arXiv

  19. [27]

    Optimal brain damage

    Yann LeCun, John Denker, and Sara Solla. Optimal brain damage. Advances in neural information processing systems, 2, 1989

    1989

  20. [28]

    Owq: Outlier-aware weight quantization for efficient fine-tuning and inference of large language models

    Changhun Lee, Jungyu Jin, Taesu Kim, Hyungjun Kim, and Eunhyeok Park. Owq: Outlier-aware weight quantization for efficient fine-tuning and inference of large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 13355--13364, 2024

    2024

  21. [29]

    Towards debiasing sentence representations

    Paul Pu Liang, Irene Mengze Li, Emily Zheng, Yao Chong Lim, Ruslan Salakhutdinov, and Louis-Philippe Morency. Towards debiasing sentence representations. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, Proceedings of the 58th Annual Meeting of the A...

    2020 doi

  22. [30]

    Do emergent abilities exist in quantized large language models: An empirical study

    Peiyu Liu, Zikang Liu, Ze-Feng Gao, Dawei Gao, Wayne Xin Zhao, Yaliang Li, Bolin Ding, and Ji rong Wen. Do emergent abilities exist in quantized large language models: An empirical study. ArXiv, abs/2307.08072, 2023. URL https://api.semanticscholar.org/CorpusID:259937594

    2023 arXiv

  23. [31]

    B lack is to criminal as C aucasian is to police: Detecting and removing multiclass bias in word embeddings

    Thomas Manzini, Lim Yao Chong, Alan W Black, and Yulia Tsvetkov. B lack is to criminal as C aucasian is to police: Detecting and removing multiclass bias in word embeddings. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the...

    2019

  24. [32]

    Social bias probing: Fairness benchmarking for language models

    Marta Marchiori Manerba, Karolina Stanczak, Riccardo Guidotti, and Isabelle Augenstein. Social bias probing: Fairness benchmarking for language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Nat...

    2024 doi

  25. [33]

    Kelly Marchisio, Saurabh Dash, Hongyu Chen, Dennis Aumiller, Ahmet \"U st \"u n, Sara Hooker, and Sebastian Ruder. How does quantization affect multilingual LLM s? In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Ling...

    2024 doi

  26. [34]

    An empirical survey of the effectiveness of debiasing techniques for pre-trained language models

    Nicholas Meade, Elinor Poole-Dayan, and Siva Reddy. An empirical survey of the effectiveness of debiasing techniques for pre-trained language models. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association...

    2022 doi

  27. [35]

    Pointer sentinel mixture models

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843, 2016

    2016 arXiv

  28. [36]

    Alireza Mohammadshahi, Vassilina Nikoulina, Alexandre Berard, Caroline Brun, James Henderson, and Laurent Besacier. What do compressed multilingual machine translation models forget? In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors, Findings of the Association for C...

    2022 doi

  29. [37]

    LSDS em 2017 shared task: The story cloze test

    Nasrin Mostafazadeh, Michael Roth, Annie Louis, Nathanael Chambers, and James Allen. LSDS em 2017 shared task: The story cloze test. In Michael Roth, Nasrin Mostafazadeh, Nathanael Chambers, and Annie Louis, editors, Proceedings of the 2nd Workshop on Linking Models of Lexical...

    2017 doi

  30. [38]

    S tereo S et: Measuring stereotypical bias in pretrained language models

    Moin Nadeem, Anna Bethke, and Siva Reddy. S tereo S et: Measuring stereotypical bias in pretrained language models. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and ...

    2021 doi

  31. [39]

    Up or down? adaptive rounding for post-training quantization

    Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Christos Louizos, and Tijmen Blankevoort. Up or down? adaptive rounding for post-training quantization. In International conference on machine learning, pages 7197--7206. PMLR, 2020

    2020

  32. [40]

    Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R. Bowman. C row S -pairs: A challenge dataset for measuring social biases in masked language models. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods...

    2020 doi

  33. [41]

    Social-group-agnostic bias mitigation via the stereotype content model

    Ali Omrani, Alireza Salkhordeh Ziabari, Charles Yu, Preni Golazizian, Brendan Kennedy, Mohammad Atari, Heng Ji, and Morteza Dehghani. Social-group-agnostic bias mitigation via the stereotype content model. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Procee...

    2023 doi

  34. [42]

    BBQ : A hand-built bias benchmark for question answering

    Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel Bowman. BBQ : A hand-built bias benchmark for question answering. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Findings of the Associ...

    2022 doi

  35. [43]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019

    2019

  36. [44]

    Layered bias: Interpreting bias in pretrained large language models

    Nirmalendu Prakash and Roy Ka-Wei Lee. Layered bias: Interpreting bias in pretrained large language models. In Yonatan Belinkov, Sophie Hao, Jaap Jumelet, Najoung Kim, Arya McCarthy, and Hosein Mohebbi, editors, Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interp...

    2023 doi

  37. [45]

    Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J

    Colin Raffel, Noam M. Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21: 0 140:1--140:67, 2019. URL https://api....

    2019

  38. [46]

    A comparative study on the impact of model compression techniques on fairness in language models

    Krithika Ramesh, Arnav Chavan, Shrey Pandit, and Sunayana Sitaram. A comparative study on the impact of model compression techniques on fairness in language models. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Proceedings of the 61st Annual Meeting of the A...

    2023 doi

  39. [47]

    Null it out: Guarding protected attributes by iterative nullspace projection

    Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. Null it out: Guarding protected attributes by iterative nullspace projection. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, Proceedings of the 58th Annual Meeting of the...

    2020 doi

  40. [48]

    Self-diagnosis and self-debiasing: A proposal for reducing corpus-based bias in NLP

    Timo Schick, Sahana Udupa, and Hinrich Sch \"u tze. Self-diagnosis and self-debiasing: A proposal for reducing corpus-based bias in NLP . Transactions of the Association for Computational Linguistics, 9: 0 1408--1424, 2021. doi:10.1162/tacl_a_00434. URL https://aclanthology.or...

    2021 doi

  41. [49]

    Upstream mitigation is not all you need: Testing the bias transfer hypothesis in pre-trained language models

    Ryan Steed, Swetasudha Panda, Ari Kobren, and Michael Wick. Upstream mitigation is not all you need: Testing the bias transfer hypothesis in pre-trained language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long ...

    2022

  42. [50]

    Transformers: State-of-the-art natural language processing

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, et al. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 conference on empirical methods in na...

    2020

  43. [51]

    Smoothquant: Accurate and efficient post-training quantization for large language models

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models. In International conference on machine learning, pages 38087--38099. PMLR, 2023

    2023

  44. [52]

    Beyond perplexity: Multi-dimensional safety evaluation of LLM compression

    Zhichao Xu, Ashim Gupta, Tao Li, Oliver Bentham, and Vivek Srikumar. Beyond perplexity: Multi-dimensional safety evaluation of LLM compression. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024,...

    2024 doi

  45. [53]

    Zeroquant: Efficient and affordable post-training quantization for large-scale transformers

    Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, and Yuxiong He. Zeroquant: Efficient and affordable post-training quantization for large-scale transformers. Advances in Neural Information Processing Systems, 35: 0 27168--27183, 2022

    2022

  46. [54]

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. H ella S wag: Can a machine really finish your sentence? In Anna Korhonen, David Traum, and Llu \' s M \`a rquez, editors, Proceedings of the 57th Annual Meeting of the Association for Computational Lingui...

    2019 doi

  47. [55]

    Opt: Open pre-trained transformer language models

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022

    2022 arXiv

  48. [56]

    Unibias: Unveiling and mitigating llm bias through internal attention and ffn manipulation

    Hanzhang Zhou, Zijian Feng, Zixiao Zhu, Junlang Qian, and Kezhi Mao. Unibias: Unveiling and mitigating llm bias through internal attention and ffn manipulation. Advances in Neural Information Processing Systems, 37: 0 102173--102196, 2024

    2024

  49. [57]

    A survey on model compression for large language models

    Xunyu Zhu, Jian Li, Yong Liu, Can Ma, and Weiping Wang. A survey on model compression for large language models. Transactions of the Association for Computational Linguistics, 12: 0 1556--1577, 2024

    2024

  50. [58]

    Mielke, Hanna Wallach, and Ryan Cotterell

    Ran Zmigrod, Sabrina J. Mielke, Hanna Wallach, and Ryan Cotterell. Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology. In Anna Korhonen, David Traum, and Llu \'i s M \`a rquez, editors, Proceedings of the 57th Annual Meeting of...

    2019 doi

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.