{"id":"a09c5296-af1c-4fbb-93ba-e56428ff68f1","arxiv_id":"2509.15206","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Fair-GPTQ modifies GPTQ's quantization objective with a paired-stereotype bias penalty, reducing measured social bias in 4-bit models with modest accuracy loss.","lead":"Fair-GPTQ adds a fairness penalty to the GPTQ quantization objective, so that compressing a language model to 4-bit weights also steers it away from stereotyped outputs about gender, race, and religion. The method reduces measured stereotype scores on several benchmarks while keeping at least 90% of zero-shot accuracy and the memory savings of 4-bit quantization.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1's quantization loop uses H_acc^{-1}, not the full H from Eq. (4), so the reported models may not implement the derived OBS solution.","rationale":"The reader's verdict is CONDITIONAL, and I agree that the paper should not be accepted without resolving the implementation/derivation gap. The reader's stated weakest_assumption is about transfer of the calibration objective to holdout bias benchmarks, but the reader's rationale independently flags the Algorithm 1 H_acc vs H issue, which is exactly the load-bearing concern I identify. I mark agreement as partial because my focus is the internal consistency of the method, not the transfer-to-holdout assumption. The transfer concern is real but secondary: the paper reports some holdout evidence (BBQ, SoFA), and even if transfer is imperfect, the method's core claim would still be meaningful if it reliably reduces measured stereotype scores. The H_acc/H mismatch, by contrast, undermines the core claim that the reported reductions come from the derived OBS solution. Without code, one cannot tell whether the results are produced by the theory, by a heuristic pre-update, or by some other implementation detail. This does not warrant outright rejection, because the empirical pattern is plausible and the discrepancy could be benign; it does warrant a condition: release code or correct the algorithm and confirm that the fix does not change the conclusions. I therefore keep the reader's CONDITIONAL verdict unchanged.","tokens_in":22859,"tokens_out":7368,"duration_ms":78677,"concrete_test":"Reimplement Fair-GPTQ with the full Hessian in the quantization loop: replace Algorithm 1 line 6 with C ← Cholesky(H⁻¹)ᵀ, where H is the full matrix from Eq. (4) (same α and any GPTQ damping), keeping all other settings identical. Quantize OPT-6.7B and Mistral-7B with the ALL/U/L/UL layer strategies and compare Wiki PPL, CrowS-Pairs, StereoSet, and zero-shot accuracy against Table 1. If CP/SS shift by more than ~1 point or PPL changes by >5%, the published results depend on the Hessian mismatch; if they reproduce Table 1, the mismatch is numerically benign. Additionally, run an ablation that applies only the line-5 debiasing update with no quantization compensation, to determine whether the reported debiasing effect is driven by the pre-update rather than the OBS correction.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that Fair-GPTQ adds the bias penalty α||W'(X0−X1)||² to the GPTQ objective and that the derived OBS update yields the reported debiasing. But Algorithm 1 as written solves a different problem. Eq. (4) defines H = 2(X0X0ᵀ + X1X1ᵀ + αΔXΔXᵀ), and Proposition 1/Eq. (6) requires H⁻¹ in both the debiasing correction and the quantization-compensation term. In Algorithm 1, line 2 sets H_acc = X0X0ᵀ + X1X1ᵀ, line 6 computes C ← Cholesky(H_acc⁻¹)ᵀ, and lines 10–11 use C for the quantization error feedback. Thus the quantization loop minimizes only the first two reconstruction terms, omitting the bias penalty from the compensation step. The pre-update on line 5 uses the full H, but the subsequent OBS-style correction does not. The paper provides no code, so the reported CP/SS improvements and PPL numbers cannot be checked against the derivation. Moreover, line 9 quantizes W after the line-5 debiasing shift, whereas Eq. (6) is stated in terms of quant(w_q) for the original weights, adding a second discrepancy. If the implementation actually follows Algorithm 1, the headline claim that experiments 'validate our theoretical solution' is not supported; if the implementation follows Eq. (6), the pseudocode misdescribes it. Either way, the link between the theory and the empirical results is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Fair-GPTQ, a post-training quantization method that augments the GPTQ layer-wise reconstruction objective with a bias penalty α||W'(X0−X1)||², where X0 and X1 are paired stereotypical/anti-stereotypical inputs. The authors derive an OBS-style closed-form update (Proposition 1, Eq. 6) that accounts for a non-zero gradient at the pretrained weights, and they present Algorithm 1 to perform debiasing and quantization. Experiments on OPT and Mistral models at 4 bits report lower CrowS-Pairs, StereoSet, and BBQ stereotype scores than FP16 and GPTQ baselines, with some degradation in zero-shot accuracy and perplexity, and claim at least 90% of baseline zero-shot accuracy is preserved. The method is also compared with INLP, Self-Debias, and SentenceDebias, and a sanity check with unrelated calibration pairs is included.","tokens_in":23284,"tokens_out":4415,"duration_ms":46506,"significance":"If the claimed results hold, Fair-GPTQ would be a useful contribution: it is, to my knowledge, the first quantization scheme that explicitly builds a group-fairness term into the GPTQ objective, and the algebraic derivation of the OBS-like update with a non-zero gradient is clean and clearly presented. The paper also includes a meaningful sanity check (Appendix D.1), evaluates on external holdout benchmarks (CrowS-Pairs test, BBQ, SoFA), and reports scaling across OPT model sizes. These are strengths. However, the current manuscript has a load-bearing mismatch between the stated theoretical solution and Algorithm 1, and no code is provided, so the link between the derivation and the reported empirical gains is not established.","major_comments":[{"comment":"Algorithm 1 does not implement the optimization problem solved in Proposition 1. Eq. (4) defines H = X0X0^T + X1X1^T + αΔXΔX^T, and Eq. (6) requires H_w^{-1} = (H⊗I)^{-1} in both the debiasing correction and the quantization-compensation term. However, Algorithm 1 line 6 sets C ← Cholesky(H_acc^{-1})^T with H_acc = X0X0^T + X1X1^T, and lines 10–11 use this C for error compensation. The bias term αΔXΔX^T is therefore omitted from the compensation step. As written, the algorithm minimizes a different objective and does not solve Eq. (5). If the experiments actually use the full H in the loop, the pseudocode misdescribes the implementation; if they follow the pseudocode, the claim that experiments 'validate our theoretical solution' is unsupported. Without released code, this cannot be checked.","section":"§3.2, Algorithm 1, Eq. (4)–(6)"},{"comment":"A second discrepancy: Eq. (6) is derived for the original weights w and their quantized values quant(w_q). In Algorithm 1, line 5 first applies the debiasing shift W ← W − (H^{-1}H_bias W^T)^T, and only then line 9 quantizes W:,j. The quantized values are therefore quant(w_q + δ_q), not quant(w_q), while the compensation term in Eq. (6) is expressed in terms of (w_q − quant(w_q)). Unless the derivation is reworked to account for the pre-shifted weights, the update used in the algorithm is not the one derived in Proposition 1.","section":"§3.2, Algorithm 1, lines 5–9; Eq. (6)"},{"comment":"The claim that Fair-GPTQ 'preserves at least 90% of baseline accuracy on zero-shot benchmarks' is not supported by the reported results for all configurations. In Table 1, Mistral-7B with Fair-GPTQ_l achieves Cloze 62.87 vs FP16 78.29 (≈80.3%) and HSwag 47.11 vs 60.93 (≈77.3%). Since the lower-layer strategy is one of the paper's main findings, the abstract's unqualified 90% assertion is an overstatement. The claim should be restricted to the configurations where it holds, or revised.","section":"Abstract and §5.1, Table 1"},{"comment":"The fairness penalty in Eq. (1) is a proxy for the stereotype benchmarks, and α is selected on the development split of the same benchmark family used for calibration (StereoSet dev). The paper reports reductions on StereoSet test and CrowS-Pairs, which reduces but does not eliminate the concern of selection on the evaluation axis. I would like to see either a sensitivity analysis across α values on holdout benchmarks or a confirmation that the reported qualitative conclusions are stable for a range of α rather than only for the tuned value.","section":"§5.1, α selection; §4.2 calibration data"}],"minor_comments":[{"comment":"The derivation of H and J is correct, but the notation J for both the matrix gradient and the Jacobian could be confusing. Also, Eq. (4) writes H = 2(X0X0^T + X1X1^T + αΔXΔX^T), while the text later refers to H_acc = X0X0^T + X1X1^T and H_bias = 2αΔXΔX^T; the factor 2 placement is inconsistent between the main text and Algorithm 1. Please align the notation.","section":"§3.1, Eqs. (2)–(4)"},{"comment":"In the expression for L(λ), the term λ[−e_q^T H^{-1}(λe_q + H_w) + ...] appears to contain a typo: H_w should be J_w. The final result is correct, but the intermediate expression should be fixed.","section":"Appendix A, proof of Proposition 1"},{"comment":"The α grid is described as 'between 0.1 and 1 in steps of 0.5', which is ambiguous (0.1, 0.6, 1.1? or 0.1, 0.5, 1.0?). Please specify the exact values tested.","section":"§4.2 and Table 1"},{"comment":"On SoFA, Fair-GPTQ does not uniformly reduce bias: for OPT-6.7B, Fair-GPTQ_ALL increases race bias (0.101 vs GPTQ-SS 0.092) and disability bias (0.048 vs 0.044), and Fair-GPTQ_l increases disability bias to 0.057. The paper should comment on this rather than only highlighting improvements on other benchmarks.","section":"Appendix D.4, Table 9"},{"comment":"The y-axis label 'ΔW / W' and the logarithmic scale are not fully defined. Please specify whether ΔW is the Frobenius norm of the debiasing update, and how the relative magnitudes are averaged over layers.","section":"§5.1, Figure 4"},{"comment":"The paper states that code will be released upon acceptance. Given the discrepancy between Algorithm 1 and Eq. (6), providing the implementation or pseudocode that exactly matches the experiments is essential for reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The core idea is timely and the theoretical derivation is mostly clean, but the algorithm-experiment gap is a serious obstacle. I would encourage the editor to request a corrected pseudocode and, ideally, a code release or a detailed verification that the implementation uses the full H in the quantization loop. If the experiments were run with H_acc in the compensation, the reported debiasing would need to be reinterpreted and the theory/experiment link re-established."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nShort version: Fair-GPTQ is a genuinely new idea — add a paired-stereotype penalty to the GPTQ objective and derive the OBS update with nonzero gradient — and the evaluation is broad and mostly honest. But there is a load-bearing mismatch between the derivation and Algorithm 1, and no code, so the headline empirical claims are not yet tied to the theory.\n\nWhat's new and good: the objective in Eq. 1 is a natural extension of GPTQ, and the derivation of Proposition 1 is algebraically clean. The matrix-level analysis of which layers and matrices contribute most to bias during quantization is a nice, novel diagnostic. The eval is serious: CrowS-Pairs, StereoSet, BBQ, SoFA, multiple OPT sizes plus Mistral, perplexity and zero-shot, a sanity check with unrelated pairs, and a runtime comparison. The authors are honest about limitations in Appendix F. That's real work.\n\nSoft spots, in order of significance. First, Algorithm 1 line 6 computes the Cholesky factor from H_acc inverse, not the full H from Eq. 4, so the quantization loop uses the GPTQ Hessian without the bias term. Line 5 applies the debiasing pre-shift, but the subsequent error compensation should still use full H if Eq. 6 is the solution. As written, the algorithm does not implement the derived solution. Since code is not released, the reader cannot tell whether the reported debiasing comes from the pre-shift alone, from the full derivation, or from something else. This is not a cosmetic issue; the paper's central claim — 'results validate our theoretical solution' — is unsupported as written.\n\nSecond, there are no error bars or repeated runs. Given that alpha is tuned on the dev split and the fairness penalty is a proxy for the benchmark metrics, the reported differences (e.g., CrowS-Pairs 65.95 to 63.92) need some sense of variance before they can be taken at face value.\n\nThird, the method is applied only to two matrices per layer; the rest are vanilla GPTQ. That's a fine design choice, but it means the 'Fair-GPTQ' label covers a partial intervention, and it would be good to report which matrices were treated in the main results tables.\n\nThe circularity concern is real but partly mitigated: they report on holdout and external benchmarks, so it is not pure tautology.\n\nBottom line: this deserves a serious referee. The idea is important enough, and the evaluation broad enough, that a careful revision could make it a solid contribution. But I would not accept it as is; the authors need to release code and either align Algorithm 1 with the derivation or derive the exact alternative they actually implement. I'd send it to review, with the expectation of major revision.","headline":"A promising fairness-aware quantization objective, but the paper's own pseudocode doesn't match the derived update, so the empirical claims are not yet anchored to the theory.","tokens_in":23685,"tokens_out":3633,"would_cite":false,"duration_ms":39174,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fair-GPTQ claims that adding a bias penalty to GPTQ's quantization objective produces 4-bit LLMs with lower stereotype scores than the FP16 models they are compressed from, while preserving at least 90% of zero-shot accuracy and retaining 4","keywords":["quantization","large language models","fairness","stereotype bias","GPTQ","post-training quantization","bias mitigation","group fairness"],"falsifier":"Quantize a model with Fair-GPTQ using the same StereoSet pairs, then evaluate on a benchmark of newly authored minimal pairs that share no templates with StereoSet. If stereotype scores do not improve, the debiasing effect is calibration-overlap rather than a general property. Alternatively, probe layer-by-layer bias scores after applying the update only to lower layers to check whether unmodified upper layers re-amplify the difference.","tokens_in":22786,"feed_emoji":"⚖️","tokens_out":5529,"duration_ms":55196,"temperature":0.7,"pith_summary":"Fair-GPTQ claims that the bias amplification observed in quantized language models is not a fixed cost of compression: by adding a bias penalty to the quantization objective, the rounding of weights can be steered to reduce stereotype generation. The method modifies GPTQ's optimization so that quantized weights actively minimize the gap in model behavior between stereotypical and anti-stereotypical sentence pairs, and it derives a closed-form update from the optimal brain surgeon framework. On OPT and Mistral models quantized to 4 bits, it reports lower CrowS-Pairs and StereoSet bias scores than both the half-precision models and GPTQ baselines, while preserving at least 90% of zero-shot accuracy and retaining 4-bit memory and speed benefits. This matters because debiasing is folded into compression rather than added as an expensive post-hoc step, making fairness-aware deployment of large models more practical.","feed_headline":"Fair-GPTQ cuts stereotype bias in 4-bit LLMs","feed_subtitle":"Adding a fairness term to GPTQ's objective beats FP16 on bias benchmarks while keeping 90% of accuracy.","key_machinery":"The bias-aware Hessian H = 2(X0X0ᵀ + X1X1ᵀ + α(X0−X1)(X0−X1)ᵀ) and the non-zero gradient J = 2αW(X0−X1)(X0−X1)ᵀ; the OBS solution yields a closed-form debiasing update W ← W − (H⁻¹H_biasWᵀ)ᵀ that is applied before the standard GPTQ quantization loop. Because the Hessian is block-diagonal across weight rows, the update is computed once per column and applied across all rows, keeping the per-layer cost the same as GPTQ.","core_discovery":"The central claim is that a fairness constraint can be embedded directly into the quantization objective. The paper starts from GPTQ's layer-wise weight-reconstruction loss and adds the term α ||W′(X0−X1)||², where X0 and X1 are paired texts that differ only in a protected-attribute token. Keeping the gradient of this term non-zero at the pretrained weights, it solves the resulting constrained OBS problem and obtains a weight update that separates into a debiasing correction applied before the GPTQ rounding loop. Experiments on OPT and Mistral show that this correction lowers stereotype likelihood on CrowS-Pairs, StereoSet, and BBQ relative to FP16 and GPTQ, with zero-shot accuracy staying w","pith_inferences":["Inference: The same paired-difference penalty could be ported to other weight-rounding schemes (e.g., round-to-nearest with error compensation), potentially generalizing fairness-aware compression beyond GPTQ-family methods.","Inference: The lower-layer effect hints that stereotype-relevant computation is concentrated in early layers in these transformers; if true, debiasing could be targeted even more cheaply by adjusting only the bottom few layers.","Inference: A direct stress test would be to calibrate on StereoSet but evaluate on a newly constructed minimal-pair benchmark; if the bias reduction disappears, the method's gains are tied to calibration-set overlap rather than a general mechanism."],"forward_implications":["Debiasing can be incorporated into a single quantization pass, increasing runtime by only about 20% compared to GPTQ, rather than requiring separate fine-tuning or inference-time interventions.","4-bit quantized models can have lower stereotype scores than the half-precision models they are compressed from, showing that compression need not amplify bias if the rounding is fairness-aware.","The attention output projection and the MLP output (down-projection) matrices carry the largest bias-correction signal, so targeting these matrices is sufficient for most of the debiasing effect.","Applying the fairness update to the lower layers yields the largest stereotype-score reductions, giving a cheap strategy when full-layer updates are infeasible.","The method matches or exceeds iterative null-space projection debiasing on racial stereotypes, while also delivering 4-bit memory and speed benefits."],"fun_headline_variants":["Fair-GPTQ: 4-bit quantization beats FP16 on bias","Add fairness to quantization: Fair-GPTQ reduces LLM bias","Fair-GPTQ embeds fairness into quantization, reducing LLM bias","Fair-GPTQ: Bias-aware 4-bit quantization for LLMs","Quantize LLMs fairly: Fair-GPTQ reduces bias"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The method assumes that lower bias on the paired calibration sentences (StereoSet development pairs) transfers to lower stereotype rates on unrelated benchmarks like CrowS-Pairs and BBQ; if that transfer fails, the reported debiasing is an artifact of calibration-data overlap.","fun_headline_variants_meta":{"raw":{"variants":["Fair-GPTQ: 4-bit quantization beats FP16 on bias","Add fairness to quantization: Fair-GPTQ reduces LLM bias","Fair-GPTQ embeds fairness into quantization, reducing LLM bias","Fair-GPTQ: Bias-aware 4-bit quantization for LLMs","Quantize LLMs fairly: Fair-GPTQ reduces bias"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000994,"raw_usage":{"total_tokens":4024,"prompt_tokens":700,"completion_tokens":3324,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":444,"completion_tokens_details":{"reasoning_tokens":3231}},"tokens_in":444,"tokens_out":3324,"duration_ms":20882,"temperature":1.0,"reasoning_tokens":3231,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T16:13:45.266380+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Quantize a model with Fair-GPTQ using the same StereoSet pairs, then evaluate on a benchmark of newly authored minimal pairs that share no templates with StereoSet. If stereotype scores do not improve, the debiasing effect is calibration-overlap rather than a general property. Alternatively, probe layer-by-layer bias scores after applying the update only to lower layers to check whether unmodified upper layers re-amplify the difference.","supporting_citations":[],"review_version":1}