Pith. sign in

REVIEW 5 major objections 5 minor 29 references

Group Fairness Meets the Black Box: Enabling Fair Algorithms on Closed LLMs via Post-Processing

T0 review · 5 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Closed-weight LLMs can be turned into fair classifiers by prompting them for label and group probabilities, then feeding those probabilities to a standard fair post-processing algorithm.

desk verdict A solid, honest engineering paper: low-dimensional features from LLM probabilities enable fair algorithms on closed-weight models, with a real low-data advantage; the sufficiency premise is idealized but the authors say so themselves and the empirical result stands. read the letter →

arxiv 2508.11258 v1 pith:IVQ5IDSW submitted 2025-08-15 cs.LG cs.CLcs.CY

Ruicheng Xian , Yuxuan Wan , Han Zhao This is my paper
classification cs.LGcs.CLcs.CY MSC 68T0562H30
keywords groupfairnesslargelanguagemodelspost-processingin-contextlearningsufficientstatisticsclosed-weightfairclassificationtabulardata
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a way to enforce group fairness on classifiers built from closed-weight LLMs that expose no weights, embeddings, or fine-tuning access. The idea is to treat the LLM as a fixed feature extractor: prompts elicit its probabilistic predictions — token log probabilities — for the class label and for the label together with the sensitive attribute, and those low-dimensional vectors become the input to a standard fair algorithm that trains a lightweight fair classifier. The load-bearing claim is a sufficiency result: if the elicited probabilities are accurate estimates of $P(Y \mid X)$ and $P(A, B \mid X)$, they already contain the information needed for optimal fair classification. Experiments across five datasets, four LLMs (including GPT-4o), and three fair algorithms show accuracy-fairness tradeoffs that match or beat fair classifiers trained on LLM embeddings or raw tabular features, with the clearest edge when training data is scarce. The paper states its own limitation: the guarantee requires the LLM's predictions to be Bayes-optimal, which is unlikely in practice.

What carries the argument

The sufficient-statistics reduction: recent post-processing theory shows the Bayes-optimal fair classifier is a function of $P(Y \mid X)$ and $P(A, B \mid X)$, collapsing under mild continuity to a linear mapping. The operational identity is the prompt decomposition $P(A, Y \mid X) = P(Y \mid X)\,P(A \mid Y, X)$: separate multiple-choice prompts elicit label probabilities and group probabilities conditioned on each hypothetical label, and the logits are combined so their softmax equals the estimated joint. Logistic-regression calibration refits the joint logits to ground-truth $(A, Y)$ labels; the calibrated probabilities, lying in the simplex $\Delta^{G\times K}$, become the feature space f

What would settle it

On a task where the sensitive attribute is invisible in the text the LLM sees, train the framework's classifier on elicited probabilities and compare the smallest attainable fairness violation with the violation the same fair algorithm reaches when handed the true $P(A, Y \mid X)$. If the violation floor stays bounded above the oracle level as training data grows, the sufficient-statistics premise fails for that model-task pair. A cheaper probe: measure the calibration gap between the LLM's elicited $P(Y \mid X)$ and empirical label frequencies; large gaps mark where the sufficiency argument b

Watch

Extended reading notes

Core claim

Central claim: under standard group-fairness criteria, the Bayes-optimal fair classifier is a function of $P(Y \mid X)$ and $P(A, B \mid X)$, the label given the input and the joint with the criterion's conditioning variables. If an LLM's prompted probabilities are accurate estimates of these, they are sufficient statistics for fair classification. The framework elicits them as token log probabilities from multiple-choice prompts, refits them to ground-truth $(A,Y)$ labels via logistic calibration, and trains a lightweight fair classifier on $G\times K$-dimensional features. Empirically these features rival fair classifiers trained on LLM embeddings or raw tabular features in low-data regime

Load-bearing premise

The guarantee only holds if the LLM's elicited probabilities are accurate estimates of the true label and group distributions; the paper concedes in its conclusion that this would require Bayes-optimal predictions, which real LLMs will not produce.

Editorial extensions

If this is right

  • Closed-weight LLMs such as GPT-4o become usable as base models for fair classification with no weight or embedding access — only token log probabilities are needed.
  • Because the extracted features are $G\times K$-dimensional (4 on Adult, versus 4096 for Llama 3.1 8B embeddings), the framework sharply cuts sample complexity and wins in very-low-data regimes.
  • The framework covers statistical parity, TPR/FPR parity, and equalized odds, in the harder attribute-blind setting where the sensitive attribute is unavailable at test time.
  • Prompting for the group and label jointly matters: ablating the group predictions and keeping only label predictions generally degrades the fairness-accuracy tradeoff.
  • The pipeline composes with few-shot and chain-of-thought prompting and with prompt-based fairness interventions, so it can be applied to closed models as they improve.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The guarantee is an upper ceiling: any systematic bias or overconfidence in the LLM's probabilities that logistic calibration cannot correct will push the attainable fairness-accuracy frontier below the sufficiency bound — a gap the paper itself flags as likely. Whether that gap is small enough to matter in practice is testable and will vary by model and task.
  • The paper's regime analysis suggests a hybrid recipe it does not itself recommend: use elicited probabilities when labeled data is scarce, then switch to embedding-based or raw-feature training as data accumulates.
  • The framework implies a constructive test for when group prompts are worth their extra queries: if the label logits alone already predict the sensitive attribute well (as on BiasBios), joint elicitation adds little; an adaptive prompt-selection rule could exploit this.
  • The sufficiency logic transfers in principle to other post-processing objectives, such as calibration or robustness constraints, whenever the optimal predictor is known to depend on the input only through a low-dimensional conditional distribution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a post-processing framework for enforcing group fairness on closed-weight LLMs. The LLM is treated as a frozen feature extractor: prompted to output calibrated probabilities for P(Y|X) and P(A,B|X), which are converted into low-dimensional features, and then a standard fair algorithm (Reductions, MinDiff, LinearPost) is applied to train a lightweight fair classifier. The framework is evaluated on five datasets (three tabular, two textual), four LLMs (three open-weight, one closed-weight), and four fairness criteria. The main metric is AUTC, an area under the accuracy-fairness tradeoff curve. The paper reports that the proposed 'preds' features are data-efficient, outperforming embeddings and raw tabular features in low-data regimes, and that ablating the group-information prompts generally degrades performance.

Significance. If the sufficiency claim in Section 4 were established, the framework would be a valuable contribution: it would enable principled fairness mitigation on closed-weight LLMs where only token logits are available. The empirical study is broad and the code is released, which is commendable. However, the paper's own limitation statement (Section 7) concedes that the sufficiency condition holds only if the LLM produces Bayes-optimal predictions, which is unlikely. Consequently, the theoretical motivation is not operational for real LLMs, and the empirical results do not isolate sufficiency from the benefits of low-dimensional regularization. The framework remains a potentially useful practical heuristic, but its central theoretical claim is unsupported as stated. The AUTC evaluation also relies on a data-dependent cutoff choice that needs sensitivity analysis. These issues are fixable by reframing and additional experiments, but they affect the paper's main claims.

major comments (5)
  1. [Section 4 and Section 7] The sufficiency claim is load-bearing but explicitly conceded as unrealistic. Section 4 states that LLM predictions of P(Y|X) and P(A,B|X) 'constitute sufficient statistics for (optimal) fair classification,' while Section 7 admits 'This condition holds only if the LLM produces Bayes-optimal predictions, which is unlikely in practice.' The experiments do not test sufficiency; they only compare AUTC, which can favor the low-dimensional preds features for reasons of regularization rather than information content. Figure 4 shows a plateau for preds while embeddings and tabular features continue to improve with more data—consistent with an information bottleneck. Please either weaken the claim to a heuristic motivation, or provide evidence of approximate sufficiency, e.g., by comparing against features built from oracle P(Y|X), P(A,B|X) on a synthetic or semi-synthetic task, or by testing wh
  2. [Appendix B.1, Table 2] The AUTC metric is defined with a cutoff v that is chosen as 'the fairness violation of the highest-accuracy classifier among all configurations using either the preds or embeds features.' This makes the headline metric depend on the compared configurations: a configuration that achieves high accuracy at a moderate violation can set a large v, changing the area under the curve for all competitors. The source of v (validation vs test) is also unspecified, and no sensitivity analysis is reported. Please either fix v a priori per dataset/fairness criterion, or report AUTC under several v values to show that the conclusions in Fig. 3 are robust. The same applies to the penalty gamma.
  3. [Figure 3 and Section 6.1] The main results are presented without any uncertainty quantification. Each configuration is averaged over five random seeds, but Fig. 3 and the appendix tradeoff curves show only point estimates. This is particularly concerning because Section 6.2 notes that TPR results 'exhibit high variance.' Claims such as 'consistently produces fair classifiers with strong accuracy tradeoffs' and the preds-vs-embeds comparison need error bars or at least bootstrap intervals to support them. Please add confidence intervals or standard errors to the AUTC plots and to the tradeoff curves.
  4. [Abstract and Section 6.2] The abstract claims the framework 'outperforms fair classifiers trained on LLM embeddings ... or from scratch on raw tabular features' without the low-data qualifier that the paper's own experiments establish. Figure 4 shows that in the high-data regime (e.g., 20,000 examples), both tabular features and embeddings outperform preds on Adult. The claim should be scoped to the very low-data regime (below roughly 1,000 examples), or the abstract should be rephrased to accurately reflect the regime dependence reported in Section 6.2.
  5. [Appendix C, Overlapping Groups] On CivilComments, the feature construction for overlapping groups assumes conditional independence: the joint feature is built as a product of marginal logits for Y and each A_i. The fairness violation, however, is computed over events {A in I} that require membership in a subset but do not exclude membership in other groups (Appendix B.2), while the constructed features represent exact membership combinations ('all and only the groups in I'). This mismatch may cause the features to misestimate the probability of the events used in the fairness constraints. Please justify the independence assumption, discuss its potential impact on the CivilComments results, or evaluate sensitivity to it.
minor comments (5)
  1. [Section 4, step 1] 'see X for examples' contains an unresolved cross-reference; it should point to a specific table or listing.
  2. [Listing 4 caption] The caption says the placeholder {class_condition} is replaced with 'unlikely to recidivate' / 'likely to recidivate', but these are COMPAS class conditions, not BiasBios occupations or sex. This appears to be a copy-paste error.
  3. [Related Work / References] The author name 'Alexandru T, ifrea' appears with an unusual comma; please verify the correct spelling (likely 'Alexandru Tifrea').
  4. [Figure 2] Figure 2 is referenced in Section 5.2 before its formal definition in Appendix B.1. Consider moving the figure or adding a forward pointer.
  5. [Section 6.2] The three regimes are described qualitatively. It would help to mark the approximate training-size boundaries on Fig. 4 or state them explicitly.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; the central claim is a conditional application of independent prior theory, and the paper explicitly flags its key assumption as a limitation.

full rationale

The paper's core derivation is conditional: if the LLM produces accurate estimates of P(Y|X) and P(A,B|X), then these are sufficient statistics for fair classification. This is supported by prior theoretical work (Chen et al. 2024; Zeng et al. 2024; Xian and Zhao 2024). The Xian and Zhao citation involves overlapping authors, but it is a separate theoretical result with stated assumptions, not an unverified assertion, and the sufficiency claim is also independently attributed to Chen et al. and Zeng et al. The feature-engineering step fits logistic regression to ground-truth (A,Y) labels, but this calibration does not by construction determine the downstream fair classifier's accuracy-fairness tradeoff: Reductions, MinDiff, and LinearPost are standard post-hoc/in-processing algorithms applied to those features. The AUTC metric is a defined evaluation score, not a derivation of the framework's effectiveness. The paper explicitly acknowledges the main vulnerability: 'This condition holds only if the LLM produces Bayes-optimal predictions, which is unlikely in practice' (Section 7, Conclusion). That is a correctness/robustness limitation, not circularity: the empirical comparisons against embeddings and tabular features remain meaningful as feature-extraction comparisons. No fitted parameter is renamed as a prediction, and no equation is shown to reduce to its own input by construction. The one self-citation (Xian and Zhao 2024) is not load-bearing for the framework's empirical claims, so the appropriate score is low.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The framework introduces no new ontological entities. It relies on two key assumptions: the sufficiency of the conditional probabilities (from prior theory) and the accuracy of LLM probabilities (admittedly fragile). The AUTC cutoff and gamma are evaluation-metric hyperparameters that are data-dependent and could affect comparisons.

free parameters (4)
  • AUTC cutoff v = varies by dataset/criterion; e.g., Adult SP=0.177, EO=0.083 (Table 2)
    Set to the fairness violation of the highest-accuracy classifier among preds/embeds configurations; data-dependent and can alter which method scores best.
  • AUTC penalty gamma = 1
    Hand-chosen weight for low-violation region in the AUTC metric.
  • GPT-4o missing logit fill = -50
    Assigned when top-k=20 API logits omit options (28-class BiasBios); a heuristic that could affect feature values.
  • MinDiff MMD bandwidth sigma = 0.1
    Copied from Prost et al. (2019); kernel bandwidth for distribution matching.
assumptions (5)
  • domain assumption The Bayes-optimal fair classifier is a function of P(Y|X) and P(A,B|X)
    Section 4 cites Chen et al. (2024), Zeng et al. (2024), Xian and Zhao (2024) for this sufficiency result; it is the theoretical load-bearing claim of the framework.
  • domain assumption LLM log probabilities are accurate estimates of P(Y|X) and P(A,B|X)
    Required for the elicited features to be sufficient statistics. The paper admits in the Conclusion that Bayes-optimality 'is unlikely in practice'.
  • ad hoc to paper Conditional independence A ⊥ Y | X on BiasBios
    Section 5.1: 'in the prompt design, we assume conditional independence A ⊥ Y | X'; used to reduce prompts to two. Section 6.3 shows Y logits already encode A (0.877 balanced accuracy), so the assumption is violated.
  • ad hoc to paper Independence among overlapping group indicators A_i and Y on CivilComments
    Appendix C constructs qA,Y as a product of independent softmaxes: softmax(qA,Y)_{I,k} = softmax(qY)_k * prod_i softmax(qAi). Comments may mention multiple religions jointly, so this independence may not hold.
  • domain assumption Continuity condition for LinearPost (small random perturbations)
    Appendix D.3 invokes Xian and Zhao (2024) where linear post-processors are optimal under a continuity condition satisfied via random perturbations to pA,Y.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Group Fairness Meets the Black Box: Enabling Fair Algorithms on Closed LLMs via Post-Processing." pith.science (2026). https://pith.science/paper/IVQ5IDSW

@misc{pith2026250811258,
  author       = {Pith},
  title        = {Pith review of: Group Fairness Meets the Black Box: Enabling Fair Algorithms on Closed LLMs via Post-Processing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IVQ5IDSW}},
  note         = {Machine review of arXiv:2508.11258}
}
read the original abstract

Instruction fine-tuned large language models (LLMs) enable a simple zero-shot or few-shot prompting paradigm, also known as in-context learning, for building prediction models. This convenience, combined with continued advances in LLM capability, has the potential to drive their adoption across a broad range of domains, including high-stakes applications where group fairness -- preventing disparate impacts across demographic groups -- is essential. The majority of existing approaches to enforcing group fairness on LLM-based classifiers rely on traditional fair algorithms applied via model fine-tuning or head-tuning on final-layer embeddings, but they are no longer applicable to closed-weight LLMs under the in-context learning setting, which include some of the most capable commercial models today, such as GPT-4, Gemini, and Claude. In this paper, we propose a framework for deriving fair classifiers from closed-weight LLMs via prompting: the LLM is treated as a feature extractor, and features are elicited from its probabilistic predictions (e.g., token log probabilities) using prompts strategically designed for the specified fairness criterion to obtain sufficient statistics for fair classification; a fair algorithm is then applied to these features to train a lightweight fair classifier in a post-hoc manner. Experiments on five datasets, including three tabular ones, demonstrate strong accuracy-fairness tradeoffs for the classifiers derived by our framework from both open-weight and closed-weight LLMs; in particular, our framework is data-efficient and outperforms fair classifiers trained on LLM embeddings (i.e., head-tuning) or from scratch on raw tabular features.

Figures

Figures reproduced from arXiv: 2508.11258 by the authors.

Figure 1
Figure 1. Flowchart of our proposed framework for deriving fair classifiers from LLMs. Given a task [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. AUTC, in its basic form, is the area under Pareto-optimal tradeoffs within the active region (i.e., above the base rate). Area Under the Tradeoff Curve (AUTC). To enable quan￾titative comparisons across the many configurations evaluated, we introduce a metric called area under the tradeoff curve, which summarizes each curve into a single value. AUTC is defined rel￾ative to the problem’s base rate (i.e., accuracy of … view at source ↗
Figure 3
Figure 3. Area under the tradeoff curve (Appendix B.1) achieved by each configuration (Section 5.2). [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: AUTC scores on the Adult dataset using LinearPost and Llama 3.1 open-weight models, [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: AUTC scores comparing our proposed framework, which prompts the LLM for both [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: AUTC according to Eqs. (1) and (2), with penalty [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: Accuracy-fairness tradeoffs on the Adult dataset for the experiments in Section 6.1. [PITH_FULL_IMAGE:figures/full_fig_p025_7.png]
Figure 8
Figure 8. Figure 8: Accuracy-fairness tradeoffs on the ACSIncome dataset for the experiments in Section 6.1. [PITH_FULL_IMAGE:figures/full_fig_p026_8.png]
Figure 9
Figure 9. Figure 9: Accuracy-fairness tradeoffs on the COMPAS dataset for the experiments in Section 6.1. [PITH_FULL_IMAGE:figures/full_fig_p027_9.png]
Figure 10
Figure 10. Figure 10: Accuracy-fairness tradeoffs on the BiasBios dataset for the experiments in Section 6.1. [PITH_FULL_IMAGE:figures/full_fig_p028_10.png]
Figure 11
Figure 11. Figure 11: Accuracy-fairness tradeoffs on the CivilComments dataset for the experiments in Sec [PITH_FULL_IMAGE:figures/full_fig_p029_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

29 extracted references · 19 canonical work pages

  1. [1]

    If a dataset includes a pre-defined split (e.g., Adult and BiasBios), we merge all splits and re-split randomly. Due to resource constraints, we do not use the full dataset in every case: for example, we sample 40000 out of 1664500 examples from ACSIncome, 90000 out of 393423 from BiasBios (the version scrapped and hosted by Ravfogel et al. (2020)), and 9...

    work page 2020

  2. [3]

    Rachel K. E. Bellamy, Kuntal Dey, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Kalapriya Kannan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilovic, Seema Nagar, Karthikeyan Natesan Ramamurthy, John Richards, Diptikalyan Saha, Prasanna Sattigeri, Monin- der Singh, Kush R. Varshney, and Yunfeng Zhang. AI Fairness 360: An Extensible T...

    arXiv 2018

  3. [4]

    Shikha Bordia and Samuel R. Bowman. Identifying and Reducing Gender Bias in Word-Level Language Models. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop,

    work page 2019

  4. [8]

    Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting

    Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting. InProceedings of the 2019 Conference on Fairness, Accountability, and Transparency,

    work page 2019

  5. [9]

    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, volume 1,

    work page 2019

  6. [11]

    Gemma 3 Technical Report, 2025.arXiv:2503.19786 [cs.CL]

    Gemma Team. Gemma 3 Technical Report, 2025.arXiv:2503.19786 [cs.CL]. 14 Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A Kernel Two-Sample Test.Journal of Machine Learning Research, 13(25),

    arXiv 2025

  7. [13]

    Strategic Demonstration Selection for Improved Fairness in LLM In-Context Learning

    Jingyu Hu, Weiru Liu, and Mengnan Du. Strategic Demonstration Selection for Improved Fairness in LLM In-Context Learning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,

    work page 2024

  8. [14]

    Yunqi Li, Lanjing Zhang, and Yongfeng Zhang

    arXiv:2005.11720 [cs.LG]. Yunqi Li, Lanjing Zhang, and Yongfeng Zhang. Fairness of ChatGPT,

    arXiv 2005

Show all 29 references
  1. [15]

    Yanchen Liu, Srishti Gautam, Jiaqi Ma, and Himabindu Lakkaraju

    arXiv:2305.18569 [cs.LG]. Yanchen Liu, Srishti Gautam, Jiaqi Ma, and Himabindu Lakkaraju. Confronting LLMs with Tra- ditional ML: Rethinking the Fairness of Large Language Models in Tabular Classifications. In Proceedings of the 2024 Conference of the North American Chapter of...

    2024 arXiv

  2. [16]

    The Llama 3 Herd of Models, 2024.arXiv:2407.21783 [cs.AI]

    Llama Team. The Llama 3 Herd of Models, 2024.arXiv:2407.21783 [cs.AI]. ChandlerMay, AlexWang, ShikhaBordia, SamuelR.Bowman, andRachelRudinger. OnMeasuring Social Biases in Sentence Encoders. InProceedings of the 2019 Conference of the North Amer- ican Chapter of the Associatio...

    2024 arXiv

  3. [17]

    Williamson

    15 Aditya Krishna Menon and Robert C. Williamson. The Cost of Fairness in Binary Classification. In Proceedings of the 2018 Conference on Fairness, Accountability, and Transparency,

    2018

  4. [18]

    Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R. Bowman. CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing,

    2020

  5. [19]

    GPT-4 Technical Report, 2024.arXiv:2303.08774 [cs.CL]

    OpenAI. GPT-4 Technical Report, 2024.arXiv:2303.08774 [cs.CL]. OpenAI. OpenAI o1 System Card, 2024.arXiv:2412.16720 [cs.AI]. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Gray, John Sch...

    2024 arXiv

  6. [21]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever

    arXiv:1910.11779 [cs.LG]. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language Models are Unsupervised Multitask Learners,

    1910 arXiv

  7. [22]

    Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi

    arXiv:2405.04025 [cs.LG]. Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs. InThe Twelfth International Conference on Learning Representations,

    arXiv

  8. [24]

    17 A Datasets and Pre-Processing Table 1: Dataset split sizes used in the experiments. Dataset Train Validation Test Adult (Sections 6.1 and 6.3) 2000 2000 20000 Adult (Section 6.2) 100–20000 5000 20000 ACSIncome 10000 10000 20000 COMPAS 2000 1000 2000 BiasBios 20000 20000 500...

    2000

  9. [26]

    {column_name}: {value}

    For the experiments in Section 6.2 that examine performance on the Adult dataset under varying training sizes, we fix the test set size to 20000, validation to 5000, and vary the training set size across 100, 200, 400, 600, 800, 1000, 2000, 4000, 6000, 8000, 10000, and 20000 e...

    2000

  10. [27]

    workclass: State-gov

    typically populate the placeholders{col umn_name} and {value} with raw dataset encodings. For example, inAdult, the line “workclass: State-gov” means that the individual’s “Class of Worker” is “State government employee”. However, these codings are often terse or opaque, espec...

    2024

  11. [28]

    allowed fairness constraint violation

    = P(A ∈ I, Y = 0), FPR(I) = P( bY = 1 | A ∈ I, Y = 0), FPR is the weighted average FPR over all nonempty subgroups. C Featurizing LLM Predictions The final step of our framework in Section 4 involves transforming LLM predictions into a feature vector before applying the fair a...

    2018

  12. [512]

    Emptying

    We sweep the regularization strength λ over {10, 8, 6, 4, 3.5, 3, 2.5, 2, 1.5, 1, 0.7, 0.5, 0.3, 0.1, 0.05, 0.01, 0.0} (when λ = 0.0, the training objective reduces to only minimizing the cross entropy loss without fairness constraint). Following Prost et al. (2019), we use th...

    2019

  13. [2013]

    Bayes-Optimal Fair Classification with Linear Disparity Constraints via Pre-, In-, and Post-processing, 2024.arXiv:2402.02817 [stat.ML]

    Xianli Zeng, Guang Cheng, and Edgar Dobriban. Bayes-Optimal Fair Classification with Linear Disparity Constraints via Pre-, In-, and Post-processing, 2024.arXiv:2402.02817 [stat.ML]. Han Zhao. Costs and benefits of fair regression.Transactions on Machine Learning Research,

    2024 arXiv

  14. [2016]

    James Atwood, Nino Scherrer, Preethi Lahoti, Ananth Balashankar, Flavien Prost, and Ahmad Beirami

    URL https://www.propublica.org/article/machine-bias-risk-assessments-in-crimina l-sentencing. James Atwood, Nino Scherrer, Preethi Lahoti, Ananth Balashankar, Flavien Prost, and Ahmad Beirami. Inducing Group Fairness in Prompt-Based Language Model Decisions. InICLR 2025 Worksh...

    2025

  15. [2019]

    Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification

    Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification. InCompanion Proceedings of The 2019 World Wide Web Conference,

    2019

  16. [2020]

    Building Classifiers with Independency Constraints

    Toon Calders, Faisal Kamiran, and Mykola Pechenizkiy. Building Classifiers with Independency Constraints. In 2009 IEEE International Conference on Data Mining Workshops,

    2009

  17. [2021]

    Balancing out Bias: Achieving Fairness Through Balanced Training

    Xudong Han, Timothy Baldwin, and Trevor Cohn. Balancing out Bias: Achieving Fairness Through Balanced Training. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,

    2022

  18. [2022]

    AliciaParrish, AngelicaChen, NikitaNangia, VishakhPadmakumar, JasonPhang, JanaThompson, Phu Mon Htut, and Samuel R. Bowman. BBQ: A Hand-Built Bias Benchmark for Question Answering. InFindings of the Association for Computational Linguistics: ACL 2022,

    2022

  19. [2023]

    Chi, and Alex Beutel

    Sahaj Garg, Vincent Perot, Nicole Limtiaco, Ankur Taly, Ed H. Chi, and Alex Beutel. Counterfac- tual Fairness in Text Classification through Robustness. InProceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society,

    2019

  20. [2024]

    Evgenii Chzhen, Christophe Denis, Mohamed Hebiri, Luca Oneto, and Massimiliano Pontil

    arXiv:2412.04642 [cs.LG]. Evgenii Chzhen, Christophe Denis, Mohamed Hebiri, Luca Oneto, and Massimiliano Pontil. Fair Regression with Wasserstein Barycenters. InAdvances in Neural Information Processing Systems, volume 33,

    arXiv

  21. [2025]

    Griffiths

    Xuechunzi Bai, Angelina Wang, Ilia Sucholutsky, and Thomas L. Griffiths. Measuring Implicit Bias in Explicitly Unbiased Large Language Models, 2024.arXiv:2402.04105 [cs.CY]. Ioana Baldini, Dennis Wei, Karthikeyan Natesan Ramamurthy, Moninder Singh, and Mikhail Yurochkin. Your ...

    2024 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.