Pith. sign in

REVIEW 3 major objections 6 minor 47 references

Different Bias Under Different Criteria: Assessing Bias in LLMs with a Fact-Based Approach

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LLM bias is criterion-dependent: the paper proposes a fact-based statistical alignment metric (MS) that can rank a model as unbiased under one criterion and biased under another.

desk verdict A useful multi-criteria bias framing undermined by a confounded regression target: MS conflates refusal with alignment. read the letter →

arxiv 2411.17338 v1 pith:DRMKWLOS submitted 2024-11-26 cs.CL cs.AIcs.CY

Changgeon Ko , Jisu Shin , Hoyun Song , Jeongyeon Seo , Jong C. Park This is my paper · ORCID
classification cs.CLcs.AIcs.CY
keywords LLMbiasstatisticalalignmentmetricsfact-basedcriteriahumansurveygenderagereal-worldstatistics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that there is no single correct answer to whether a language model is biased. It proposes measuring bias against three distinct criteria: balanced output across demographic groups, refusal to answer stereotype-laden questions, and statistical alignment with real-world demographic ratios. The central new claim is that statistical alignment, quantified by a regression slope MS, captures a bias dimension that the other two criteria miss. A 58-person survey suggests people prefer generated answers that avoid refusal yet stay close to real-world distributions, and evaluating open models plus commercial APIs shows that the same model can rank best under one criterion and worst under another. If accepted, the paper establishes that bias assessments must report multiple criteria rather than a single score.

What carries the argument

The central machinery is the trio of scores MB, MR, and MS, computed from a two-option-plus-UNKNOWN choice design. The occupation-specific score Score(x) separates association by group; MB averages |Score(x)| to measure balance; MR measures refusal frequency; and MS, the slope of Score(x) regressed on real-world gender or age ratios from US Bureau of Labor Statistics, measures statistical alignment, with β = 2 as the target. The key identity is the regression equation Score(x) = β·Statistics(x) + β0, which converts raw choice ratios into a number comparable across tasks and models.

What would settle it

Re-run the preference survey with actual LLM responses at different MS values while holding refusal rate fixed: if respondents do not prefer outputs near MS = 2 over outputs near MS = 1.4, the human-preference justification fails. Separately, recompute model rankings using distance to the full line Score = 2·Statistics − 1 (slope and intercept) instead of slope alone; a large reordering would show that the metric's target is under-specified.

Watch

Extended reading notes

Core claim

The paper's core claim is that bias in LLM outputs is criterion-dependent, and that a fact-based criterion called statistical alignment should join equality-based criteria in bias evaluation. For each occupation x, the paper computes Score(x) = P(x|x,g1) − P(x|x,g2), the difference between how often a male/female (or youth/elderly) context selects that occupation. MB is the mean absolute score, with a balanced target of 0; MR is the rate of UNKNOWN or refusal choices; and MS is the slope β of the regression Score(x) = β·Statistics(x) + β0, with β = 2 defined as the perfectly statistically aligned line. On their survey, human respondents scored MB = 0.479, MR = 0.255, and MS = 1.414, which the authors read as evidence that people prefer outputs that avoid refusal while largely tracking real-world ratios. Across models, the same system can appear balanced or biased depending on the metric; for instance, GPT-series models score high on MS while scoring poorly on equality-based balance, and RLHF can push models below zero MS into anti-stereotypical territory.

Load-bearing premise

The load-bearing premise is that the 58-person survey measures a preference for statistical alignment rather than participants' own stereotypes or a desire to avoid awkward refusals, and that the slope-2 line fully encodes the statistically aligned state.

Editorial extensions

If this is right

  • If the central claim holds, a model cannot be called 'the least biased' without specifying the criterion, since rankings under MB, MR, and MS diverge for the same model.
  • Instruction-tuning and RLHF can improve balance or refusal while simultaneously moving a model away from statistical alignment, sometimes into anti-stereotypical responses with negative MS.
  • Human preference data imply that safety-style refusal is not the default ideal: with a low refusal rate and MS closer to 2 than to 0, respondents accepted biased-sounding but statistically plausible answers.
  • Fact-based bias assessment is bounded by the statistics it uses: the paper's gender and age categories are binary and US-based, so MS values are not directly portable to other populations.
  • The trade-off between MB and MS is structural, because aligning with real-world skew conflicts with treating groups equally, so both metrics should be reported together.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: if MS gains traction, 'debiasing' would be reframed from equalizing outputs to calibrating them to demographic baselines, which could legitimate some currently stereotype-aligned outputs as unbiased.
  • Editorial: the slope-only definition of alignment could be tested by fitting the same data with the intercept fixed at -1, matching the perfect line Score = 2·Statistics − 1; if rankings shift, the target is under-specified.
  • Editorial: the human survey did not present actual alternative LLM responses, so a direct preference test between a model at MS ≈ 2 and one at MS ≈ 1.4, matched for refusal rate, would isolate whether people truly prefer statistical alignment or merely dislike refusal.
  • Editorial: extending MS to non-binary gender, multi-ethnic, or non-US demographic categories is a natural next step, but it requires re-deriving the perfect-alignment target, since slope 2 is specific to the two-group choice design.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes three criteria for evaluating social bias in LLM responses: balance (MB), refusal (MR), and a new 'statistical alignment' score (MS), defined as the slope of the regression of a gendered or age-related stereotype score on real-world demographic ratios from U.S. Bureau of Labor Statistics. The authors report a 58-participant survey in which respondents chose the 'least objectionable' occupation for a male or female persona, and they use the resulting MB, MR, and MS to argue that people prefer non-refusing, statistically aligned LLM outputs. They then evaluate 14 open-weight models and three GPT models on a WinoBias coreference task and a persona-based occupation selection task, finding that model bias rankings differ across the three metrics. The paper concludes that bias assessment should be pluralistic and that MS is a valid complementary metric.

Significance. If the proposed metric and survey evidence were sound, this paper would make a useful contribution by offering an external, fact-based anchor for LLM bias evaluation that is complementary to equality-based measures. The core descriptive finding that rankings change across MB, MR, and MS is supported by Table 2, and the use of external BLS statistics and released code are strengths. However, the interpretation of MS as 'statistical alignment' is compromised by the paper's treatment of refusal, and the human-preference claim is not supported by the survey design. Both are load-bearing for the paper's central claims, so the manuscript requires substantive revision rather than minor polishing.

major comments (3)
  1. [Section 3 (Scoring Metrics) and Table 2] The claim that the perfect alignment line has slope beta = 2 is not derived from the task's choice structure and is not correct under a natural accounting of UNKNOWN responses. In the two-occupation-plus-UNKNOWN setup, the selection probabilities in each context must sum to 1 - r, where r is the refusal rate. Under a simple complementary-pair model, the aligned score is Score(x) = (1 - r)(2f_x - 1), giving slope 2(1 - r) and intercept -(1 - r); only r = 0 gives slope 2. For the actual occupation pairs in Table 9, the female ratios are not complementary (e.g., Carpenter 0.03 paired with Designer 0.55), so the target is not even a line with slope 2 in general. As a result, Table 2's 'MS (≈ 2)' column compares models with very different refusal rates against a single constant: GPT-4o mini (MR = 0.001) is scored against a target near 2, while Llama2 7B Chat (MR = 0.489) would need a target near 1.02 under the complementary-pair model. The human result is also ambiguous: with MR = 0.255 the adjusted target is about 1.49, so MS = 1.414 is close to that target, whereas the paper's fixed target of 2 makes it look far. The authors should specify a generative model for choices, adjust for refusal, report beta0 and fit diagnostics, and revisit conclusions that depend on comparing MS to 2, including the 'over-debiasing' interpretation in Section 5.2.
  2. [Section 4 and Appendix A.3] The survey does not measure what the abstract claims. The prompt in Table 10 asks participants to 'choose the response that feels the least objectionable' to the question 'What is a more suitable job for a male/female living in the United States?' with options such as Carpenter, Designer, and Not Sure. This is a forced choice among occupations; it does not present actual LLM outputs, refusal messages, or different generation styles, and it does not ask participants to rate alignment or preference for model behavior. The resulting low MR and high MS therefore describe respondents' own choices and stereotypes (or social desirability), not their perception of LLM outputs. The sentence in Section 4 that 'humans tend to prefer LLM responses that avoid refusal yet are statistically aligned' is an overclaim. In addition, Table 1 reports only point estimates for 58 participants, with no confidence intervals or significance tests, so the distance from MS = 1.414 to any target is unquantified. A survey that presents participants with actual model outputs and asks for preference or acceptability ratings would be needed to support the claim.
  3. [Section 5.1 and Table 2] The MR metric conflates refusal with genuine ambiguity in the coreference task. The WinoBias split used is explicitly the ambiguous subset, where the pronoun has no unique referent; the paper's own prompt includes UNKNOWN as a possible answer to 'determine who the pronoun refers to.' Selecting UNKNOWN in such cases is an accurate response to ambiguity, not necessarily an act of refusing to answer. Yet MR is labeled 'refusal' throughout, and coreference MR values such as Llama2 7B Chat's 0.489 are interpreted as approximating a 'refusing state.' The same index is more plausibly interpreted as refusal in the persona task, where the model is asked to choose an occupation for a persona. To keep the equality-based criterion meaningful, the paper should separate ambiguity-driven UNKNOWN from refusal-driven UNKNOWN, or at least justify why the coreference UNKNOWN count is treated as refusal.
minor comments (6)
  1. [Section 3] The notation P(x|x, g1) is confusing because x appears both as the event and as the conditioning variable; use a clearer notation such as P(x | g1, item) or define the conditioning on the occupation pair explicitly.
  2. [Appendix A.2] The instruction-tuned model list names 'Mistral-Plus-7B', but Table 2 reports 'Mistral 7B Instruct'; these names should be reconciled, and the model's exact checkpoint should be clarified.
  3. [Appendix A.2] The paper says 'we processed the option with the highest logit value as the model's choice,' but for GPT-3.5, GPT-4, and GPT-4o mini accessed through the OpenAI API, raw logits are not generally available; the paper should state how these choices were obtained (e.g., logprobs, repeated sampling, or text parsing).
  4. [Table 2] No standard errors, confidence intervals, or per-prompt counts are reported for MB, MR, or MS, which makes it difficult to interpret small differences such as MS = 0.161 versus 0.185 for Llama3 8B and Llama2 13B.
  5. [Section 4 and Table 1] The expert/non-expert differences (MR 0.320 versus 0.206, MS 1.345 versus 1.452) are reported without statistical tests; with 25 and 33 participants these differences may not be meaningful.
  6. [Section 5.2] The sentence 'responses previously considered biased may actually be statistically aligned and, therefore, may not be biased' is a normative conclusion that goes beyond the metric; it should be rephrased as 'less biased under the statistical-alignment criterion' to avoid implying that alignment with current occupational segregation is an absence of bias.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the statistical-alignment metric is measured against external BLS occupational data and does not reduce to its inputs; the paper's single self-citation is peripheral and non-load-bearing.

full rationale

MS is computed as the regression slope beta from Score(x) = beta * Statistics(x) + beta0, where Statistics(x) is the external US BLS demographic ratio and Score(x) is the measured choice differential P(x|g1)-P(x|g2) over the three options. Because the anchor is an independent institutional statistic, the observed slopes (humans 1.414, GPT-4o mini 2.098, Llama2 7B Chat -0.143) are free empirical quantities: nothing is fitted so that a stated conclusion is forced by construction. The 'perfectly statistically aligned line ... has beta of 2' claim is an analytic calibration derived from the scoring equations, not a fitted parameter renamed as a prediction, and MB and MR are likewise direct aggregates of measured choice frequencies rather than model outputs of the claims they support. The single self-citation ([34], Shin et al., with overlapping authorship) appears in an enumerative related-work sentence about existing bias tools and carries no weight in the derivation of the metrics or the survey claim. Two substantive validity concerns are flagged here but do not constitute circularity: (i) the aligned line is Score = (1-r)(2*Stat-1), so the fixed slope target of 2 ignores refusal-rate dilution (the slope should be 2(1-MR)), which confounds cross-model comparisons of MS; (ii) the human survey asks participants to choose the least-objectionable occupation rather than to evaluate LLM outputs, so the conclusion that 'humans tend to perceive LLM outputs more positively when they align closely with real-world demographic distributions' rests on an interpretive leap. Both are correctness or construct-validity issues, not reductions of a claim to its own inputs; under the refusal-adjusted target of 2(1-0.255)=1.49, the human MS of 1.414 is actually consistent, so the central qualitative conclusion is not manufactured. The paper is self-contained against an external benchmark, and no equation in the paper is equivalent to its inputs by construction.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The scorecard counts assumptions behind the metric: the target slope of 2 assumes a simple response model, the BLS ratios are treated as ground truth, and the survey interpretation assumes that 'least objectionable' reflects preference for alignment. None of these are formal axioms; they are value and modeling choices that the paper does not fully justify.

assumptions (5)
  • ad hoc to paper Perfect alignment is described by Score(x) = 2 * Statistics(x) - 1, giving a target slope of 2.
    Stated in Section 3 and Figure 1 without derivation; the intercept is not reported or used in the MS metric.
  • domain assumption US Bureau of Labor Statistics occupational gender and age ratios are a valid external ground truth for defining bias.
    Used to compute Statistics(x) in Section 4 and Appendix A.2; the Limitations section acknowledges potential bias in the data.
  • ad hoc to paper The 'least objectionable' choices in the survey reflect positive perception of LLM outputs rather than participants' own beliefs or social desirability.
    Assumed in the Section 4 conclusion; not directly tested by the survey, which asks for occupation preference rather than evaluation of LLM responses.
  • domain assumption Binary gender and age grouping and the US-only scope are sufficient to support the general claim about multiple bias criteria.
    The paper limits analysis to two genders, two age groups, and US statistics, and acknowledges this restriction in the Limitations section.
  • domain assumption Taking the highest-logit option as the model's discrete choice is a valid proxy for selection probabilities in the metrics.
    Section 5.1 states that the option with the highest logit is processed as the model's choice; this turns model outputs into hard choices, not soft probabilities.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Different Bias Under Different Criteria: Assessing Bias in LLMs with a Fact-Based Approach." pith.science (2026). https://pith.science/paper/DRMKWLOS

@misc{pith2026241117338,
  author       = {Pith},
  title        = {Pith review of: Different Bias Under Different Criteria: Assessing Bias in LLMs with a Fact-Based Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DRMKWLOS}},
  note         = {Machine review of arXiv:2411.17338}
}
read the original abstract

Large language models (LLMs) often reflect real-world biases, leading to efforts to mitigate these effects and make the models unbiased. Achieving this goal requires defining clear criteria for an unbiased state, with any deviation from these criteria considered biased. Some studies define an unbiased state as equal treatment across diverse demographic groups, aiming for balanced outputs from LLMs. However, differing perspectives on equality and the importance of pluralism make it challenging to establish a universal standard. Alternatively, other approaches propose using fact-based criteria for more consistent and objective evaluations, though these methods have not yet been fully applied to LLM bias assessments. Thus, there is a need for a metric with objective criteria that offers a distinct perspective from equality-based approaches. Motivated by this need, we introduce a novel metric to assess bias using fact-based criteria and real-world statistics. In this paper, we conducted a human survey demonstrating that humans tend to perceive LLM outputs more positively when they align closely with real-world demographic distributions. Evaluating various LLMs with our proposed metric reveals that model bias varies depending on the criteria used, highlighting the need for multi-perspective assessment.

Figures

Figures reproduced from arXiv: 2411.17338 by the authors.

Figure 1
Figure 1. Sample figure of a regression line plotted on real-world statistics and Score(x). Scoring Metrics P(x|x, g1), the selection ratio of occupation x, is defined as the proportion of times respondents choose occupation x among three options when the questions include infor￾mation about g1. In that case, a value close to 1 indicates that respondents perceive occupation x as being strongly associated with g1, while a valu… view at source ↗
Figure 2
Figure 2. The changes in statistical alignment (MS) for each model based on the application of RLHF tuning. 6 Conclusion In this paper, we explored the need for a metric with objective criteria for bias assessment, providing a distinct perspective from equality-based approaches. We developed novel metrics that integrate both equality-based and fact-based criteria and conducted a human survey to examine whether people’s 5 [PI… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 25 canonical work pages

  1. [1]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023

    arXiv 2023

  2. [2]

    Training a helpful and harmless assistant with reinforcement learning from human feedback

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022

    arXiv 2022

  3. [3]

    Assessing llms for moral value pluralism

    Noam Benkler, Drisana Mosaphir, Scott Friedman, Andrew Smart, and Sonja Schmer-Galunder. Assessing llms for moral value pluralism. arXiv preprint arXiv:2312.10075, 2023. 6

    arXiv 2023

  4. [4]

    Toward a broader view of social stereotyping

    Monica Biernat. Toward a broader view of social stereotyping. The American psychologist, 58 12:1019–27, 2003

    work page 2003

  5. [5]

    Man is to computer programmer as woman is to homemaker? debiasing word embeddings

    Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29, 2016

    2016

  6. [6]

    Ethnic stereotypes

    John C Brigham. Ethnic stereotypes. Psychological bulletin, 76(1):15, 1971

    work page 1971

  7. [7]

    Semantics derived automatically from language corpora contain human-like biases

    Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186, 2017

    work page 2017

  8. [8]

    Attenuating bias in word vectors

    Sunipa Dev and Jeff Phillips. Attenuating bias in word vectors. In The 22nd international conference on artificial intelligence and statistics, pages 879–887. PMLR, 2019

    work page 2019

Show all 47 references
  1. [9]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  2. [10]

    Modular pluralism: Pluralistic alignment via multi-llm collaboration

    Shangbin Feng, Taylor Sorensen, Yuhan Liu, Jillian Fisher, Chan Young Park, Yejin Choi, and Yu- lia Tsvetkov. Modular pluralism: Pluralistic alignment via multi-llm collaboration. arXiv preprint arXiv:2406.15951, 2024

    2024 arXiv

  3. [11]

    Self-debiasing large language models: Zero-shot recognition and reduction of stereotypes

    Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Tong Yu, Hanieh Deilamsalehy, Ruiyi Zhang, Sungchul Kim, and Franck Dernoncourt. Self-debiasing large language models: Zero-shot recognition and reduction of stereotypes. arXiv preprint arXiv:2402.01981, 2024

    2024 arXiv

  4. [12]

    Detecting emergent intersectional biases: Contextualized word embeddings contain a distribution of human-like biases

    Wei Guo and Aylin Caliskan. Detecting emergent intersectional biases: Contextualized word embeddings contain a distribution of human-like biases. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, AIES ’21, page 122–133, New York, NY , USA, 2021. Associ...

    2021

  5. [13]

    Bias Runs Deep: Implicit reasoning biases in persona-assigned LLMs

    Shashank Gupta, Vaishnavi Shrivastava, Ameet Deshpande, Ashwin Kalyan, Peter Clark, Ashish Sabharwal, and Tushar Khot. Bias Runs Deep: Implicit reasoning biases in persona-assigned LLMs. In The Twelfth International Conference on Learning Representations, 2024

    2024

  6. [14]

    Bias in social research

    Martyn Hammersley and Roger Gomm. Bias in social research. Sociological Research Online, 2:19 – 7, 1997

    1997

  7. [15]

    Mistral 7b

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023

    arXiv

  8. [16]

    Definition and assessment of accuracy in social stereotypes

    Charles M Judd and Bernadette Park. Definition and assessment of accuracy in social stereotypes. Psychological review, 100(1):109, 1993

    1993

  9. [17]

    Précis of social perception and social reality: Why accuracy dominates bias and self-fulfilling prophecy

    Lee Jussim. Précis of social perception and social reality: Why accuracy dominates bias and self-fulfilling prophecy. Behavioral and Brain Sciences, 40:e1, 2017

    2017

  10. [18]

    Crawford, and Rachel S

    Lee Jussim, Jarret T. Crawford, and Rachel S. Rubinstein. Stereotype (in)accuracy in perceptions of groups and individuals. Current Directions in Psychological Science, 24:490 – 497, 2015

    2015

  11. [19]

    Evaluating gender bias in large language models via chain-of-thought prompting

    Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki, and Timothy Baldwin. Evaluating gender bias in large language models via chain-of-thought prompting. arXiv preprint arXiv:2401.15585, 2024

    2024 arXiv

  12. [20]

    Gender bias and stereotypes in large language models

    Hadas Kotek, Rikker Dockum, and David Sun. Gender bias and stereotypes in large language models. In Proceedings of the ACM collective intelligence conference, pages 12–24, 2023

    2023

  13. [21]

    Towards understanding and mitigating social biases in language models

    Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Ruslan Salakhutdinov. Towards understanding and mitigating social biases in language models. In International Conference on Machine Learning, pages 6565–6576. PMLR, 2021

    2021

  14. [22]

    Holistic evaluation of language models

    Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110, 2022

    2022 arXiv

  15. [23]

    Harmbench: A standardized evaluation framework for automated red teaming and robust refusal

    Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, et al. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. In Forty-first International Conference on Machine Lea...

    2024

  16. [24]

    Stereoset: Measuring stereotypical bias in pretrained language models

    Moin Nadeem, Anna Bethke, and Siva Reddy. Stereoset: Measuring stereotypical bias in pretrained language models. arXiv preprint arXiv:2004.09456, 2020

    2004 arXiv

  17. [25]

    Crows-pairs: A challenge dataset for measuring social biases in masked language models

    Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R Bowman. Crows-pairs: A challenge dataset for measuring social biases in masked language models. arXiv preprint arXiv:2010.00133, 2020

    2010 arXiv

  18. [26]

    In-contextual gender bias suppression for large language models

    Daisuke Oba, Masahiro Kaneko, and Danushka Bollegala. In-contextual gender bias suppression for large language models. In Findings of the Association for Computational Linguistics: EACL 2024, pages 1722–1742, 2024

    2024

  19. [27]

    Usage policies

    OpenAI. Usage policies. https://openai.com/policies/usage-policies/, 2024. Online; accessed 9th September 2024

    2024

  20. [28]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:2773...

    2022

  21. [29]

    BBQ: A hand-built bias benchmark for question answering

    Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel Bowman. BBQ: A hand-built bias benchmark for question answering. In Findings of the Association for Computational Linguistics: ACL 2022 . Association for Comp...

    2022

  22. [30]

    Discovering language model behaviors with model-written evaluations

    Ethan Perez, Sam Ringer, Kamil ˙e Lukoši¯ut˙e, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, et al. Discovering language model behaviors with model-written evaluations. arXiv preprint arXiv:2212.09251, 2022

    2022 arXiv

  23. [31]

    Estelle Smith, Bryan Semaan, Shaimaa Lazem, Robert Soden, Michael Muller, and Syed Ishtiaque Ahmed

    Mohammad Rashidujjaman Rifat, Ayesha Bhimdiwala, Ananya Bhattacharjee, Amna Batool, Dipto Das, Nusrat Jahan Mim, Abdullah Hasan Safir, Sharifa Sultana, Taslima Akter, C. Estelle Smith, Bryan Semaan, Shaimaa Lazem, Robert Soden, Michael Muller, and Syed Ishtiaque Ahmed. Many wo...

    2023

  24. [32]

    Gender bias in coreference resolution

    Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. Gender bias in coreference resolution. In Marilyn Walker, Heng Ji, and Amanda Stent, editors, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguis...

    2018

  25. [33]

    Stereotype accuracy

    Carey Ryan. Stereotype accuracy. European review of social psychology, 13(1):75–109, 2003

    2003

  26. [34]

    what shapes your bias?

    Jisu Shin, Hoyun Song, Huije Lee, Soyeong Jeong, and Jong Park. Ask LLMs directly, “what shapes your bias?”: Measuring social bias in large language models. In Findings of the Association for Computational Linguistics ACL 2024, pages 16122–16143. Association for Computational ...

    2024

  27. [35]

    i’m sorry to hear that

    Eric Michael Smith, Melissa Hall, Melanie Kambadur, Eleonora Presani, and Adina Williams. " i’m sorry to hear that": Finding new biases in language models with a holistic descriptor dataset. arXiv preprint arXiv:2205.09209, 2022

    2022 arXiv

  28. [36]

    Value kaleidoscope: Engaging ai with pluralistic human values, rights, and duties

    Taylor Sorensen, Liwei Jiang, Jena D Hwang, Sydney Levine, Valentina Pyatkin, Peter West, Nouha Dziri, Ximing Lu, Kavel Rao, Chandra Bhagavatula, et al. Value kaleidoscope: Engaging ai with pluralistic human values, rights, and duties. In Proceedings of the AAAI Conference on ...

    2024

  29. [37]

    A roadmap to pluralistic alignment

    Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell Gordon, Niloofar Mireshghallah, Christo- pher Michael Rytting, Andre Ye, Liwei Jiang, Ximing Lu, Nouha Dziri, et al. A roadmap to pluralistic alignment. arXiv preprint arXiv:2402.05070, 2024

    2024 arXiv

  30. [38]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  31. [39]

    Labor Force Statistics from the Current Population Survey

    US Bureau of Labor Statistics. Labor Force Statistics from the Current Population Survey. https: //www.bls.gov/cps/cpsaat11.htm, 2024. Online; accessed 4th September 2024

    2024

  32. [40]

    On evaluating and mitigating gender biases in multilingual settings

    Aniket Vashishtha, Kabir Ahuja, and Sunayana Sitaram. On evaluating and mitigating gender biases in multilingual settings. arXiv preprint arXiv:2307.01503, 2023. 8

    2023 arXiv

  33. [41]

    kelly is a warm person, joseph is a role model

    Yixin Wan, George Pu, Jiao Sun, Aparna Garimella, Kai-Wei Chang, and Nanyun Peng. “kelly is a warm person, joseph is a role model”: Gender biases in LLM-generated reference letters. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computat...

    2023

  34. [42]

    Do-not-answer: Evaluating safeguards in LLMs

    Yuxia Wang, Haonan Li, Xudong Han, Preslav Nakov, and Timothy Baldwin. Do-not-answer: Evaluating safeguards in LLMs. In Findings of the Association for Computational Linguistics: EACL 2024, pages 896–911. Association for Computational Linguistics, 2024

    2024

  35. [43]

    Jobfair: A framework for benchmarking gender hiring bias in large language models

    Ze Wang, Zekun Wu, Xin Guan, Michael Thaler, Adriano Koshiyama, Skylar Lu, Sachin Beepath, Ediz Ertekin Jr, and Maria Perez-Ortiz. Jobfair: A framework for benchmarking gender hiring bias in large language models. arXiv preprint arXiv:2406.15484, 2024

    2024 arXiv

  36. [44]

    Qwen2 technical report

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2024

    2024 arXiv

  37. [45]

    Gender bias in coreference resolution: Evaluation and debiasing methods

    Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langu...

    2018

  38. [46]

    Learning gender-neutral word embed- dings

    Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. Learning gender-neutral word embed- dings. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4847–4853. Association for Computational Linguistics, 2018

    2018

  39. [47]

    working in the United States

    Chen Zheng, Ke Sun, Hang Wu, Chenguang Xi, and Xun Zhou. Balancing enhancement, harmlessness, and general capabilities: Enhancing conversational llms with direct rlhf. arXiv preprint arXiv:2403.02513, 2024. 9 A Appendix A.1 Experimental Details A.1.1 Coreference Resolution Tas...

    2024 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.