Pith. sign in

REVIEW 3 major objections 5 minor 53 references

Colombian Waitresses y Jueces canadienses: Gender and Country Biases in Occupation Recommendations from LLMs

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that intersectional country-gender bias in LLM occupation recommendations persists even when single-axis gender or country bias is absent, and that instruction tuning and prompt language shape that bias.

desk verdict Useful multilingual fairness benchmark; the intersectional headline is asserted, not measured, and output-length differences muddy the instruction-tuning result. read the letter →

arxiv 2505.02456 v2 pith:VIJKCCZ3 submitted 2025-05-05 cs.CL

Elisa Forcada Rodríguez , Olatz Perez-de-Viñaspre , Jon Ander Campos , Dietrich Klakow , Vagrant Gautam This is my paper · ORCID
classification cs.CL
keywords intersectionalbiasoccupationmultilingualfairnessgendercountrylargelanguagemodelsinstructiontuningdemographicparity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that large language models carry occupation stereotypes that depend jointly on a person's country of origin and their gender, and that these intersectional biases can persist even when a model shows no overall gender bias and no overall country bias. To test this, the authors built a benchmark of 300,000 prompts in English, Spanish, and German, varying 25 origin countries and four pronoun sets, and asked five Llama-family models to recommend jobs for a laid-off friend. They report significant single-axis and intersectional biases, with instruction-tuned models consistently the least biased, prompt language changing the size and shape of bias (Spanish showing the least), and host country playing at most a modulating role. If this is right, fairness evaluations that check only one axis or only English are not merely incomplete: they can certify models that still steer specific country-gender subgroups toward stereotyped work.

What carries the argument

The load-bearing object is a multilingual prompt benchmark. For each of three languages (English, Spanish, German), three templates ask for job recommendations for a friend who may have to return to one of 25 origin countries while currently in one of five host countries, and four pronoun sets stand in for gender (he/she/they/xe in English; él/ella/elle/singular ellos in Spanish; er/sie/xier/sier in German). Each of five Llama-based models is prompted 50 times per condition, giving about 300,000 generations; generations are cleaned and clustered into 22 U.S. Bureau of Labor Statistics occupation categories, and bias is quantified by an $L^2$ norm and Jensen-Shannon divergence against a uniform 'ideal' distribution. The analysis repeatedly compares marginal (single-axis) distributions with the joint country-by-gender distribution, and this joint comparison is what surfaces biases hidden in single-axis checks.

What would settle it

Run the same 300,000-prompt benchmark but score bias against actual employment distributions by country and gender instead of a uniform ideal, and check whether the intersectional gaps persist and whether instruction-tuned models still look best; alternatively, on models that show single-axis parity, test conditional independence of occupation from the country-gender pair with a chi-square or permutation test, because if no dependence remains after accounting for the marginal gender and country distributions, the claim of hidden intersectional bias falls.

Watch

Extended reading notes

Core claim

The paper claims to present the first multilingual study of intersecting country-gender bias in occupation recommendations, and its central discovery is that large language models exhibit intersectional country-gender occupational bias that is not reducible to gender bias or country bias alone. Using a benchmark of 300,000 prompts in English, Spanish, and German that vary 25 origin countries and four pronoun sets across five Llama-family models, the authors find that a model can show rough parity for gender or country taken separately while still channeling specific country-gender combinations into stereotyped occupations, such as low-status feminized jobs for women and non-binary people from some countries and high-status roles for men from Western countries. Instruction-tuned models (Alpaca and Llama3-Instruct) show the lowest and most stable bias levels, the prompt language changes the bias (Spanish least, English and German more), and host-country choice has at most a modulating effect. The paper concludes that single-axis, English-only fairness evaluations can miss compounded harms.

Load-bearing premise

The whole bias measurement rests on treating an equal spread of job categories across every country and gender group as the 'unbiased' ideal; if the right ideal is instead a distribution matching real labor markets or individual qualifications, the measured biases and model rankings could change.

Editorial extensions

If this is right

  • Fairness audits that report only gender parity or only country parity can certify models that still give stereotyped occupation advice to specific subgroups, so intersectional measurements should become standard in such audits.
  • Instruction tuning consistently lowered single-axis and intersectional bias in the tested suite, making tuning a concrete mitigation lever for occupation stereotyping even when it does not change model scale.
  • Bias measurements do not transfer across prompt languages: with the models that support all three languages, Spanish prompts showed the least bias, so a clean bill of health in English does not imply clean behavior in another language.
  • Host country had only a secondary, model-dependent effect, while prompt language and model family were the main drivers; this points to pretraining and prompting design as the active levers for bias rather than the user's location.
  • The released prompt framework, country set, and clustering pipeline can be reused to audit other models, languages, and occupation categories for the same intersectional effects.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If fairness is instead defined as matching real-world labor-force distributions rather than a uniform spread, the paper's bias scores and model rankings could change substantially; testing that alternative baseline is a direct next step the authors leave open.
  • The Spanish-language result hints that models may fixate less on pronouns in pro-drop languages because pronouns are often omitted in natural text; prompting in a language without grammatical gender or without pronouns entirely would test this mechanism.
  • The same joint-distribution method could be applied to other job-relevant attributes, such as race or disability, where single-axis audits remain common, and could reveal analogous compounded harms.
  • Because the evaluation covers only Llama-family models, the instruction-tuning advantage may not generalize to other architecture and alignment families; comparing against differently trained models would show how general the mitigation is.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies multilingual, intersectional country-gender biases in LLM occupation recommendations. The authors construct a benchmark of naturalistic prompts in English, Spanish, and German, varying 25 origin countries, four pronoun sets, and five host countries, and they prompt five Llama-family models (Llama2, Alpaca, Latxa, Llama3, Llama3-Instruct) a total of 300,000 times. Generated job titles are clustered into 22 BLS occupation categories, and bias is quantified using the L2 norm and Jensen-Shannon divergence to a uniform reference distribution, together with Mann-Whitney U tests and qualitative analysis. The main claims are that intersectional country-gender biases persist even when single-axis gender or country bias appears at parity; that instruction-tuning reduces single-axis and intersectional bias; and that prompt language affects bias, with Spanish showing the least bias.

Significance. If the claims hold, the paper would provide a useful, publicly released benchmark and a strong argument that fairness audits must go beyond single-axis and English-only evaluations. The scale of the probing (300k prompts, 25 countries, non-binary pronoun sets, open-ended generation) and the release of code and prompts are concrete strengths, as is the inclusion of multiple languages and intersectional subgroups. However, the central intersectional-persistence claim currently rests on qualitative inspection rather than a formal interaction test, and the instruction-tuning mitigation claim is confounded by systematic output-length differences between base and tuned models. The paper's contribution is therefore promising but needs substantial methodological work before the headline findings can be considered established.

major comments (3)
  1. [Section 5.2 (and abstract)] The headline claim that intersectional biases persist even when models show parity along a single demographic axis is not quantitatively supported. Section 3.4 defines L2 and JSD only as distances from a uniform distribution over 22 occupation clusters; there is no formal definition of 'single-axis parity' and no interaction statistic that separates intersectional from single-axis effects. The evidence in Section 5.2 is qualitative (e.g., the Latxa/Canada example), and subgroup-specific deviations from uniform can cancel in the marginals. The authors should formalize single-axis parity and test whether the joint country-by-gender distribution deviates significantly from the product of the marginals (or use an equivalent interaction test); without this, the paper's central novelty is not demonstrated.
  2. [Table 1 and Section 4.2] The claim that instruction-tuning mitigates bias is confounded by output length. Table 1 shows that Llama3-Instruct and Alpaca generate roughly 2.6 to 36 times more job tokens per prompt than their base counterparts. Because L2 and JSD are computed on pooled job counts across all generated jobs, longer outputs mechanically move the empirical distribution toward uniform simply because more samples are aggregated. The comparison between Llama3 and Llama3-Instruct therefore conflates instruction-tuning with increased output length. The authors should control for this, for example by computing metrics on the first recommended job, on a matched number of sampled jobs, or by reporting the distribution of top-1 recommendations. Without such a control, the instruction-tuning mitigation result is not identifiable.
  3. [Section 6.1 (with Limitations)] The finding that Spanish prompts yield the least bias is presented as a main result but depends entirely on the uniform-reference choice made in Section 3.4. The Limitations section candidly acknowledges that a different fairness norm could change the ranking, yet the paper does not test any alternative baseline. Because claims such as 'Spanish shows the least bias' are reference-dependent, the authors should either evaluate an alternative reference (e.g., a labor-market or empirically grounded baseline) or explicitly reframe the language finding as conditional on a demographic-parity definition. This is a standard robustness request, not a demand to adopt a different normative stance.
minor comments (5)
  1. [Section 3.2 and Appendix A.2] The decoding parameters (temperature, top-p, maximum generated tokens, random seed) are not reported. Since the paper samples each condition 50 times, these details are essential for reproducibility and for interpreting the output-length differences observed in Table 1.
  2. [Section 2] There is a typo in the related-work paragraph: 'age Zee et al. (2024)' should read 'age (Zee et al., 2024)' or similar, with a proper citation separator.
  3. [Figure 3 caption] The caption states that country-internal assignments are 'more evenly balanced, although this is harder to see visually'; this hedge suggests the figure does not fully support the claim. A quantitative annotation or a complementary figure would make the point clearer.
  4. [Section 6.2] The comparison between Llama2 and Latxa is said to study pre-training language, but the two models also differ in continued pre-training data and other factors; the claim that Latxa 'amplifies gendered assumptions when operating in grammatically gendered languages' is speculative given the confounds the authors themselves acknowledge. Consider softening the conclusion or adding a controlled condition.
  5. [Abstract and Introduction] The paper claims to be 'the first study of multilingual intersecting country and gender biases,' but Barriere and Cifuentes (2024) is cited as already examining country and gender in a multilingual context. The claim should be qualified, e.g., 'the first study of multilingual intersecting country and gender biases in generative occupation recommendations.'

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: behavior is measured against an explicit uniform reference, and the intersectional claim is a qualitative empirical finding rather than a tautology.

full rationale

The paper does not fit parameters to target data and then rename them as predictions; it computes L2 and JSD distances from model-generated occupation distributions to an explicitly chosen uniform reference. Section 3.4 states: "In both cases, we compare observed distributions to a reference distribution of perfect equality, i.e., a uniform distribution." This is an operationalization, not a circular derivation: the metrics are applied to raw model outputs, and the results are empirical observations under that definition. The Limitations section candidly acknowledges the normative nature of the reference: "it is not clear that this is the only distribution we can compare to... the ideal behaviour may not be to generate occupation names at all." Thus the central bias magnitudes are transparently defined rather than smuggled in by construction. The headline intersectional claim in Section 5.2 ("biases were not simply additive but compounded") is supported by qualitative examples such as Latxa associating masculine pronouns with project manager and feminine pronouns with caregiver for Canada. While the paper does not provide a formal interaction metric separating intersectional from single-axis bias, this is a measurement-validity gap, not circularity: the claim is not equivalent to its inputs by definition. Similarly, the observation that instruction-tuned models emit more job tokens (Table 1) is a potential confound for pooled distribution comparisons, but it is an empirical confound, not a circular step. The self-citations (e.g., Gautam et al. 2024c for name-proxy validity, Subramonian et al. 2025 for open-ended generation) are used to motivate methodological choices and do not carry the empirical conclusions. No uniqueness theorem or ansatz is imported from the authors' prior work to force the paper's choices. The paper is self-contained as a measurement study: every reported bias value is a direct function of model generations plus an openly stated fairness norm, and the limitations of that norm are disclosed in the manuscript itself.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper introduces no free parameters or invented entities. It rests on definitional assumptions, chiefly the uniform reference distribution, and on modeling choices such as pronoun-as-gender proxies and the BLS clustering scheme. These are stated in the paper but not independently validated, which raises the soundness risk.

assumptions (5)
  • domain assumption A uniform distribution across 22 occupation categories is the appropriate reference for 'unbiased' behavior.
    All bias metrics in Section 3.4 compute deviation from uniform; the paper acknowledges in Limitations that other fairness definitions exist.
  • domain assumption Pronoun sets are valid proxies for gender, including non-binary pronouns like xe, elle, and xier.
    Section 3.2 states pronouns are used as a proxy for gender and notes the relationship is not one-to-one.
  • domain assumption The 22 BLS occupation categories provide adequate coverage of the generated job titles.
    Section 3.3 selects these categories and reports 87.6% manual agreement for the clustering.
  • domain assumption Automatic translation and manual validation produced prompt templates that are behaviorally equivalent across English, Spanish, and German.
    Section 3.2 describes translation and native-speaker validation, but cross-lingual equivalence of the prompts is not measured.
  • domain assumption The command-r-plus model used for extracting job titles and clustering is reliable enough for the analysis.
    Sections A.3 and 3.3 rely on this API model; the paper validates clustering but not extraction quality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Colombian Waitresses y Jueces canadienses: Gender and Country Biases in Occupation Recommendations from LLMs." pith.science (2026). https://pith.science/paper/VIJKCCZ3

@misc{pith2026250502456,
  author       = {Pith},
  title        = {Pith review of: Colombian Waitresses y Jueces canadienses: Gender and Country Biases in Occupation Recommendations from LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VIJKCCZ3}},
  note         = {Machine review of arXiv:2505.02456}
}
read the original abstract

One of the goals of fairness research in NLP is to measure and mitigate stereotypical biases that are propagated by NLP systems. However, such work tends to focus on single axes of bias (most often gender) and the English language. Addressing these limitations, we contribute the first study of multilingual intersecting country and gender biases, with a focus on occupation recommendations generated by large language models. We construct a benchmark of prompts in English, Spanish and German, where we systematically vary country and gender, using 25 countries and four pronoun sets. Then, we evaluate a suite of 5 Llama-based models on this benchmark, finding that LLMs encode significant gender and country biases. Notably, we find that even when models show parity for gender or country individually, intersectional occupational biases based on both country and gender persist. We also show that the prompting language significantly affects bias, and instruction-tuned models consistently demonstrate the lowest and most stable levels of bias. Our findings highlight the need for fairness researchers to use intersectional and multilingual lenses in their work.

Figures

Figures reproduced from arXiv: 2505.02456 by the authors.

Figure 1
Figure 1. Examples of our multilingual evaluation of [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Map highlighting the 25 countries we select. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Occupation recommendations by country, from [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Occupation predictions by gender, from Latxa (left) and Llama3-Instruct (right) when prompted in English with USA as the host country. Latxa shows greater gender bias (e.g., there are clusters in which “she” is hardly present), even though it has a numerically more bal…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 18 canonical work pages

  1. [1]

    AI@Meta. 2024. https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md Llama 3 model card . Accessed: 2024-12-23

    work page 2024

  2. [2]

    Haozhe An, Christabel Acquaye, Colin Wang, Zongxia Li, and Rachel Rudinger. 2024. https://doi.org/10.18653/v1/2024.acl-short.37 Do large language models discriminate in hiring decisions on the basis of race, ethnicity, and gender? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 386...

    doi:10.18653/v1/2024.acl-short.37 2024

  3. [3]

    Valentin Barriere and Sebastian Cifuentes. 2024. https://aclanthology.org/2024.lrec-main.134/ Are text classifiers xenophobic? a country-oriented bias detection method with least confounding variables . In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 1511--...

    work page 2024

  4. [4]

    Jayadev Bhaskaran and Isha Bhallamudi. 2019. https://doi.org/10.18653/v1/W19-3809 Good secretaries, bad truck drivers? occupational gender stereotypes in sentiment analysis . In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, pages 62--68, Florence, Italy. Association for Computational Linguistics

    work page doi:10.18653/v1/w19-3809 2019

  5. [5]

    Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183--186

    2017

  6. [6]

    Ant \'o nio C \^a mara, Nina Taneja, Tamjeed Azad, Emily Allaway, and Richard Zemel. 2022. https://doi.org/10.18653/v1/2022.ltedi-1.11 Mapping the multilingual margins: Intersectional biases of sentiment analysis systems in E nglish, S panish, and A rabic . In Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion,...

    work page doi:10.18653/v1/2022.ltedi-1.11 2022

  7. [7]

    Cohere. 2024. https://docs.cohere.com/v2/docs/models Models documentation - cohere . Accessed: 2024-12-23

    work page 2024

  8. [8]

    Kirby Conrod. 2020. https://doi.org/10.1093/oxfordhb/9780190212926.013.63 Pronouns and gender in language . In The Oxford Handbook of Language and Sexuality. Oxford University Press

    arXiv 2020

Show all 53 references
  1. [9]

    Jeffrey Dastin. 2022. Amazon scraps secret ai recruiting tool that showed bias against women. In Ethics of data and analytics, pages 296--299. Auerbach Publications

    2022

  2. [10]

    Sunipa Dev, Masoud Monajatipoor, Anaelia Ovalle, Arjun Subramonian, Jeff Phillips, and Kai-Wei Chang. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.150 Harms of gender exclusivity and challenges in non-binary representation in language technologies . In Proceedings of the ...

    2021 doi

  3. [11]

    o rklund, and Henrik Bj \

    Hannah Devinney, Jenny Bj \"o rklund, and Henrik Bj \"o rklund. 2024. https://doi.org/10.18653/v1/2024.gebnlp-1.3 We don`t talk about that: Case studies on intersectional analysis of social bias in large language models . In Proceedings of the 5th Workshop on Gender Bias in Na...

    2024 doi

  4. [12]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of models . Preprint, arXiv:2407.21783

    2024 arXiv

  5. [13]

    Julen Etxaniz, Oscar Sainz, Naiara Miguel, Itziar Aldabe, German Rigau, Eneko Agirre, Aitor Ormazabal, Mikel Artetxe, and Aitor Soroa. 2024. https://doi.org/10.18653/v1/2024.acl-long.799 Latxa: An open language model and evaluation suite for B asque . In Proceedings of the 62n...

    2024 doi

  6. [14]

    Emilio Ferrara. 2023. Fairness and bias in artificial intelligence: A brief survey of sources, impacts, and mitigation strategies. Sci, 6(1):3

    2023

  7. [15]

    Foulds, Rashidul Islam, Kamrun Naher Keya, and Shimei Pan

    James R. Foulds, Rashidul Islam, Kamrun Naher Keya, and Shimei Pan. 2020. https://doi.org/10.1109/ICDE48307.2020.00203 An intersectional definition of fairness . In 2020 IEEE 36th International Conference on Data Engineering (ICDE), pages 1918--1921

    2020

  8. [16]

    Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. 2024. Bias and fairness in large language models: A survey. Computational Linguistics, pages 1--79

    2024

  9. [17]

    Vagrant Gautam, Eileen Bingert, Dawei Zhu, Anne Lauscher, and Dietrich Klakow. 2024 a . https://doi.org/10.1162/tacl_a_00719 Robust pronoun fidelity with E nglish LLM s: Are they reasoning, repeating, or just biased? Transactions of the Association for Computational Linguistic...

    2024 doi

  10. [18]

    Vagrant Gautam, Julius Steuer, Eileen Bingert, Ray Johns, Anne Lauscher, and Dietrich Klakow. 2024 b . https://doi.org/10.18653/v1/2024.crac-1.6 W ino P ron: Revisiting E nglish W inogender schemas for consistency, coverage, and grammatical case . In Proceedings of the Seventh...

    2024 doi

  11. [19]

    Vagrant Gautam, Arjun Subramonian, Anne Lauscher, and Os Keyes. 2024 c . https://doi.org/10.18653/v1/2024.gebnlp-1.20 Stop! in the name of flaws: Disentangling personal names and sociodemographic attributes in NLP . In Proceedings of the 5th Workshop on Gender Bias in Natural ...

    2024 doi

  12. [20]

    Maarten Grootendorst. 2022. Bertopic: Neural topic modeling with a class-based tf-idf procedure. arXiv preprint arXiv:2203.05794

    2022 arXiv

  13. [21]

    Krystal Hu. 2023. https://www.reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/ Chatgpt sets record for fastest-growing user base - analyst note . Reuters

    2023

  14. [22]

    HuggingFace. 2022. sentence-transformers/all-minilm-l6-v2. Retrieved December 29, 2024 from https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2

    2022

  15. [23]

    Svetlana Kiritchenko and Saif Mohammad. 2018. https://doi.org/10.18653/v1/S18-2005 Examining gender and race bias in two hundred sentiment analysis systems . In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pages 43--53, New Orleans, Louis...

    2018 doi

  16. [24]

    John Lalor, Yi Yang, Kendall Smith, Nicole Forsgren, and Ahmed Abbasi. 2022. https://doi.org/10.18653/v1/2022.naacl-main.263 Benchmarking intersectional biases in NLP . In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Lin...

    2022 doi

  17. [25]

    Rong Li, Ashwini Kamaraj, Jing Ma, and Sarah Ebling. 2024. https://doi.org/10.18653/v1/2024.nlp4pi-1.22 Decoding ableism in large language models: An intersectional approach . In Proceedings of the Third Workshop on NLP for Positive Impact, pages 232--249, Miami, Florida, USA....

    2024 doi

  18. [26]

    Weicheng Ma, Brian Chiang, Tong Wu, Lili Wang, and Soroush Vosoughi. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.575 Intersectional stereotypes in large language models: Dataset and analysis . In Findings of the Association for Computational Linguistics: EMNLP 2023, ...

    2023 doi

  19. [27]

    Bowman, and Rachel Rudinger

    Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019. https://doi.org/10.18653/v1/N19-1063 On measuring social biases in sentence encoders . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational...

    2019 doi

  20. [28]

    Leland McInnes, John Healy, and James Melville. 2018. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426

    2018 arXiv

  21. [29]

    Niloofar Mireshghallah, Maria Antoniak, Yash More, Yejin Choi, and Golnoosh Farnadi. 2024. https://openreview.net/forum?id=tIpWtMYkzU Trust no bot: Discovering personal disclosures in human- LLM conversations in the wild . In First Conference on Language Modeling

    2024

  22. [30]

    Pranav Narayanan Venkit, Sanjana Gautam, Ruchi Panchanadikar, Ting-Hao Huang, and Shomir Wilson. 2023. https://doi.org/10.18653/v1/2023.eacl-main.9 Nationality bias in text generation . In Proceedings of the 17th Conference of the European Chapter of the Association for Comput...

    2023 doi

  23. [31]

    NLTK. 2024. Natural language toolkit (nltk). Retrieved December 24, 2024 from https://www.nltk.org/index.html

    2024

  24. [32]

    Anaelia Ovalle, Arjun Subramonian, Vagrant Gautam, Gilbert Gee, and Kai-Wei Chang. 2023. https://doi.org/10.1145/3600211.3604705 Factoring the matrix of domination: A critical review and reimagination of intersectionality in ai fairness . In Proceedings of the 2023 AAAI/ACM Co...

    2023

  25. [33]

    Martine Paris. 2025. https://www.forbes.com/sites/martineparis/2025/04/12/chatgpt-hits-1-billion-users-openai-ceo-says-doubled-in-weeks/ Chatgpt hits 1 billion users? ‘doubled in just weeks’ says openai ceo

    2025

  26. [34]

    Rajesh Ranjan, Shailja Gupta, and Surya Narayan Singh. 2024. A comprehensive survey of bias in llms: Current landscape and future directions. arXiv preprint arXiv:2409.16430

    2024 arXiv

  27. [35]

    Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. https://doi.org/10.18653/v1/N18-2002 Gender bias in coreference resolution . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computational Linguistics: H...

    2018 doi

  28. [36]

    Abel Salinas, Parth Shah, Yuzhong Huang, Robert McCormack, and Fred Morstatter. 2023. https://doi.org/10.1145/3617694.3623257 The unequal opportunities of large language models: Examining demographic biases in job recommendations by chatgpt and llama . In Proceedings of the 3r...

    2023

  29. [37]

    Abhilasha Sancheti, Haozhe An, and Rachel Rudinger. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.29 On the influence of gender and race in romantic relationship prediction from large language models . In Proceedings of the 2024 Conference on Empirical Methods in Natural L...

    2024 doi

  30. [38]

    Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2024. https://openreview.net/forum?id=RIu5lyNXjT Quantifying language models' sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting . In The Twelfth International...

    2024

  31. [39]

    Karolina Stanczak and Isabelle Augenstein. 2021. https://arxiv.org/abs/2112.14168 A survey on gender bias in natural language processing . Preprint, arXiv:2112.14168

    2021 arXiv

  32. [40]

    Smith, and Luke Zettlemoyer

    Gabriel Stanovsky, Noah A. Smith, and Luke Zettlemoyer. 2019. https://doi.org/10.18653/v1/P19-1164 Evaluating gender bias in machine translation . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1679--1684, Florence, Italy. Ass...

    2019 doi

  33. [41]

    Arjun Subramonian, Vagrant Gautam, Preethi Seshadri, Dietrich Klakow, Kai-Wei Chang, and Yizhou Sun. 2025. https://arxiv.org/abs/2504.17075 Agree to disagree? a meta-evaluation of llm misgendering . Preprint, arXiv:2504.17075

    2025 arXiv

  34. [42]

    Zeerak Talat, Aur \'e lie N \'e v \'e ol, Stella Biderman, Miruna Clinciu, Manan Dey, Shayne Longpre, Sasha Luccioni, Maraim Masoud, Margaret Mitchell, Dragomir Radev, Shanya Sharma, Arjun Subramonian, Jaesung Tae, Samson Tan, Deepak Tunuguntla, and Oskar Van Der Wal. 2022. ht...

    2022 doi

  35. [43]

    Hashimoto

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca

    2023

  36. [44]

    Samia Touileb, Lilja vrelid, and Erik Velldal. 2022. https://doi.org/10.18653/v1/2022.gebnlp-1.21 Occupational biases in N orwegian and multilingual language models . In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 200--211, Sea...

    2022 doi

  37. [45]

    Hugo Touvron et al. 2023. https://arxiv.org/abs/2307.09288 Llama 2: Open foundation and fine-tuned chat models . Preprint, arXiv:2307.09288

    2023 arXiv

  38. [46]

    Bureau of Labor Statistics

    U.S. Bureau of Labor Statistics . 2024. Employment-population ratio for ages 16 and over by race and ethnicity. Retrieved December 29, 2024 from https://www.bls.gov/cps/cpsaat11.htm

    2024

  39. [47]

    Smith, Daniel Khashabi, and Hannaneh Hajishirzi

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. https://doi.org/10.18653/v1/2023.acl-long.754 Self-instruct: Aligning language models with self-generated instructions . In Proceedings of the 61st Annual Mee...

    2023 doi

  40. [48]

    Anna Zee, Marc Zee, and Anders S gaard. 2024. https://doi.org/10.18653/v1/2024.findings-naacl.143 Group fairness in multilingual speech recognition models . In Findings of the Association for Computational Linguistics: NAACL 2024, pages 2213--2226, Mexico City, Mexico. Associa...

    2024 doi

  41. [49]

    Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. https://doi.org/10.18653/v1/N18-2003 Gender bias in coreference resolution: Evaluation and debiasing methods . In Proceedings of the 2018 Conference of the North A merican Chapter of the Associati...

    2018 doi

  42. [50]

    Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024. https://openreview.net/forum?id=Bl8u7ZRlbM Wildchat: 1m chat GPT interaction logs in the wild . In The Twelfth International Conference on Learning Representations

    2024

  43. [51]

    Shucheng Zhu, Weikang Wang, and Ying Liu. 2024. https://aclanthology.org/2024.lrec-main.1180/ Quite good, but not enough: Nationality bias in large language models - a case study of C hat GPT . In Proceedings of the 2024 Joint International Conference on Computational Linguist...

    2024

  44. [52]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  45. [53]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.