Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

This paper claims that GPT-3.5 and GPT-4o make systematically uneven decisions across demographic groups, favoring women, younger people, and African-American backgrounds, while their summarization outputs show almost no such bias. It argue

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

LLMs show significant demographic bias in decision-making, favoring women, younger ages, and certain minority backgrounds; summarization shows little bias, and bias patterns largely transfer from English to Dutch.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection Solid cross-lingual audit of LLM decision bias with useful mitigation tests; the external-validity caveat is real but the paper is honest about it—worth serious review. the 4 major comments →

arxiv 2509.09735 v1 pith:BBMJKXES submitted 2025-09-10 cs.CL

Discrimination by LLMs: Cross-lingual Bias Assessment and Mitigation in Decision-Making and Summarisation

Willem Huijzer , Jieying Chen This is my paper
classification cs.CL
keywords LLM biascross-lingual biasdecision-makingsummarisationprompt mitigationGPT-3.5GPT-4obeta regression
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that commercially deployed GPT models make systematically uneven decisions across demographics—favoring women over men, younger people over older, and African-American backgrounds over European-American—while their summaries show almost no such bias. It claims these patterns largely survive translating the test prompts from English to Dutch, and that a carefully worded instruction combining equality rules, an unbiased role, and step-by-step reasoning cuts the average gap between the most and least favored groups by about 27%. The evidence comes from more than 150,000 decision prompts and 176,000 summarization prompts built from a validated template set, scored through the log-probability of a yes/no choice.

Core claim

On the decision task, both GPT-3.5 and GPT-4o are significantly biased: African-American background, female gender, and younger ages receive systematically higher 'Yes' probabilities than the 65-year-old European-American male reference group, with effects strongest when demographics are explicitly stated. The same relative pattern appears in English and Dutch, though Dutch prompts produce larger discrepancies. On summarization, however, demographic variables rarely change sentiment, length, or fidelity to the source, with the only significant effect being an age-related difference for GPT-3.5 in English. The paper also finds that prompt-based mitigation works unevenly; the 'Rules + Role + S

What carries the argument

The core object is an adapted version of Tamkin et al.'s 70 decision templates, translated into Dutch and populated with 3 ages, 2 genders, 6 backgrounds, and explicit or implicit demographic cues. Decision bias is measured by converting the model's log-probability for a forced 'Yes'/'No' answer into a probability at temperature 0, then running a mixed-effects beta regression with demographic fixed effects and request-level random intercepts to test which group differences are significant. Summarization bias is measured by masking demographic words in generated summaries and then scoring them with a repeated decision task, VADER sentiment, and ROUGE similarity.

Load-bearing premise

The paper assumes that the probability of a binary 'Yes' response at temperature 0 reveals the model's true decision tendencies and therefore represents how the model would behave in real-world, free-form decision-making.

What would settle it

Run the same demographic variation with free-form responses at higher temperature and compare approval rates, reasoning, and final decisions to the binary log-probability bias pattern; if the favoritism toward women, younger people, and African-American backgrounds disappears or reverses, the measured bias is an artifact of the forced-choice setup.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • LLM deployment in high-stakes decisions such as finance, hiring, or government services may reproduce the measured demographic skew, since the bias appears in forced approval decisions across both models and languages.
  • Bias does not necessarily transfer from decision-making to text-generation tasks: summarization remained largely neutral in this setup, suggesting task-specific bias testing is necessary.
  • Prompt instructions that explicitly state equality rules, assign an unbiased role, and structure reasoning can reduce but not eliminate demographic gaps, with larger reductions for newer models like GPT-4o.
  • Cross-lingual bias is not simply stronger or weaker in another language; the same groups are favored but with different magnitudes, so mitigation and auditing should be done per language.
  • Context matters: finance and personal/education requests show the largest demographic discrepancies, so bias checks should target the specific domain of deployment.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's decision bias estimates rest on a forced binary choice at temperature 0; it remains untested whether free-form, higher-temperature responses in real use would show the same favoritism toward women, younger people, and African-American backgrounds.
  • Because the Dutch prompts were generated by translating English templates and then pairing them with culture-specific names, the claim that bias transfers across languages may partly reflect the translation and name-generation procedure rather than a true property of the model's cross-lingual reasoning.
  • The 27% reduction in the demographic gap could partly stem from mitigation prompts making the model more cautious or less decisive overall, rather than actually equalizing treatment; the paper reports correlations with default responses but does not isolate this mechanism.
  • For summarization, neutrality measured through sentiment and ROUGE may miss biases expressed through hallucinated demographic details or subtle representational choices, so the conclusion of 'minimal bias' should be read as limited to the metrics used.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper presents a large-scale audit of bias in GPT-3.5 and GPT-4o across English and Dutch. Using an adapted dataset from Tamkin et al. (2023), the authors create over 150,000 decision prompts and 176,000 summarisation prompts, varying background, gender, age, salience (explicit vs. implicit), instruction, and language. For the decision task, they measure the log-probability of a forced binary 'Yes'/'Ja' response at temperature 0 and fit mixed-effects beta regressions to estimate demographic effects. For summarisation, they generate summaries, mask demographic words, and then run the same decision task on the masked summaries, alongside ROUGE and VADER metrics. They report that both models favor female, younger, and African-American applicants in decisions; summarisation shows little bias; bias patterns are broadly similar across languages; and a 'Rules + Role + Step-wise Guidance' instruction reduces the max-min demographic gap by about 27%. The paper frames these results as evidence of context-dependent LLM bias and of the potential of prompt-based mitigation.

Significance. The paper attempts a systematic, multilingual, multi-model bias audit with a very large prompt corpus and a reasonable statistical framework. If the measurement approach were validated, the finding that both GPT-3.5 and GPT-4o consistently favor female, younger, and African-American individuals over a European-American male reference would be practically important for deployment decisions, and the cross-lingual comparison adds value beyond the mostly English-only literature. The paper also contributes by testing several prompt-based mitigation strategies and reporting a substantial reduction in one aggregate gap metric. The authors are transparent about limitations, including the forced-binary format, multiple comparisons, and the indirect summarisation evaluation. However, the central decision-bias claims rest entirely on a single unvalidated measurement paradigm, and the summarisation conclusions rely on a proxy whose validity is questionable. Because the paper's headline conclusions are framed in terms of real-world decision-making and summarisation behaviour, these methodological gaps are load-bearing and need to be addressed before the findings can be accepted as stated.

major comments (4)
  1. [Section 3.3, Tables 3-5] The central decision-bias results are derived solely from the log-probability of a forced binary 'Yes'/'No' token at temperature 0. The authors assert that this reveals 'the model's true underlying decision tendencies' (Section 3.3), but no evidence is provided that this quantity corresponds to the decisions the model would make in free-form, higher-temperature, or conversational settings. The reported standard deviation of 3.71e-6 over 200 repeated instances only demonstrates determinism, not construct validity. Since Table 3, Table 5, the category analysis, and the mitigation claim (27% reduction) all flow from this metric, this is not a peripheral limitation. Please add a validation study (e.g., compare token-probability rankings with free-form responses on a sample of prompts, with human or LLM judges coding the decisions) or, failing that, substantially soften the real-world decisio
  2. [Section 3.4, Table 6] The summarisation-bias evaluation is indirect: after masking demographic words, the authors run the already-biased decision task on the summaries and treat changes in 'Yes' probability as evidence of summarisation bias. This assumes the decision task is a faithful and unbiased evaluator of summary content. But the decision task itself exhibits large demographic biases (Section 5.1), so the measured effects in Table 6 may reflect decision-task bias rather than bias introduced during summarisation. Moreover, masking demographic words does not guarantee that all demographic information is removed, and the masking may itself alter text fluency or meaning in ways that interact with the decision task. The conclusion that 'the summarisation task showed minimal evidence of bias' (Section 6) is therefore not well supported. Please validate the proxy (e.g., human evaluation of summary neutrality,
  3. [Section 5.1, Tables 4-5] The analysis involves a very large number of significance tests (2 models x 2 languages x 2 salience levels x 3 demographic dimensions x many categories), yet the paper uses uncorrected p<0.05 thresholds and only mentions the possibility of Type I errors in Section 6. The authors do examine p<0.001 as a robustness check, which helps, but with this many comparisons the risk of false positives remains material, and the paper does not report how many of the significant coefficients survive the stricter threshold. Please apply a family-wise error or false-discovery-rate correction, or at least present the p<0.001 comparison systematically. In addition, Table 5 filters to 'non-extreme' requests but the number of included request IDs ranges from 10 to 29 across conditions; differences in sample size and in the composition of retained requests make cross-condition comparisons unreliable. The cl
  4. [Section 5.3, Figure 3] The mitigation effectiveness is measured as the percentage change in the mean maximum difference between social groups per request. This metric is unstable when the default maximum difference is small, and the paper does not report the distribution of baseline ranges or confidence intervals for the percentage change. The statement that tasks with percentage changes greater than 5.36% are significantly different from the default prompt is not fully explained: the mixed linear regression model is not specified, and it is unclear how the 5.36% threshold was derived. Please describe the model, include standard errors or confidence intervals for the mitigation effects, and consider reporting absolute changes as well as percentage changes to avoid overstating effects on near-zero baselines.
minor comments (6)
  1. [Abstract and Sections 5.3, 6, 7] The reduction achieved by the best mitigation strategy is reported inconsistently as 27%, 27.00%, and 27.70%. Please harmonize these numbers and clarify whether they are the same estimate.
  2. [Tables 3-6] The word 'Morrocan' is misspelled; it should be 'Moroccan' in the tables and the text.
  3. [Table 2] In the 'Rules + Role + Step-wise guidance' description, the text contains a duplicated word: 'based solely on this information information ensuring'. Please fix.
  4. [Section 3.3] The description of 'log probabilities for these output options are transformed into regular probabilities' should specify the normalization (presumably softmax over the two allowed tokens), since this is essential for reproducing the 0-to-1 bounded outcome used in the beta regression.
  5. [Section 5.2] The text refers to 'GPT-4 demonstrated higher fidelity' but the paper studies GPT-3.5 and GPT-4o. Please correct the model name and ensure all such mentions are consistent.
  6. [Section 5.1] The category analysis reports aggregated discrepancy percentages (e.g., 5.31% for Finance) without confidence intervals or significance tests, making it hard to judge whether the differences across categories are meaningful. At least provide standard errors or a significance test.

Circularity Check

0 steps flagged

No circularity: bias estimates are direct measurements, not derivations from fitted parameters or self-citations.

full rationale

The paper's decision-bias estimates are direct observations of model output probabilities (log-probabilities of 'Yes'/'Ja' at temperature 0) across demographic variants of identical request templates; the beta regression describes those probabilities and does not introduce a fitted quantity that is later called a prediction. The dataset is taken from Tamkin et al. (2023), an external benchmark, and the paper's own contribution is the translation, expansion, and new mitigation prompts. No load-bearing self-citation appears: the authors cite prior work for dataset construction and prompt-engineering techniques, but no uniqueness theorem or ansatz is adopted from their own prior publications. The mitigation result ('27% mean reduction') is an observed post-intervention change in the same measured probabilities, not a parameter fitted to the outcome and renamed as a prediction. The 'Equality Rules' prompt explicitly instructs demographic independence, so a reduction in demographic gaps is a compliance effect, but it is measured empirically rather than derived from the instruction by definition. The acknowledged limitations (forced binary responses, log-probability metric, Type I error risk) concern external validity and statistical inference, not circularity. The summarisation-bias evaluation uses a sequential decision-on-masked-summary metric that, while indirect, is not defined in terms of the conclusion it supports. Overall the derivation chain is self-contained with respect to circularity concerns.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The central claim is an empirical measurement, so the ledger captures the modeling and data assumptions that could bias the measurement.

free parameters (3)
  • Non-extreme request filter bounds = 5% and 95%
    In Table 5, requests with average Yes probability outside 5-95% are excluded; this threshold is hand-chosen and substantially increases reported effect sizes (max range from about 4% to about 19%).
  • Summarisation temperature = 0.7
    Chosen to reflect real-world use, but the number of repeated runs (2) per prompt is a setting that affects variance estimates.
  • p-value significance thresholds = <0.05 and <0.001
    Standard thresholds but no multiple-comparison correction is applied.
axioms (4)
  • domain assumption Tamkin et al. (2023) dataset templates are valid, balanced, and measure discrimination after adaptation and translation.
    The entire study relies on the 70-templates dataset being a reliable probe for bias; adaptation and translation could alter this.
  • standard math Beta regression with mixed effects is appropriate for these probability data and yields unbiased coefficient estimates.
    Section 4 assumes this model; if the random effects structure or link function is mis-specified, coefficient estimates could be biased.
  • domain assumption The log-probability of the 'Yes' token at temperature 0 is a valid measure of the model's decision tendency.
    Section 3.3 uses log probabilities as the outcome; this assumes that this metric corresponds to real-world choice behavior.
  • domain assumption Names generated by GPT-4o reliably and exclusively signal the intended demographic group in the implicit condition.
    Section 3.2 uses four names per background-gender combination; if names are ambiguous or carry stereotypes, implicit-condition results are confounded.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Discrimination by LLMs: Cross-lingual Bias Assessment and Mitigation in Decision-Making and Summarisation." pith.science (2026). https://pith.science/paper/BBMJKXES

@misc{pith2026250909735,
  author       = {Pith},
  title        = {Pith review of: Discrimination by LLMs: Cross-lingual Bias Assessment and Mitigation in Decision-Making and Summarisation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BBMJKXES}},
  note         = {Machine review of arXiv:2509.09735}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

The rapid integration of Large Language Models (LLMs) into various domains raises concerns about societal inequalities and information bias. This study examines biases in LLMs related to background, gender, and age, with a focus on their impact on decision-making and summarization tasks. Additionally, the research examines the cross-lingual propagation of these biases and evaluates the effectiveness of prompt-instructed mitigation strategies. Using an adapted version of the dataset by Tamkin et al. (2023) translated into Dutch, we created 151,200 unique prompts for the decision task and 176,400 for the summarisation task. Various demographic variables, instructions, salience levels, and languages were tested on GPT-3.5 and GPT-4o. Our analysis revealed that both models were significantly biased during decision-making, favouring female gender, younger ages, and certain backgrounds such as the African-American background. In contrast, the summarisation task showed minimal evidence of bias, though significant age-related differences emerged for GPT-3.5 in English. Cross-lingual analysis showed that bias patterns were broadly similar between English and Dutch, though notable differences were observed across specific demographic categories. The newly proposed mitigation instructions, while unable to eliminate biases completely, demonstrated potential in reducing them. The most effective instruction achieved a 27\% mean reduction in the gap between the most and least favorable demographics. Notably, contrary to GPT-3.5, GPT-4o displayed reduced biases for all prompts in English, indicating the specific potential for prompt-based mitigation within newer models. This research underscores the importance of cautious adoption of LLMs and context-specific bias testing, highlighting the need for continued development of effective mitigation strategies to ensure responsible deployment of AI.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Cross-Lingual Bias in Large Language Models: A Comparative Analysis of English and Swahili

    cs.CL 2026-08 conditional novelty 6.0

    Bias in GPT-5.2 and Gemini 2.5 Flash changes rather than transfers between English and Swahili, with GPT-5.2 refusal behavior appearing only in English.

Reference graph

Works this paper leans on

40 extracted references · 7 canonical work pages · cited by 1 Pith paper · 6 internal anchors

  1. [1]

    Claude 2 model card

    Anthropic. Claude 2 model card. Technical re- port, Anthropic, 2023. URL https://www-cdn.anthropic. com/bd2a28d2535bfb0494cc8e2a3bf135d2e7523226/ Model-Card-Claude-2.pdf. Accessed: 2025-02-11

    2023

  2. [2]

    The claude 3 model family: Opus, sonnet, haiku

    Anthropic. The claude 3 model family: Opus, sonnet, haiku. Model card, Anthropic, 2024. URL https://www-cdn.anthropic.com/files/ 4zrzovbb/website/bd2a28d2535bfb0494cc8e2a3bf135d2e7523226. pdf. Accessed: March 2024

    2024

  3. [3]

    Armstrong, A

    L. Armstrong, A. Liu, S. MacNeil, and D. Metaxa. The silicon ceiling: Auditing gpt’s race and gender biases in hiring, 2024. URL https://arxiv. org/abs/2405.04412

    Pith/arXiv arXiv 2024

  4. [4]

    Looking for a Handsome Carpenter! Debiasing GPT-3 Job Advertisements

    C. Borchers, D. S. Gala, B. Gilburt, E. Oravkin, W. Bounsi, Y . M. Asano, and H. R. Kirk. Looking for a handsome carpenter! debias- ing GPT-3 job advertisements.CoRR, abs/2205.11374, 2022. doi: 10.48550/ARXIV .2205.11374. URL https://doi.org/10.48550/arXiv. 2205.11374

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2205.11374 2022

  5. [5]

    Brown and R

    H. Brown and R. Shokri. How (un)fair is text summarization?, 2023. URL https://openreview.net/forum?id=-UsbRlXzMG

    2023

  6. [6]

    Devlin, M.-W

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805, 2018

    Pith/arXiv arXiv 2018

  7. [7]

    Eigner and T

    E. Eigner and T. Händler. Determinants of llm-assisted decision- making, 2024. URL https://arxiv.org/abs/2402.17385

    Pith/arXiv arXiv 2024

  8. [8]

    Ferrari and F

    S. Ferrari and F. Cribari-Neto. Beta regression for modelling rates and proportions.Journal of applied statistics, 31(7):799–815, 2004

    2004

  9. [9]

    This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models

    I. García-Ferrero, B. Altuna, J. Álvez, I. Gonzalez-Dios, and G. Rigau. This is not a dataset: A large negation benchmark to challenge large language models.CoRR, abs/2310.15941, 2023. doi: 10.48550/ARXIV . 2310.15941. URL https://doi.org/10.48550/arXiv.2310.15941

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2310.15941 2023

  10. [10]

    L. Giray. Prompt engineering with chatgpt: a guide for academic writ- ers.Annals of biomedical engineering, 51(12):2629–2633, 2023

    2023

  11. [11]

    Hahn and I

    U. Hahn and I. Mani. The challenges of automatic summarization. Computer, 33(11):29–36, 2000. doi: 10.1109/2.881692. URL https: //doi.org/10.1109/2.881692

    doi:10.1109/2.881692 2000

  12. [12]

    Huang, H

    N. Huang, H. Fayek, and X. Zhang. Bias in opinion summarisation from pre-training to adaptation: A case study in political bias. In Y . Gra- ham and M. Purver, editors,Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2024 - Volume 1: Long Papers, St. Julian’s, Malta, March 17- 22, 2024, page...

    2024

  13. [13]

    Hutto and E

    C. Hutto and E. Gilbert. Vader: A parsimonious rule-based model for sentiment analysis of social media text. InProceedings of the inter- national AAAI conference on web and social media, volume 8, pages 216–225, 2014

    2014

  14. [14]

    M. P. Joachimiak, J. H. Caufield, N. L. Harris, H. Kim, and C. J. Mungall. Gene set summarization using large language models.CoRR, abs/2305.13338, 2023. doi: 10.48550/ARXIV .2305.13338. URL https: //doi.org/10.48550/arXiv.2305.13338

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2305.13338 2023

  15. [15]

    Joshi, S

    A. Joshi, S. Kale, S. Chandel, and D. K. Pal. Likert scale: Explored and explained.British journal of applied science & technology, 7(4): 396–403, 2015

    2015

  16. [16]

    Lauscher and G

    A. Lauscher and G. Glavas. Are we consistently biased? multidimen- sional analysis of biases in distributional word vectors. In R. Mihal- cea, E. Shutova, L. Ku, K. Evang, and S. Poria, editors,Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics, *SEM@NAACL-HLT 2019, Minneapolis, MN, USA, June 6-7, 2019, pages 85–91. Associat...

    work page doi:10.18653/v1/s19-1010 2019

  17. [17]

    S. Levy, N. A. John, L. Liu, Y . Vyas, J. Ma, Y . Fujinuma, M. Balles- teros, V . Castelli, and D. Roth. Comparing biases and the impact of multilingual training across multiple languages. In H. Bouamor, J. Pino, and K. Bali, editors,Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-1...

    doi:10.18653/v1/2023.emnlp-main.634 2023

  18. [18]

    Lewis, Y

    M. Lewis, Y . Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V . Stoyanov, and L. Zettlemoyer. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and compre- hension.arXiv preprint arXiv:1910.13461, 2019

    Pith/arXiv arXiv 1910

  19. [19]

    C. Li, J. Wang, Y . Zhang, K. Zhu, W. Hou, J. Lian, F. Luo, Q. Yang, and X. Xie. Large language models understand and can be enhanced by emotional stimuli, 2023

    2023

  20. [20]

    C.-Y . Lin. ROUGE: A package for automatic evaluation of summaries. InText Summarization Branches Out, pages 74–81, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL https:// aclanthology.org/W04-1013

    2004

  21. [21]

    L. Lippens. Computer says ‘no’: Exploring systemic bias in chatgpt using an audit approach.Computers in Human Behavior: Artificial Hu- mans, 2(1):100054, Jan. 2024. ISSN 2949-8821. doi: 10.1016/j.chbah. 2024.100054. URL http://dx.doi.org/10.1016/j.chbah.2024.100054

    arXiv 2024

  22. [22]

    N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang. Lost in the middle: How language models use long contexts. CoRR, abs/2307.03172, 2023. doi: 10.48550/ARXIV .2307.03172. URL https://doi.org/10.48550/arXiv.2307.03172

    Pith/arXiv arXiv doi:10.48550/arxiv.2307.03172 2023

  23. [23]

    Naveed, A

    H. Naveed, A. U. Khan, S. Qiu, M. Saqib, S. Anwar, M. Usman, N. Akhtar, N. Barnes, and A. Mian. A comprehensive overview of large language models, 2024

    2024

  24. [24]

    Navigli, S

    R. Navigli, S. Conia, and B. Ross. Biases in large language models: Origins, inventory, and discussion.J. Data and Information Quality, 15 (2), jun 2023. ISSN 1936-1955. doi: 10.1145/3597307. URL https: //doi.org/10.1145/3597307

    doi:10.1145/3597307 2023

  25. [25]

    How chatgpt and our language models are developed

    OpenAI. How chatgpt and our language models are developed. OpenAI Help Center, 2024. URL https://help.openai.com/en/articles/ 7842364-how-chatgpt-and-our-language-models-are-developed. Ac- cessed on 2024-04-03

    2024

  26. [26]

    Gpt-4 technical report, 2024

    OpenAI et al. Gpt-4 technical report, 2024. URL https://arxiv.org/abs/ 2303.08774

    Pith/arXiv arXiv 2024

  27. [27]

    Y . Qian, U. Muaz, B. Zhang, and J. W. Hyun. Reducing gender bias in word-level language models with a gender-equalizing loss function, 2019

    2019

  28. [28]

    Raffel, N

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, and P. J. Liu. Exploring the limits of trans- fer learning with a unified text-to-text transformer.arXiv preprint arXiv:1910.10683, 2019

    Pith/arXiv arXiv 1910

  29. [29]

    S. Raza, C. Ding, and D. Pandya. Mitigating bias in conversa- tions: A hate speech classifier and debiaser with prompts.CoRR, abs/2307.10213, 2023. doi: 10.48550/ARXIV .2307.10213. URL https: //doi.org/10.48550/arXiv.2307.10213

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2307.10213 2023

  30. [30]

    A. Sant, C. Escolano, A. Mash, F. de Luca Fornaciari, and M. Melero. The power of prompts: Evaluating and mitigating gender bias in MT with llms.CoRR, abs/2407.18786, 2024. doi: 10.48550/ARXIV .2407. 18786. URL https://doi.org/10.48550/arXiv.2407.18786

    Pith/arXiv arXiv doi:10.48550/arxiv.2407.18786 2024

  31. [31]

    Sclar, Y

    M. Sclar, Y . Choi, Y . Tsvetkov, and A. Suhr. Quantifying language mod- els’ sensitivity to spurious features in prompt design or: How I learned to start worrying about prompt formatting.CoRR, abs/2310.11324,

    Pith/arXiv arXiv

  32. [32]

    Steen and K

    J. Steen and K. Markert. Bias in news summarization: Measures, pit- falls and corpora. In L. Ku, A. Martins, and V . Srikumar, editors, Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, pages 5962–5983. Association for Computational Linguistics, 2024. URL https://aclanthology.or...

    2024

  33. [33]

    Tamkin, A

    A. Tamkin, A. Askell, L. Lovitt, E. Durmus, N. Joseph, S. Kravec, K. Nguyen, J. Kaplan, and D. Ganguli. Evaluating and mitigating dis- crimination in language model decisions. (arXiv:2312.03689), Dec

    Pith/arXiv arXiv

  34. [34]

    The Extractive-Abstractive Axis: Measuring Content "Borrowing" in Generative Language Models

    N. Teneva. The extractive-abstractive axis: Measuring content "borrow- ing" in generative language models.CoRR, abs/2307.11779, 2023. doi: 10.48550/ARXIV .2307.11779. URL https://doi.org/10.48550/arXiv. 2307.11779

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2307.11779 2023

  35. [35]

    K. Vida, F. Damken, and A. Lauscher. Decoding multilingual moral preferences: Unveiling llm’s biases through the moral machine exper- iment.CoRR, abs/2407.15184, 2024. doi: 10.48550/ARXIV .2407. 15184. URL https://doi.org/10.48550/arXiv.2407.15184

    Pith/arXiv arXiv doi:10.48550/arxiv.2407.15184 2024

  36. [36]

    URL http://arxiv.org/abs/2312

    doi: 10.48550/arXiv.2312.03689. URL http://arxiv.org/abs/2312. 03689. arXiv:2312.03689 [cs]

    Pith/arXiv arXiv doi:10.48550/arxiv.2312.03689

  37. [37]

    Leveraging Large Language Model to Generate a Novel Metaheuristic Algorithm with CRISPE Framework

    R. Zhong, Y . Xu, C. Zhang, and J. Yu. Leveraging large language model to generate a novel metaheuristic algorithm with CRISPE framework. CoRR, abs/2403.16417, 2024. doi: 10.48550/ARXIV .2403.16417. URL https://doi.org/10.48550/arXiv.2403.16417

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2403.16417 2024

  38. [39]

    J. Zhao, T. Wang, M. Yatskar, R. Cotterell, V . Ordonez, and K. Chang. Gender bias in contextualized word embeddings. InProceedings of the 2019 Conference NAACL-HLT , Minneapolis, MN, USA, June 2-7, 2019, Volume 1, pages 629–634. Association for Computational Lin- guistics, 2019. doi: 10.18653/V1/N19-1064. URL https://doi.org/10. 18653/v1/n19-1064

    doi:10.18653/v1/n19-1064 2019

  39. [2023]

    URL https://doi.org/10

    doi: 10.48550/ARXIV .2310.11324. URL https://doi.org/10. 48550/arXiv.2310.11324

    Pith/arXiv arXiv doi:10.48550/arxiv.2310.11324

  40. [2024]

    URL https://aclanthology.org/2024.eacl-long.63

    2024

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.