Pith. sign in

REVIEW 4 major objections 5 minor 66 references

Personalisation or Prejudice? Addressing Geographic Bias in Hate Speech Detection using Debias Tuning in Large Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that LLM memory features that add country or language context introduce geographic bias into hate speech detection, and that a debias-tuning fine-tune with a consistency penalty reduces that bias while improving baseline…

desk verdict Country and language personas really do shift LLM hate-speech verdicts, but the debias-tuning loss as written is just reweighted cross-entropy, not a consistency penalty, so the mitigation needs an ablation before it is taken at face value. read the letter →

arxiv 2505.02252 v1 pith:K2MDRR7L submitted 2025-05-04 cs.CL

Paloma Piot , Patricia Martín-Rodilla , Javier Parapar This is my paper · ORCID
classification cs.CL
keywords hatespeechdetectiongeographicbiasLLMpersonalisationdebiastuningcountrypersonasmultilingualconsistencylossfalsenegativerate
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the memory and personalisation features now built into large language models—which inject user details such as country and language into the context—change how those models classify hate speech, creating systematic geographic and language bias. It claims that adding a country persona lowers F1 scores and raises false-negative rates for most tested models, with especially large effects for some countries, and that translating the same posts into the persona's language often worsens the gap. To fix this, the paper fine-tunes models with a custom loss that penalises predictions that flip between a no-context baseline and a country-persona version, forcing consistency. If right, personalisation features can silently degrade a sensitive moderation task, and a relatively cheap debias-tuning step can recover much of the lost performance while also improving the no-context baseline.

What carries the argument

The central mechanism is 'debias tuning', a fine-tuning procedure built on a custom consistency loss. For each post, the model sees both the plain text and the same text wrapped in a country-persona prompt; the loss combines ordinary cross-entropy on both views with an extra penalty term, active only when the context-free prediction is correct but the persona-context prediction is wrong or the output is invalid, scaled by a hyperparameter $\alpha$. This forces the model toward answers that do not flip because of location or language context. The paper also builds CountryHate, a country-augmented subset of the MetaHate collection, and uses a fixed persona template plus a strict True/False task prompt as the test harness.

What would settle it

Take the paper's own 100 false-negative sample, have annotators re-label each post under a strict hate-speech definition, and recompute per-country false-negative rates on the cleaned labels; if the large gaps between high-bias countries and the UK or US collapse or shrink to noise, the claimed geographic bias is largely a label-quality artefact.

Watch

Extended reading notes

Core claim

The paper's central claim is that injecting geographic and linguistic context into an LLM's prompt—the way memory features do—biases hate speech detection: the same text can be labelled hate speech for one country persona and not for another. Across five open-source LLMs, the no-context baseline consistently outperforms the country-persona setting on F1, and false-negative rates rise sharply for countries the model associates with negative stereotypes; for Llama 3.1, the false-negative rate jumps from about 32% to over 71% under country persona prompts. The paper further claims that debias tuning—fine-tuning with a loss that penalises inconsistency between context-free and country-context predictions when the context-free prediction is correct—reduces these discrepancies, improves F1 in both personalised and non-personalised settings, and generalises to countries not seen during tuning.

Load-bearing premise

The gold labels in the country-augmented hate speech dataset are reliable enough that differences in false-negative rates across personas measure model bias rather than disagreement with questionable annotations.

Editorial extensions

If this is right

  • If LLM memory personalisation includes country or language, hate speech detection can vary by user, so systems relying on such personalisation may under-protect users from some regions.
  • Debias tuning recovers performance not only in personalised contexts but also in the no-context baseline, so fine-tuning for consistency does not trade away general accuracy.
  • The debiasing effect generalises to countries not used in tuning, though unevenly; broader country coverage during tuning would likely improve fairness further.
  • Multilingual debias tuning, using persona languages during fine-tuning, can generalise to unseen languages, though not uniformly across all languages.
  • Residual errors concentrate in doubtful labels and implicit or sarcastic hate speech, so benchmark label quality becomes the next bottleneck for measuring bias.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: a natural extension is to apply the same consistency-penalty tuning to other sensitive tasks, such as sentiment or content moderation for self-harm, to test whether personalisation bias is a general phenomenon rather than specific to hate speech.
  • The paper's own error analysis implies the measured bias magnitudes may be upper bounds; re-benchmarking with adjudicated labels could separate genuine country bias from disagreement with questionable annotations.
  • The debias loss penalises only the direction where the context-free prediction is correct and the persona-context prediction is wrong; adding symmetric penalties for flips in the opposite direction might change which errors are suppressed.
  • Because memory features in commercial models are hidden, a fixed public benchmark of country-persona prompts could serve as a low-cost monitoring tool for new model releases.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper investigates whether country and language personalization, simulated through persona prompts, introduces bias in hate speech classification performed by five open-source LLMs (Llama 3.1, Nemo, Gemma, DeepSeek, Phi 4). Using 'CountryHate,' a subset of the MetaHate dataset augmented with twelve countries selected from low-scoring human-rights, LGBTQ+, and women's-rights indices, the authors prompt models with and without country personas, in English and in the personas' official languages. They report that country context generally lowers F1 and raises false-negative rates, with the largest shifts for Llama 3.1 (FNR rising from 32.37% to over 71%). To mitigate this, they propose 'debias tuning,' a fine-tuning procedure with a custom loss intended to penalize inconsistent classifications with and without context, applied to Llama 3.1, Nemo, and Phi 4. Results in Table 6 and Figure 2 show improved F1 scores and reduced country-specific FNRs for the debiased models, with the Llama variants showing the largest gains. The paper concludes that memory personalization features can introduce geographic and language bias in hate speech detection, and that debias tuning significantly improves model behavior.

Significance. The paper addresses a timely and important problem: memory-enabled personalization in LLMs may alter decisions in sensitive tasks such as hate speech detection. The zero-shot evaluation across five open-source models on a large aggregated dataset provides credible evidence that contextual prompts can shift classification outcomes, and the country-level FNR analysis is a useful diagnostic. The proposed debias tuning, if it worked as described, would be a practical contribution, and the authors provide a public code repository and explicitly acknowledge label noise in their error analysis. However, the central mitigation claim is not yet supported: the loss function as written does not implement a consistency penalty, no α=0 control is reported, and the debias results lack uncertainty quantification. In addition, the bias measurement is weakened by the removal of invalid responses without re-basing denominators and by the paper's own finding that 74% of sampled false negatives are 'doubtful labels.' These issues prevent the conclusions from being accepted as stated.

major comments (4)
  1. [Custom loss, Equations (1)-(2)] The loss defined in Equations (1)-(2) is not a consistency penalty. Equation (2) is L_loss = L_avg_class + α·L_avg_class, which equals (1+α)·L_avg_class whenever the condition on the predictions holds. This is simply a scaled version of the same average cross-entropy; no term compares the context and non-context predictions or logits, so the model is not penalized for inconsistency in any mechanism distinct from ordinary classification. The abstract and Conclusions attribute the gains to 'penalising inconsistent classifications,' but the equation cannot support that mechanism. Furthermore, the value of α is never reported, and no α=0 control (ordinary fine-tuning on CountryHate without the penalty) is run. Without that control, the F1/FNR improvements in Table 6 could be entirely due to standard supervised fine-tuning on 19,306 training instances. The authors should either provide the actual implemented loss (e.g., with a KL-divergence term between the two output distributions), report α, and add an α=0 condition, or revise the claim that the method is a consistency-based debiasing procedure.
  2. [Output processing] The paper removes invalid responses (e.g., 'I cannot perform this action') before computing metrics, but it does not report how many were removed per condition or re-base the denominators. For Llama 3.1, Gemma, and DeepSeek, the country-lang variant produced more than 7,000 invalid generations, which is a substantial fraction of the 4,826 test instances per condition. If the F1 and FNR values in Tables 2-4 are computed only over valid responses, then differences across conditions may reflect variation in the models' refusal rates rather than in classification accuracy. The authors should report the number and rate of valid responses for each model and condition, and recompute the main comparisons either on the full set (treating invalid outputs as errors) or with appropriate per-condition denominators.
  3. [Error Classes of False Negatives] The paper's own error analysis states that 74% of 100 sampled false negatives are 'doubtful labels,' with examples such as 'I hate racist people very much' being labelled as hate speech. This directly undermines the reliability of the gold labels used to establish bias in RQ1-RQ3 and to evaluate debias tuning in RQ4. If a large fraction of the labels are questionable, then the FNR and F1 gaps between contexts (Tables 2-5) are partly disagreements with noisy annotations rather than clean measures of model bias. The authors should quantify the impact of label noise, for example by manually validating a random sample of test instances per condition, reporting the agreement rate, or re-running the key comparisons on a cleaned subset.
  4. [Debias Tuning Results, Table 6] The claim that debias tuning 'significantly improved' model behaviour is not supported by any significance testing or uncertainty quantification. The fine-tuning procedure is stochastic (LoRA, batch sampling), yet only a single run appears to be reported, with no confidence intervals, repeated-seed variance, or paired tests. The chi-squared tests in Tables 5 and 7 assess whether country-context predictions differ from baseline (or from the UK), not whether the differences between the base and debiased models are statistically significant. The authors should report variance across seeds and test the improvement in F1/FNR for significance, or soften the 'significantly' language in the Abstract and Conclusions.
minor comments (5)
  1. [Country augmentation selection] The text says the twelve countries were 'randomly selected' from the bottom-25 indices, but the list appears to be a purposive sample. Please clarify the random procedure (e.g., seed and sampling frame) or state that the selection was purposive to achieve regional coverage.
  2. [Languages] The paper does not name the translation tool used to translate the English posts into the personas' official languages, nor does it discuss translation quality. This is a reproducibility issue; please identify the tool and any quality checks.
  3. [Analysis Results, chi-squared test] The sentence 'We performed a chi-squared test (X2) at p > 0.01' appears to be a typo; the subsequent discussion says the null hypothesis is rejected at p < 0.01. Please correct the notation.
  4. [Table 5] The FNR for Australia in Table 5 is 69.64%, but the text just below the table says '69.00%.' Please ensure the numbers are consistent.
  5. [Abstract and Conclusions] The word 'significantly' is used in the Abstract and Conclusions (e.g., 'debias tuning significantly improved LLMs'' behaviour') without a statistical test. If no significance test is added, please replace 'significantly' with 'substantially' or another neutral qualifier.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; bias measurement and debias evaluation are empirically grounded on held-out data.

full rationale

The paper's load-bearing claims are the measured country/language-dependent classification gaps (RQ1-RQ3) and the debias-tuning improvements (RQ4). The bias measurement compares zero-shot model outputs for persona vs. non-persona prompts against gold labels in CountryHate, a subset of the externally aggregated MetaHate collection; those labels are not derived from the model predictions, so the F1/FNR gaps are an empirical finding rather than an input. The mitigation evaluation fine-tunes on a 19,306-instance training split and reports F1/FNR on a 4,826-instance held-out test split; the custom loss (Eq. 1-2) is not fitted to the test results, and no test metric appears in the training objective, so the reported improvements are not forced by construction. The paper does cite the authors' own MetaHate dataset and prior work, but only as data provenance and related work, not as an argument that substitutes for measurement. The custom-loss equation is open to the correctness criticism that Eq. 2 is a scalar reweighting of cross-entropy rather than an explicit consistency penalty, and the absence of an α=0 control weakens the mechanism claim; however, that is an experimental-validity issue, not circularity, because the result does not reduce to the loss definition by construction. The limitations section openly acknowledges the restricted country set, single epoch, and task scope. No step in the derivation chain was found to be self-definitional, fitted-then-renamed, or dependent on a self-citation chain.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

No new physical or conceptual entities are introduced; the paper's commitments are to standard statistical tools, a persona proxy for memory, translation preservation, label reliability, and a purposive country sample. The two unreported numeric choices (alpha and LoRA rank) are the most consequential free parameters because the central mitigation cannot be exactly replicated without them.

free parameters (2)
  • alpha (consistency penalty weight) = not reported
    Equation 2 uses alpha > 0 to scale the loss on inconsistent examples, but the paper never gives its numeric value or how it was chosen. The method cannot be exactly reproduced without it.
  • LoRA rank (Low-Rank Adaptation dimension) = not reported
    Section 'Debias fine-tuning' specifies LoRA, 4-bit precision, batch size, optimizer, and learning rate, but omits the LoRA rank and target modules, which are needed to replicate the fine-tune.
assumptions (6)
  • standard math Cross-entropy is an appropriate classification loss for the hate speech label
    Equation 1 defines the fine-tuning objective as cross-entropy with a weighting term; standard practice, not proved in the paper.
  • standard math Chi-square tests with significance level 0.01 support the claims of a statistically significant difference
    Section 'Analysis Results' and Tables 5 and 7 use chi-square tests; the paper states the significance criterion incorrectly as 'p > 0.01' instead of alpha = 0.01.
  • domain assumption A one-line country persona is a valid surrogate for actual LLM memory personalisation
    Section 'Experimental Setup' states 'To simulate LLMs' personalised information, we formulate our study by providing a user-persona with a location context'; no evidence links this surrogate to how commercial memory features embed user attributes.
  • domain assumption Machine translation of posts into each persona's official language preserves the hate speech content
    Section 'Languages' says posts were translated before prompting, but the translation system is never named; translation errors could explain cross-language differences.
  • domain assumption Hate speech labels in CountryHate are accurate enough for the bias measurement
    The error analysis reports 74% of false negatives sampled were 'doubtful labels', directly questioning label quality; F1/FNR comparisons inherit this uncertainty.
  • ad hoc to paper Countries in the bottom of human rights, LGBTQ+ and women's rights indices are the right probe for Western-narrative biases
    Section 'Country augmentation selection' purposively samples only these countries; this shapes which biases can be detected and precludes a neutral random-country baseline.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Personalisation or Prejudice? Addressing Geographic Bias in Hate Speech Detection using Debias Tuning in Large Language Models." pith.science (2026). https://pith.science/paper/K2MDRR7L

@misc{pith2026250502252,
  author       = {Pith},
  title        = {Pith review of: Personalisation or Prejudice? Addressing Geographic Bias in Hate Speech Detection using Debias Tuning in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K2MDRR7L}},
  note         = {Machine review of arXiv:2505.02252}
}
read the original abstract

Commercial Large Language Models (LLMs) have recently incorporated memory features to deliver personalised responses. This memory retains details such as user demographics and individual characteristics, allowing LLMs to adjust their behaviour based on personal information. However, the impact of integrating personalised information into the context has not been thoroughly assessed, leading to questions about its influence on LLM behaviour. Personalisation can be challenging, particularly with sensitive topics. In this paper, we examine various state-of-the-art LLMs to understand their behaviour in different personalisation scenarios, specifically focusing on hate speech. We prompt the models to assume country-specific personas and use different languages for hate speech detection. Our findings reveal that context personalisation significantly influences LLMs' responses in this sensitive area. To mitigate these unwanted biases, we fine-tune the LLMs by penalising inconsistent hate speech classifications made with and without country or language-specific context. The refined models demonstrate improved performance in both personalised contexts and when no context is provided.

Figures

Figures reproduced from arXiv: 2505.02252 by the authors.

Figure 1
Figure 1. Llama 3.1 hate speech classification with dif￾ferent country contexts. discrimination, or hostility towards individuals or groups (Davidson et al. 2017) based on attributes such as race, re￾ligion, ethnicity, or gender (ElSherief et al. 2018a,b; Das et al. 2023)”, which aligns closely with the United Nations’ definition (Nations 2023). Given that these attributes may be stored in LLMs’ memory, there is a significant… view at source ↗
Figure 2
Figure 2. (Up): FNR across countries for (1) Llama 3.1, (2) debias-llama and (3) Debias Llama Lang, for English. (Down): [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 47 canonical work pages

  1. [1]

    Badjatiya, P.; Gupta, M.; and Varma, V. 2019. Stereotypical Bias Removal for Hate Speech Detection Task using Knowledge-based Generalizations. In The World Wide Web Conference, WWW '19, 49–59. New York, NY, USA: Association for Computing Machinery. ISBN 9781450366748

    work page 2019

  2. [2]

    Chen, J.; Wang, X.; Xu, R.; Yuan, S.; Zhang, Y.; Shi, W.; ...; and Xiao, Y. 2024. From Persona to Personalization: A Survey on Role-Playing Language Agents. arXiv:2404.18231

    arXiv 2024

  3. [3]

    They are uncultured

    Dammu, P. P. S.; Jung, H.; Singh, A.; Choudhury, M.; and Mitra, T. 2024. "They are uncultured": Unveiling Covert Harms and Social Threats in LLM Generated Conversations. arXiv:2405.05378

    arXiv 2024

  4. [4]

    Das, M.; Raj, R.; Saha, P.; Mathew, B.; Gupta, M.; and Mukherjee, A. 2023. HateMM: A Multi-Modal Dataset for Hate Video Classification. Proceedings of the ICWSM 2023, 17: 1014–1023

    work page 2023

  5. [5]

    Davidson, T.; Bhattacharya, D.; and Weber, I. 2019. Racial Bias in Hate Speech and Abusive Language Detection Datasets. In Proceedings of the Third Workshop on Abusive Language Online, 25--35. Florence, Italy: ACL

    work page 2019

  6. [6]

    Davidson, T.; Warmsley, D.; Macy, M.; and Weber, I. 2017. Automated Hate Speech Detection and the Problem of Offensive Language. Proceedings of the ICWSM 2017, 11(1): 512–515

    work page 2017

  7. [7]

    DeepSeek-AI. 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948

    arXiv 2025

  8. [8]

    Demidova, A.; Atwany, H.; Rabih, N.; Sha ' ban, S.; and Abdul-Mageed, M. 2024. John vs. Ahmed: Debate-Induced Bias in Multilingual LLM s. In Proceedings of The Second Arabic Natural Language Processing Conference, 193--209. Bangkok, Thailand: ACL

    work page 2024

Show all 66 references
  1. [9]

    S.; and Caverlee, J

    Dong, X.; Wang, Y.; Yu, P. S.; and Caverlee, J. 2024. Disclosure and Mitigation of Gender Bias in LLMs. arXiv:2402.11190

    2024 arXiv

  2. [10]

    Dubey, A.; Jauhri, A.; Pandey, A.; Kadian, A.; Al-Dahle, A.; Letman, A.; ...; and Zhao, Z. 2024. The Llama 3 Herd of Models. arXiv:2407.21783

    2024 arXiv

  3. [11]

    Dwivedi, S.; Ghosh, S.; and Dwivedi, S. 2023. Breaking the Bias: Gender Fairness in LLMs Using Prompt Engineering and In-Context Learning. Rupkatha Journal on Interdisciplinary Studies in Humanities, 15(4)

    2023

  4. [12]

    ElSherief, M.; Kulkarni, V.; Nguyen, D.; Yang Wang, W.; and Belding, E. 2018 a . Hate Lingo: A Target-Based Linguistic Analysis of Hate Speech in Social Media. Proceedings of the ICWSM 2018, 12(1)

    2018

  5. [13]

    ElSherief, M.; Nilizadeh, S.; Nguyen, D.; Vigna, G.; and Belding, E. 2018 b . Peer to Peer Hate: Hate Speech Instigators and Their Targets. Proceedings of the ICWSM 2018, 12(1)

    2018

  6. [14]

    Founta, A.; Djouvas, C.; Chatzakou, D.; Leontiadis, I.; Blackburn, J.; Stringhini, G.; ...; and Kourtellis, N. 2018. Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior. Proceedings of the ICWSM 2018, 12(1)

    2018

  7. [15]

    Ganin, Y.; Ustinova, E.; Ajakan, H.; Germain, P.; Larochelle, H.; Laviolette, F.; Marchand, M.; and Lempitsky, V. 2016. Domain-adversarial training of neural networks. J. Mach. Learn. Res., 17(1): 2096–2030

    2016

  8. [16]

    H.; and Beutel, A

    Garg, S.; Perot, V.; Limtiaco, N.; Taly, A.; Chi, E. H.; and Beutel, A. 2019. Counterfactual Fairness in Text Classification through Robustness. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, AIES '19, 219–226. New York, NY, USA: Association for Com...

    2019

  9. [17]

    Gemma Team . 2025. Gemma 3

    2025

  10. [18]

    Gira, M.; Zhang, R.; and Lee, K. 2022. Debiasing Pre-Trained Language Models via Efficient Fine-Tuning. In Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion, 59--69. Dublin, Ireland: ACL

    2022

  11. [19]

    Gupta, S.; Shrivastava, V.; Deshpande, A.; Kalyan, A.; Clark, P.; Sabharwal, A.; and Khot, T. 2024. Bias R uns D eep: Implicit Reasoning Biases in Persona-Assigned LLM s. In The Twelfth International Conference on Learning Representations

    2024

  12. [20]

    Herre, B.; and Arriagada, P. 2016. Human Rights. Our World in Data. Https://ourworldindata.org/human-rights

    2016

  13. [21]

    Herre, B.; and Arriagada, P. 2023. LGBT+ Rights. Our World in Data. Https://ourworldindata.org/lgbt-rights

    2023

  14. [22]

    Herre, B.; Samborska, V.; Arriagada, P.; and Ritchie, H. 2023. Women’s Rights. Our World in Data. Https://ourworldindata.org/women-rights

    2023

  15. [23]

    Jiao, J.; Afroogh, S.; Xu, Y.; and Phillips, C. 2024. Navigating LLM Ethics: Advancements, Challenges, and Future Directions. arXiv:2406.18841

    2024 arXiv

  16. [24]

    Kamruzzaman, M.; and Kim, G. L. 2024. Exploring Changes in Nation Perception with Nationality-Assigned Personas in LLMs. arXiv:2406.13993

    2024 arXiv

  17. [25]

    Kotek, H.; Dockum, R.; and Sun, D. 2023. Gender bias and stereotypes in Large Language Models. In Proceedings of The ACM Collective Intelligence Conference, CI '23, 12–24. New York, NY, USA: Association for Computing Machinery. ISBN 9798400701139

    2023

  18. [26]

    V.; Singh, S.; and Ragupathy, S

    Kumar, A.; Murthy, S. V.; Singh, S.; and Ragupathy, S. 2024. The Ethics of Interaction: Mitigating Security Threats in LLMs. arXiv:2401.12273

    2024 arXiv

  19. [27]

    Lacoste, A.; Luccioni, A.; Schmidt, V.; and Dandres, T. 2019. Quantifying the Carbon Emissions of Machine Learning. arXiv preprint arXiv:1910.09700

    2019 arXiv

  20. [28]

    Leidinger, A.; and Rogers, R. 2024. How Are LLMs Mitigating Stereotyping Harms? Learning from Search Engine Studies. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 7(1): 839--854

    2024

  21. [29]

    Lim, S.; and Pérez-Ortiz, M. 2024. The African Woman is Rhythmic and Soulful: An Investigation of Implicit Biases in LLM Open-ended Text Generation. arXiv:2407.01270

    2024 arXiv

  22. [30]

    Lin, Z.; Guan, S.; Zhang, W.; Zhang, H.; Li, Y.; and Zhang, H. 2024. Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models. Artificial Intelligence Review, 57(9)

    2024

  23. [31]

    Ma, X.; Sap, M.; Rashkin, H.; and Choi, Y. 2020. P ower T ransformer: Unsupervised Controllable Revision for Biased Language Correction. In Proceedings of the 2020 Conference EMNLP, 7426--7441. Online: ACL

    2020

  24. [32]

    Manvi, R.; Khanna, S.; Burke, M.; Lobell, D.; and Ermon, S. 2024. Large Language Models are Geographically Biased. arXiv:2402.02680

    2024 arXiv

  25. [33]

    Maronikolakis, A.; Baader, P.; and Sch \"u tze, H. 2022. Analyzing Hate Speech Data along Racial, Gender and Intersectional Axes. In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP), 1--7. Seattle, Washington: ACL

    2022

  26. [34]

    Microsoft; :; Abouelenin, A.; Ashfaq, A.; Atkinson, A.; Awadalla, H.; Bach, N.; Bao, J.; ...; and Zhou, X. 2025. Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs. arXiv:2503.01743

    2025 arXiv

  27. [35]

    Mistral AI team . 2025. M istral N e M o | M istral A I . https://mistral.ai/news/mistral-nemo. Accessed: 09/01/2025

    2025

  28. [36]

    Mozafari, M.; Farahbakhsh, R.; and Crespi, N. 2020. Hate speech detection and racial bias mitigation in social media based on BERT model. PLOS ONE, 15(8): e0237861

    2020

  29. [37]

    Nations, U. 2023. What is hate speech? Accessed: 15/11/2023

    2023

  30. [38]

    Ollion, E.; Shen, R.; Macanovic, A.; and Chatelain, A. 2024. The dangers of using proprietary LLMs for research. Nature Machine Intelligence, 6(1): 4–5

    2024

  31. [39]

    OpenAI. 2024. Memory and new controls for ChatGPT. https://openai.com/index/memory-and-new-controls-for-chatgpt/. [Accessed 04-12-2024]

    2024

  32. [40]

    Palta, S.; and Rudinger, R. 2023. FORK : A Bite-Sized Test Set for Probing Culinary Cultural Biases in Commonsense Reasoning Models. In Findings of the ACL: ACL 2023, 9952--9962. Toronto, Canada: ACL

    2023

  33. [41]

    Parvaresh, V.; and Harvey, G. 2023. Rhetorical Questions as Conveyors of Hate Speech, 229--251. Cham: Springer Nature Switzerland. ISBN 978-3-031-38248-2

    2023

  34. [42]

    Piot, P.; Martín-Rodilla, P.; and Parapar, J. 2024. MetaHate: A Dataset for Unifying Efforts on Hate Speech Detection. Proceedings of the International AAAI Conference on Web and Social Media, 18(1): 2025--2039

    2024

  35. [43]

    Piot, P.; and Parapar, J. 2025 a . Decoding Hate: Exploring Language Models' Reactions to Hate Speech. In Chiruzzo, L.; Ritter, A.; and Wang, L., eds., Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: H...

    2025

  36. [44]

    Piot, P.; and Parapar, J. 2025 b . Towards Efficient and Explainable Hate Speech Detection via Model Distillation. In Advances in Information Retrieval, 376--392. Cham: Springer Nature Switzerland. ISBN 978-3-031-88711-6

    2025

  37. [45]

    M.; Cercas Curry, A.; Curry, A.; Abercrombie, G.; and Hovy, D

    Plaza-del Arco, F. M.; Cercas Curry, A.; Curry, A.; Abercrombie, G.; and Hovy, D. 2024 a . Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attribution. In Proceedings of the 62nd Annual Meeting of the ACL, 7682--7696. Bangkok, Thailand: ACL

    2024

  38. [46]

    M.; Curry, A

    Plaza-del Arco, F. M.; Curry, A. C.; Paoli, S.; Cercas Curry, A.; and Hovy, D. 2024 b . Divine LL a MA s: Bias, Stereotypes, Stigmatization, and Emotion Representation of Religion in Large Language Models. In Findings of the ACL: EMNLP 2024, 4346--4366. Miami, Florida, USA: ACL

    2024

  39. [47]

    Qian, Y.; Muaz, U.; Zhang, B.; and Hyun, J. W. 2019. Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Function. In Proceedings of the 57th Annual Meeting of the ACL: Student Research Workshop, 223--228. Florence, Italy: ACL

    2019

  40. [48]

    Im not Racist but

    Salinas, A.; Penafiel, L.; McCormack, R.; and Morstatter, F. 2023. "Im not Racist but...": Discovering Bias in the Internal Knowledge of Large Language Models. arXiv:2310.08780

    2023 arXiv

  41. [49]

    Schick, T.; Udupa, S.; and Sch\" u tze, H. 2021. Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP. Transactions of the ACL, 9: 1408–1424

    2021

  42. [50]

    Schmidt, B. 2015. Rejecting the gender binary: a vector-space operation. Ben’s Bookworm Blog

    2015

  43. [51]

    Schweitzer, R.; Perkoulidis, S.; Krome, S.; Ludlow, C.; and Ryan, M. 2005. Attitudes towards refugees: The dark side of prejudice in Australia. Australian Journal of Psychology, 57(3): 170–179

    2005

  44. [52]

    Shrawgi, H.; Rath, P.; Singhal, T.; and Dandapat, S. 2024. Uncovering Stereotypes in Large Language Models: A Task Complexity-based Approach. In Proceedings of the 18th Conference of the European Chapter of the ACL, 1841--1857. St. Julian ' s, Malta: ACL

    2024

  45. [53]

    Spirling, A. 2023. Why open-source generative AI models are an ethical way forward for science. Nature, 616(7957): 413–413

    2023

  46. [54]

    Thakur, V. 2023. Unveiling Gender Bias in Terms of Profession Across LLMs: Analyzing and Addressing Sociological Implications. arXiv:2307.09162

    2023 arXiv

  47. [55]

    van Aken, B.; Risch, J.; Krestel, R.; and L \"o ser, A. 2018. Challenges for Toxic Comment Classification: An In-Depth Error Analysis. In Proceedings of the 2nd Workshop on Abusive Language Online, 33--42. Brussels, Belgium: ACL

    2018

  48. [56]

    Wan, Y.; and Chang, K.-W. 2024. White Men Lead, Black Women Help? Benchmarking Language Agency Social Biases in LLMs. arXiv:2404.10508

    2024 arXiv

  49. [57]

    Xia, M.; Field, A.; and Tsvetkov, Y. 2020. Demoting Racial Bias in Hate Speech Detection. In Proceedings of the Eighth International Workshop on Natural Language Processing for Social Media, 7--14. Online: ACL

    2020

  50. [58]

    Xie, Z.; and Lukasiewicz, T. 2023. An Empirical Analysis of Parameter-Efficient Methods for Debiasing Pre-Trained Language Models. In Proceedings of the 61st Annual Meeting of the ACL, 15730--15745. Toronto, Canada: ACL

    2023

  51. [59]

    Yi, P.; and Zubiaga, A. 2024. ID-XCB: Data-independent Debiasing for Fair and Accurate Transformer-based Cyberbullying Detection. arXiv:2402.16458

    2024 arXiv

  52. [60]

    Zhang, Z.; and Luo, L. 2019. Hate speech detection: A solved problem? The challenging case of long tail on Twitter. Semant. Web, 10(5): 925–945

    2019

  53. [61]

    A.; Kveton, B.; Shao, Y.; Yang, D.; Zamani, H.; ...; and Wang, Y

    Zhang, Z.; Rossi, R. A.; Kveton, B.; Shao, Y.; Yang, D.; Zamani, H.; ...; and Wang, Y. 2024. Personalization of Large Language Models: A Survey. arXiv:2411.00027

    2024 arXiv

  54. [62]

    Zhao, J.; Wang, T.; Yatskar, M.; Ordonez, V.; and Chang, K.-W. 2017. Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints. In Proceedings of the 2017 Conference EMNLP, 2979--2989. Copenhagen, Denmark: ACL

    2017

  55. [63]

    Zhao, J.; Zhou, Y.; Li, Z.; Wang, W.; and Chang, K.-W. 2018. Learning Gender-Neutral Word Embeddings. In Proceedings of the 2018 Conference EMNLP, 4847--4853. Brussels, Belgium: ACL

    2018

  56. [64]

    Zhou, F.; Mao, Y.; Yu, L.; Yang, Y.; and Zhong, T. 2023. Causal-Debias: Unifying Debiasing in Pretrained Language Models and Fine-tuning via Causal Invariant Learning. In Proceedings of the 61st Annual Meeting of the ACL, 4227--4241. Toronto, Canada: ACL

    2023

  57. [65]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  58. [66]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.