Pith. sign in

REVIEW 1 major objections 8 minor 62 references

Steering LLM internals cuts African American English bias 5–20x

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-10 00:22 UTC pith:IEEQSGZY

load-bearing objection Solid contribution: first activation-steering method for dialect bias, plus a useful corpus and a clean finding about syntactic triggers. The translation fidelity concern is real but doesn't undermine the main mitigation claim. the 1 major comments →

arxiv 2607.06845 v1 pith:IEEQSGZY submitted 2026-07-07 cs.CL

LLMs Silently Correct African American English: Auditing and Mitigating Dialect Bias via Activation Steering

Huan Wu , Ali Emami , Muhammad Furquan Hassan , Osaretin Igbinoba , Osakpolor Idusuyi , Osamede Igbinoba , Faiza Khan Khattak , Laleh Seyyed-Kalantari This is my paper
classification cs.CL
keywords biasdialectactivationamericanenglishllmsmodelsreal-aae
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Large language models systematically rewrite African American English (AAE) into Standard American English (SAE), even when the user's input is in AAE. This paper demonstrates that six instruction-tuned LLMs from 14B to 70B parameters consistently prefer SAE continuations over valid AAE alternatives, effectively treating the dialect as something to be corrected rather than as a legitimate, rule-governed variety spoken by over 30 million people. The authors construct the largest real-AAE parallel corpus to date—17,479 triplets of authentic AAE tweets, their SAE translations, and back-translations—and use it to audit and mitigate this bias. They introduce conditional Dialect Group Invariance (cDGI), a metric that filters out translation noise to isolate genuine model bias, and a feature-level analysis showing that syntactic constructions like negative concord (e.g., 'ain't nobody') are universal bias triggers across all tested models. For mitigation, they apply activation steering for the first time to dialect bias: a training-free method that identifies which internal layers carry the dialect-preference signal via causal tracing, then injects a learned 'dialect direction' into those layers at inference time. This approach reduces dialect preference bias 5 to 20 times more effectively than prompting, while preserving the model's fluency in SAE. The finding that bias is embedded in specific internal representations—rather than being a surface-level behavior correctable by instructions—carries implications for how we understand and intervene on all forms of linguistic discrimination in language models.

Core claim

The paper's central claim is that dialect preference bias in LLMs is a structural property of internal model representations, not a surface-level behavior. The authors show this in two steps: first, that the bias is systematically triggered by specific linguistic features (especially syntactic negative concord) across all tested models, suggesting a consistent internal mechanism rather than random variation; and second, that directly modifying hidden-state activations at causally identified layers reduces the bias 5–20x more effectively than prompting, which barely moves the needle. The contrast is load-bearing: if bias were merely a surface-level tendency, instructions should suffice. The 5

What carries the argument

The core mechanism is activation steering via causal tracing. In an offline phase, the method computes a 'dialect direction' at each layer by averaging the difference between hidden-state activations produced by paired AAE and SAE inputs. It then uses corruption-and-restore causal tracing—adding Gaussian noise to AAE feature-token embeddings and measuring which layer restorations most strongly shift the model toward dialect-matched behavior—to identify the four most bias-relevant layers. At inference time, the method injects the normalized dialect direction into those four layers, weighted by their causal importance, shifting the model's internal representations toward treating AAE and SAE等效

Load-bearing premise

The method depends on LLM-generated SAE translations and back-translations being semantically faithful to the original AAE tweets. If the translation model introduces systematic drift—especially drift that correlates with specific AAE features—the 'translation-stable' examples used to isolate genuine model bias may still carry translator artifacts, and some measured bias reduction could reflect correction of translation errors rather than true dialect debiasing.

What would settle it

If one applied activation steering to a held-out set of AAE texts judged by native speakers as semantically equivalent to their SAE counterparts—bypassing LLM translation entirely—and found that the bias reduction disappeared or reversed, the claim that steering addresses genuine dialect bias (rather than translation artifacts) would be undermined.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Activation steering could be extended to other marginalized dialects—Scottish English, Singapore English, Caribbean creoles—using the same extract-direction-and-inject pipeline, since the method is dialect-agnostic and requires only paired inputs.
  • The finding that syntactic constructions are universal bias triggers suggests that dialect bias may be partially learned from training data distributions where certain grammatical structures are systematically associated with lower-quality or informal text, pointing to data curation as a complementary intervention.
  • If activation steering proves effective across additional dialects and model families, it could become a standard test-time deployment tool for organizations using LLMs in hiring, healthcare, or content moderation where dialect speakers are affected.
  • The 5–20x gap between steering and prompting implies that bias audits relying only on prompt-based mitigation may substantially understate the severity of correctable bias, skewing risk assessments.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The fact that prompting barely reduces dialect bias while activation steering does suggests that instruction-tuned models may have learned to comply with surface-level fairness instructions without changing underlying representations—a pattern that may extend to other bias dimensions (race, gender, religion) where prompting shows limited effect.
  • The universality of syntactic constructions as bias triggers across model families trained on different data suggests a common origin in how LLMs internalize prestige hierarchies of language varieties, possibly inherited from the distributional statistics of internet text where non-standard syntax correlates with informal contexts.
  • If the LLM-generated SAE translations contain systematic semantic drift that correlates with dialect features, the cDGI conditioning set could still be contaminated, and the true bias reduction from steering might be partially attributable to correcting translation artifacts rather than genuine dialect debiasing.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 8 minor

Summary. This paper presents an end-to-end framework for auditing and mitigating dialect preference bias against African American English (AAE) in large language models. The authors construct REAL-AAE, a parallel corpus of 17,479 AAE/SAE/AAE_back triplets from natural tweets, validated automatically and by three native AAE speakers. They audit six instruction-tuned LLMs (14B–70B) using a new conditional Dialect Group Invariance (cDGI) metric and generative-preference metrics (perplexity, log-probability bias, forced-choice continuation), finding that all models systematically prefer SAE continuations even under AAE context. A feature-level localization analysis identifies syntactic constructions (especially negative concord) as universal bias triggers. For mitigation, the authors introduce the first application of activation steering to dialect bias: a training-free, test-time method that extracts dialect directions via causal tracing and injects them into bias-relevant layers. The method reduces bias 5–20× more than prompting while preserving SAE fluency.

Significance. The paper makes several strong contributions. The REAL-AAE corpus is the largest real-AAE parallel resource to date, and the AAE-first construction pipeline is a meaningful methodological improvement over synthetic benchmarks. The cDGI metric addresses a real confound (translator artifacts) in prior work. The activation steering method is training-free, test-time, and applicable across model scales, making it practically deployable. The feature-level localization (identifying negative concord as a universal trigger) is a concrete, falsifiable finding that future work can build on. The experimental design is thorough: six models, multiple complementary metrics, confidence intervals via paired bootstrap, ablations on prompt wording and single- vs. multi-layer steering, and a full β sweep. The code and corpus are released. These are substantive contributions that advance both fairness in NLP and mechanistic interpretability.

major comments (1)
  1. §4.1.1, Eq. (1): The cDGI metric conditions on examples where y_AAE = y_AAE_back (prediction preserved under back-translation) AND where AAE_back was judged natural by annotators. The authors state this isolates model bias from translator artifacts. However, there is a subtle selection effect: if the model's bias is feature-specific (e.g., triggered by negative concord, as shown in §6.3), examples containing those features may be systematically excluded from the conditioning set because the model gives different predictions for AAE vs. AAE_back based on surface feature differences, not translation drift. This could make cDGI underestimate true bias for the most problematic feature categories. The paper should discuss this potential interaction between the conditioning logic and feature-specific bias, and ideally report cDGI stratified by feature category to verify that the conditioning设置
minor comments (8)
  1. §4.3, Phase 1b: The choice of α=3.0 for noise scaling is fixed across all models. Given that models differ in scale (14B–70B) and architecture, a brief justification for why a single α is appropriate across this range would strengthen the method's robustness claim.
  2. Table 3 and §6.4: The '5 to 20 times more than prompting' claim is computed on the LP metric. The paper should clarify this is specific to LP and note that the ratio differs across metrics (e.g., DGI improvements are more modest), to avoid overgeneralization.
  3. §3, Translation pipeline: The use of Gemini-3-flash for both forward and back-translation creates a potential systematic bias if the translator itself has dialect preferences. While cDGI is designed to address this, the paper could strengthen its case by reporting translation quality metrics stratified by AAE feature category, to verify that features triggering the most model bias are not also the most poorly translated.
  4. Table 2: cDGI confidence intervals are substantially wider than DGI intervals (e.g., ±0.045 vs. ±0.005 for Gemma-3), reflecting the smaller conditioning set. The paper should note the sample size underlying cDGI in the main text, not only in §6.4.
  5. §4.2.2 and Appendix B.3: The feature lexicon is adapted from Harris et al. (2022) and grouped into four classes. The paper should report the distribution of examples across these four categories in REAL-AAE, as imbalanced category sizes could affect the feature-level analysis in §6.3.
  6. Figure 3: The y-axis for Gemma-3 AAE perplexity is on a very different scale (~14,000) than other models. A log scale or separate panel would improve readability.
  7. §7 (Conclusion): The claim that the framework 'can extend to other marginalized varieties' is stated but not demonstrated. Either soften this claim or provide a brief discussion of what additional validation would be needed.
  8. Table 1: The 'Disc.' and 'Gen.' column abbreviations are not defined in the caption; their meaning is only inferable from context.

Circularity Check

0 steps flagged

No significant circularity found; the central mitigation claim is measured relative to a base model on identical inputs, and the cDGI conditioning logic, while potentially self-referential, does not undermine the headline result.

full rationale

The paper's central claim—that activation steering reduces dialect bias 5 to 20 times more effectively than prompting—is grounded in the LP metric (Eq. 4), which compares the steered model against the unsteered base model on the same held-out evaluation set. This is a standard comparative evaluation, not a self-definitional reduction. The cDGI metric (Eq. 1) conditions on the model's own prediction stability (y_AAE = y_AAE_back), which is mildly self-referential (using the model's biased outputs to define the conditioning set), but this affects only the cDGI column in Table 3, not the LP-based headline. The feature-level localization (§4.2) is an empirical analysis of which markers trigger bias, not a derived prediction. The activation steering method (§4.3) extracts directions from data and applies them at test time; its effectiveness is measured externally, not by construction. The translation pipeline (Gemini-3-flash) is validated both automatically (BERTScore) and by human annotators, providing independent checks. No step in the derivation chain reduces to its inputs by definition or by self-citation chain. The paper is self-contained against external benchmarks and its central claim has independent empirical content.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 3 invented entities

The paper introduces several free parameters (alpha, beta, K, rho, gamma) that are empirically tuned. The core axioms rely on the validity of the TwitterAAE corpus and the accuracy of LLM-based translation. The invented entities (REAL-AAE, cDGI, dialect direction) are well-motivated and supported by the paper's experiments.

free parameters (5)
  • alpha (noise scaling for causal tracing) = 3.0
    Fixed across all models to produce a clear perturbation signal without overly destabilizing the representation. Chosen empirically.
  • beta (steering strength) = [0.1, 0.2, 0.4, 0.6, 0.8, 1.0]
    Swept over a range of values to study the trade-off between bias reduction and language modeling performance. Selected per model.
  • K (number of steering layers) = 4
    Selected as the top-4 causally important layers based on the restoration effect analysis.
  • rho (balance penalty for segmentation) = 0.5
    Set to encourage roughly balanced context-continuation splits.
  • gamma (balance penalty weight) = 0.3
    Fixed balance penalty to trade off semantic shift against split length imbalance.
axioms (4)
  • domain assumption AAE is a rule-governed dialect with its own consistent grammar and phonology, spoken by over 30 million people.
    Stated in the introduction to establish the legitimacy of AAE and frame the bias as linguistic discrimination.
  • domain assumption TwitterAAE corpus identifies tweets from African American accounts via demographic inference from geolocation and network analysis.
    Used as the source data for REAL-AAE. The validity of this demographic inference is assumed.
  • domain assumption Gemini-3-flash can produce accurate SAE translations and AAE back-translations.
    The entire pipeline relies on this assumption. The cDGI metric is designed to control for residual drift, but the baseline translation quality is assumed to be sufficient.
  • standard math Activation steering can modify model behavior by adding learned directions to hidden states at inference without retraining.
    This is the foundational assumption of the activation steering technique, drawn from prior mechanistic interpretability literature.
invented entities (3)
  • REAL-AAE corpus independent evidence
    purpose: To provide a large-scale, real-AAE parallel corpus for auditing and mitigating dialect bias.
    The corpus is constructed from natural tweets and validated by human annotators. It is a tangible resource released with the paper.
  • conditional Dialect Group Invariance (cDGI) independent evidence
    purpose: To isolate true model bias from translator-induced artifacts by conditioning on translation-stable examples.
    The metric is defined mathematically and its behavior is demonstrated on the models. It is a methodological contribution, not a physical entity.
  • Dialect direction independent evidence
    purpose: To represent the direction in activation space that captures the difference between AAE and SAE representations.
    Extracted from model activations using paired AAE-SAE inputs. Its causal effect is verified through the steering experiments.

pith-pipeline@v1.1.0-glm · 25591 in / 2317 out tokens · 304953 ms · 2026-07-10T00:22:16.416573+00:00 · methodology

0 comments
read the original abstract

African American English (AAE), a rule-governed dialect spoken by over 30 million people, is routinely misinterpreted and "corrected" by large language models (LLMs). Across six instruction-tuned LLMs (14B to 70B), we show that state-of-the-art models systematically prefer Standard American English (SAE) continuations even when the preceding context is in AAE, effectively rewriting AAE into SAE. We present an end-to-end framework to audit and mitigate this bias. For auditing, we introduce conditional Dialect Group Invariance (cDGI), which isolates true model bias from translator-induced artifacts, and a feature-level localization analysis that identifies which AAE markers most strongly trigger bias; we find that syntactic constructions, especially negative concord (e.g., "ain't nobody"), are universal triggers across all models. For mitigation, we introduce, to our knowledge, the first application of activation steering to dialect bias: a training-free, test-time method that extracts dialect directions via causal tracing and injects them into bias-relevant layers. Activation steering reduces bias 5 to 20 times more than prompting while preserving SAE fluency. To enable this work, we release REAL-AAE , the largest real-AAE parallel corpus to date: 17,479 AAE/SAE/ AAE_back triplets from natural tweets (2 to 6 times larger than prior real-AAE resources), validated automatically (BERTScore F1 = 0.95) and by three native AAE speakers (83.0% semantic agreement).

Figures

Figures reproduced from arXiv: 2607.06845 by Ali Emami, Faiza Khan Khattak, Huan Wu, Laleh Seyyed-Kalantari, Muhammad Furquan Hassan, Osakpolor Idusuyi, Osamede Igbinoba, Osaretin Igbinoba.

Figure 1
Figure 1. Figure 1: Dialect preference bias on a real AAE tweet. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: REAL-AAE construction pipeline. Starting from authentic AAE tweets, we filter the data, translate AAE into SAE, remove invalid outputs, and back-translate the SAE sentences into AAE to build the final triplet dataset. approach shown to underestimate real-world di￾alect effects (Lin et al., 2025; Gupta et al., 2025). Real-AAE resources instead draw text directly from AAE speakers (Blodgett, 2021; Hassan et … view at source ↗
Figure 3
Figure 3. Figure 3: Continuation perplexity for SAE (x-axis) and AAE (y-axis) continuations, by model and con￾text. Points above y = x indicate SAE preference. All model–context combinations lie above the line, showing systematic SAE preference even under AAE context. Perplexity Analysis [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Feature-level bias localization. For each [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Distribution of BERTScore F1 between origi [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Alignment-based SAE-guided semantic segmentation. First, we select a context-continuation boundary [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Multiple-choice evaluation prompt template. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Overview of dialect debiasing with activation steering. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Bias–utility frontier across steering strengths [PITH_FULL_IMAGE:figures/full_fig_p018_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

62 extracted references · 62 canonical work pages · 9 internal anchors

  1. [1]

    Representation Engineering: A Top-Down Approach to

    Zou, Andy and Phan, Long and Chen, Sarah and Campbell, James and Guo, Phillip and Ren, Richard and Pan, Alexander and Yin, Xuwang and Mazeika, Mantas and Dombrowski, Ann-Kathrin and others , journal=. Representation Engineering: A Top-Down Approach to

    work page

  2. [2]

    Locating and Editing Factual Associations in

    Kevin Meng and David Bau and Alex J Andonian and Yonatan Belinkov , booktitle=. Locating and Editing Factual Associations in. 2022 , url=

    work page 2022

  3. [3]

    Steering Language Models With Activation Engineering

    Activation Addition: Steering Language Models Without Optimization , author=. arXiv preprint arXiv:2308.10248 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  4. [4]

    Proceedings of ICML , year=

    In-Context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering , author=. Proceedings of ICML , year=

    work page

  5. [5]

    Racial Bias in Hate Speech and Abusive Language Detection Datasets

    Davidson, Thomas and Bhattacharya, Debasmita and Weber, Ingmar. Racial Bias in Hate Speech and Abusive Language Detection Datasets. Proceedings of the Third Workshop on Abusive Language Online. 2019. doi:10.18653/v1/W19-3504

    work page doi:10.18653/v1/w19-3504 2019

  6. [6]

    Investigating

    Groenwold, Sophie and Ou, Lily and Parekh, Aesha and Honnavalli, Samhita and Levy, Sharon and Mirza, Diba and Wang, William Yang , booktitle=. Investigating

    work page

  7. [7]

    Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection

    Sap, Maarten and Swayamdipta, Swabha and Vianna, Laura and Zhou, Xuhui and Choi, Yejin and Smith, Noah A. Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022. doi:10...

    work page doi:10.18653/v1/2022.naacl-main.431 2022

  8. [8]

    Demographic Dialectal Variation in Social Media: A Case Study of

    Blodgett, Su Lin and Green, Lisa and O'Connor, Brendan , booktitle =. Demographic Dialectal Variation in Social Media: A Case Study of. 2016 , pages =

    work page 2016

  9. [9]

    T witter U niversal D ependency Parsing for A frican- A merican and Mainstream A merican E nglish

    Blodgett, Su Lin and Wei, Johnny and O ' Connor, Brendan. T witter U niversal D ependency Parsing for A frican- A merican and Mainstream A merican E nglish. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018. doi:10.18653/v1/P18-1131

    work page doi:10.18653/v1/p18-1131 2018

  10. [10]

    VALUE : U nderstanding Dialect Disparity in NLU

    Ziems, Caleb and Chen, Jiaao and Harris, Camille and Anderson, Jessica and Yang, Diyi. VALUE : U nderstanding Dialect Disparity in NLU. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022. doi:10.18653/v1/2022.acl-long.258

    work page doi:10.18653/v1/2022.acl-long.258 2022

  11. [11]

    The Llama 3 Herd of Models

    The Llama 3 Herd of Models , author=. arXiv preprint arXiv:2407.21783 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  12. [12]

    Proceedings of the 36th International Conference on Neural Information Processing Systems , articleno =

    Meng, Kevin and Bau, David and Andonian, Alex and Belinkov, Yonatan , title =. Proceedings of the 36th International Conference on Neural Information Processing Systems , articleno =. 2022 , isbn =

    work page 2022

  13. [13]

    Word Alignment by Fine-tuning Embeddings on Parallel Corpora

    Dou, Zi-Yi and Neubig, Graham. Word Alignment by Fine-tuning Embeddings on Parallel Corpora. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 2021. doi:10.18653/v1/2021.eacl-main.181

    work page doi:10.18653/v1/2021.eacl-main.181 2021

  14. [14]

    Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL) , year =

    Social Bias Frames: Reasoning about Social and Power Implications of Language , author =. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL) , year =

    work page

  15. [15]

    International AI Safety Report

    International ai safety report , author=. arXiv preprint arXiv:2501.17805 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  16. [16]

    Sentence-bert: Sentence embeddings using siamese bert-networks

    Reimers, Nils and Gurevych, Iryna. Sentence- BERT : Sentence Embeddings using S iamese BERT -Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019. doi:10.18653/v1/D19-1410

    work page doi:10.18653/v1/d19-1410 2019

  17. [17]

    Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency , pages=

    Exploring the role of grammar and word choice in bias toward african american english (aae) in hate speech classification , author=. Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency , pages=

    work page 2022

  18. [18]

    E n D ive: A Cross-Dialect Benchmark for Fairness and Performance in Large Language Models

    Gupta, Abhay and Cheung, Jacob and Meng, Philip and Sayyed, Shayan and Zhu, Kevin and Liao, Austen and O ' Brien, Sean. E n D ive: A Cross-Dialect Benchmark for Fairness and Performance in Large Language Models. Findings of the Association for Computational Linguistics: EMNLP 2025. 2025. doi:10.18653/v1/2025.findings-emnlp.913

    work page doi:10.18653/v1/2025.findings-emnlp.913 2025

  19. [19]

    Advances in neural information processing systems , volume=

    Man is to computer programmer as woman is to homemaker? debiasing word embeddings , author=. Advances in neural information processing systems , volume=

    work page

  20. [20]

    Computational Linguistics , volume=

    Bias and fairness in large language models: A survey , author=. Computational Linguistics , volume=. 2024 , publisher=

    work page 2024

  21. [21]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , editor =

    Steering Llama 2 via Contrastive Activation Addition , author =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , editor =. 2024 , address =

    work page 2024

  22. [22]

    Proceedings of the 40th International Conference on Machine Learning (ICML) , year =

    Discovering Latent Knowledge in Language Models without Supervision , author =. Proceedings of the 40th International Conference on Machine Learning (ICML) , year =

    work page

  23. [23]

    CEUR Workshop Proceedings , volume=

    Bias mitigation for large language models using adversarial learning , author=. CEUR Workshop Proceedings , volume=. 2023 , organization=

    work page 2023

  24. [24]

    2024 , eprint=

    Refusal in Language Models Is Mediated by a Single Direction , author=. 2024 , eprint=

    work page 2024

  25. [25]

    Large Language Model Bias Mitigation from the Perspective of Knowledge Editing

    Large language model bias mitigation from the perspective of knowledge editing , author=. arXiv preprint arXiv:2405.09341 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  26. [26]

    Persona Vectors: Monitoring and Controlling Character Traits in Language Models

    Persona vectors: Monitoring and controlling character traits in language models , author=. arXiv preprint arXiv:2507.21509 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  27. [27]

    arXiv preprint arXiv:2408.14845 , year=

    Aavenue: Detecting llm biases on nlu tasks in aave via a novel benchmark , author=. arXiv preprint arXiv:2408.14845 , year=

    work page arXiv

  28. [28]

    AAVENUE : Detecting LLM Biases on NLU Tasks in AAVE via a Novel Benchmark

    Gupta, Abhay and Yurtseven, Ece and Meng, Philip and Zhu, Kevin. AAVENUE : Detecting LLM Biases on NLU Tasks in AAVE via a Novel Benchmark. Proceedings of the Third Workshop on NLP for Positive Impact. 2024. doi:10.18653/v1/2024.nlp4pi-1.28

    work page doi:10.18653/v1/2024.nlp4pi-1.28 2024

  29. [29]

    UMass Amherst Doctoral Dissertations , volume=

    Sociolinguistically driven approaches for just natural language processing , author=. UMass Amherst Doctoral Dissertations , volume=

    work page

  30. [30]

    Multi- VALUE : A Framework for Cross-Dialectal E nglish NLP

    Ziems, Caleb and Held, William and Yang, Jingfeng and Dhamala, Jwala and Gupta, Rahul and Yang, Diyi. Multi- VALUE : A Framework for Cross-Dialectal E nglish NLP. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023. doi:10.18653/v1/2023.acl-long.44

    work page doi:10.18653/v1/2023.acl-long.44 2023

  31. [31]

    2025 , eprint=

    DialectGen: Benchmarking and Improving Dialect Robustness in Multimodal Generation , author=. 2025 , eprint=

    work page 2025

  32. [32]

    , title =

    Fleiss, Joseph L. , title =. Psychological Bulletin , volume =. 1971 , doi =

    work page 1971

  33. [33]

    Evaluating Dialect Robustness of Language Models via Conversation Understanding

    Srirag, Dipankar and Sahoo, Nihar Ranjan and Joshi, Aditya. Evaluating Dialect Robustness of Language Models via Conversation Understanding. Proceedings of the Second Workshop on Scaling Up Multilingual & Multi-Cultural Evaluation. 2025

    work page 2025

  34. [34]

    Linguistic Bias in C hat GPT : Language Models Reinforce Dialect Discrimination

    Fleisig, Eve and Smith, Genevieve and Bossi, Madeline and Rustagi, Ishita and Yin, Xavier and Klein, Dan. Linguistic Bias in C hat GPT : Language Models Reinforce Dialect Discrimination. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024. doi:10.18653/v1/2024.emnlp-main.750

    work page doi:10.18653/v1/2024.emnlp-main.750 2024

  35. [35]

    2024 , eprint=

    Natural Language Processing for Dialects of a Language: A Survey , author=. 2024 , eprint=

    work page 2024

  36. [36]

    Rejected Dialects: Biases Against A frican A merican Language in Reward Models

    Mire, Joel and Aysola, Zubin Trivadi and Chechelnitsky, Daniel and Deas, Nicholas and Zerva, Chrysoula and Sap, Maarten. Rejected Dialects: Biases Against A frican A merican Language in Reward Models. Findings of the Association for Computational Linguistics: NAACL 2025. 2025. doi:10.18653/v1/2025.findings-naacl.417

    work page doi:10.18653/v1/2025.findings-naacl.417 2025

  37. [37]

    Proceedings of the AAAI Symposium Series , author=

    Dialectic Preference Bias in Large Language Models , volume=. Proceedings of the AAAI Symposium Series , author=. 2025 , month=. doi:10.1609/aaaiss.v5i1.35613 , number=

    work page doi:10.1609/aaaiss.v5i1.35613 2025

  38. [38]

    and Pierrehumbert, Janet B

    Lin, Fangru and Mao, Shaoguang and La Malfa, Emanuele and Hofmann, Valentin and de Wynter, Adrian and Wang, Xun and Chen, Si-Qing and Wooldridge, Michael J. and Pierrehumbert, Janet B. and Wei, Furu. Assessing Dialect Fairness and Robustness of Large Language Models in Reasoning Tasks. Proceedings of the 63rd Annual Meeting of the Association for Computat...

    work page doi:10.18653/v1/2025.acl-long.317 2025

  39. [39]

    2024 , note =

    Hofmann, Valentin and Kalluri, Pratyusha Ria and Jurafsky, Dan and King, Sharese , title =. Nature , year =. doi:10.1038/s41586-024-07856-5 , url =

    work page doi:10.1038/s41586-024-07856-5

  40. [40]

    Bias in Language Models: Beyond Trick Tests and Towards RUTE d Evaluation

    Lum, Kristian and Anthis, Jacy Reese and Robinson, Kevin and Nagpal, Chirag and D ' Amour, Alexander Nicholas. Bias in Language Models: Beyond Trick Tests and Towards RUTE d Evaluation. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. doi:10.18653/v1/2025.acl-long.7

    work page doi:10.18653/v1/2025.acl-long.7 2025

  41. [41]

    Philosophical Psychology , volume =

    Uwe Peters , title =. Philosophical Psychology , volume =. 2024 , publisher =. doi:10.1080/09515089.2023.2261976 , URL =

    work page doi:10.1080/09515089.2023.2261976 2024

  42. [42]

    Linguistic profiling , editor =

    Black linguistics: Language, society and politics in Africa and the Americas , author =. Linguistic profiling , editor =. 2003 , pages =

    work page 2003

  43. [43]

    2007 , edition =

    Dialects in Schools and Communities , author =. 2007 , edition =

    work page 2007

  44. [44]

    and King, Sharese , journal =

    Rickford, John R. and King, Sharese , journal =. Language and linguistics on trial: Hearing. 2016 , doi =

    work page 2016

  45. [45]

    Philosophical Psychology , year =

    The complexities of linguistic discrimination , author =. Philosophical Psychology , year =

    work page

  46. [46]

    The Risk of Racial Bias in Hate Speech Detection

    Sap, Maarten and Card, Dallas and Gabriel, Saadia and Choi, Yejin and Smith, Noah A. The Risk of Racial Bias in Hate Speech Detection. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019. doi:10.18653/v1/P19-1163

    work page doi:10.18653/v1/p19-1163 2019

  47. [47]

    Proceedings of the National Academy of Sciences , volume =

    Racial disparities in automated speech recognition , author =. Proceedings of the National Academy of Sciences , volume =. 2020 , doi =

    work page 2020

  48. [48]

    Martin, J. L. and Tang, K. , booktitle =. Understanding Racial Disparities in Automatic Speech Recognition:. 2020 , doi =

    work page 2020

  49. [49]

    Evaluation of A frican A merican Language Bias in Natural Language Generation

    Deas, Nicholas and Grieser, Jessica and Kleiner, Shana and Patton, Desmond and Turcan, Elsbeth and McKeown, Kathleen. Evaluation of A frican A merican Language Bias in Natural Language Generation. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023. doi:10.18653/v1/2023.emnlp-main.421

    work page doi:10.18653/v1/2023.emnlp-main.421 2023

  50. [50]

    Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection

    Gururangan, Suchin and Card, Dallas and Dreier, Sarah and Gade, Emily and Wang, Leroy and Wang, Zeyu and Zettlemoyer, Luke and Smith, Noah A. Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022. doi:10.18653/v1/2022.emnlp-main.165

    work page doi:10.18653/v1/2022.emnlp-main.165 2022

  51. [51]

    Proceedings of the 2018

    Wang, Alex and Singh, Amanpreet and Michael, Julian and Hill, Felix and Levy, Omer and Bowman, Samuel. GLUE : A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. Proceedings of the 2018 EMNLP Workshop B lackbox NLP : Analyzing and Interpreting Neural Networks for NLP. 2018. doi:10.18653/v1/W18-5446

    work page doi:10.18653/v1/w18-5446 2018

  52. [52]

    2020 , eprint=

    SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems , author=. 2020 , eprint=

    work page 2020

  53. [53]

    2025 , eprint=

    Predicting the Target Word of Game-playing Conversations using a Low-Rank Dialect Adapter for Decoder Models , author=. 2025 , eprint=

    work page 2025

  54. [54]

    2025 , eprint=

    A Multi-Agent Framework for Mitigating Dialect Biases in Privacy Policy Question-Answering Systems , author=. 2025 , eprint=

    work page 2025

  55. [55]

    Gemma 3 Technical Report

    Gemma 3 Technical Report , author=. arXiv preprint arXiv:2503.19786 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  56. [56]

    2025 , url=

    Mistral Small 3.1 , author=. 2025 , url=

    work page 2025

  57. [57]

    Mistral Small 3.1 , year =

    work page

  58. [58]

    Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

    Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone , author=. arXiv preprint arXiv:2404.14219 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  59. [59]

    Phi-4 Technical Report

    Phi-4 Technical Report , author=. arXiv preprint arXiv:2412.08905 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  60. [60]

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning , author=. arXiv preprint arXiv:2501.12948 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  61. [61]

    Proceedings of the 29th Symposium on Operating Systems Principles , year=

    Efficient Memory Management for Large Language Model Serving with PagedAttention , author=. Proceedings of the 29th Symposium on Operating Systems Principles , year=

    work page

  62. [62]

    Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations , pages=

    Transformers: State-of-the-Art Natural Language Processing , author=. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations , pages=. 2020 , publisher=

    work page 2020