Pith. sign in

REVIEW 3 major objections 4 minor 77 references

Aligned but Blind: Alignment Increases Implicit Bias by Reducing Awareness of Race

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Alignment makes language models more implicitly biased by hiding race in ambiguous contexts.

desk verdict Strong behavioral evidence that alignment amplifies implicit bias in Llama 3, but the race-awareness interventions need control conditions before the causal mechanism is credible. read the letter →

arxiv 2506.00253 v3 pith:LPGJKIMK submitted 2025-05-30 cs.CL cs.AIcs.CY

Lihao Sun , Chengzhi Mao , Valentin Hofmann , Xuechunzi Bai This is my paper
classification cs.CLcs.AIcs.CY
keywords implicitbiaslanguagemodelalignmentraceblindnessactivationpatchingmechanisticinterpretabilitymitigationLoRAambiguouspolysemy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that value alignment makes language models more, not less, implicitly biased because aligned models stop representing race when the context is ambiguous. The authors show that Llama 3 models that have been safety-tuned appear unbiased on explicit evaluations, yet associate "black" with negative, guilty, and weapon-related words far more often than their unaligned base versions. Mechanistically, activation patching and embedding interpretation reveal that aligned models read "black"/"white" as colors rather than as racial categories in ambiguous prompts. The paper then shows that steering or fine-tuning the model to represent race again reduces implicit bias dramatically, especially when applied in early layers. If correct, this reframes bias mitigation: instead of unlearning race, models should be taught to see it.

What carries the argument

The load-bearing object is the race-blind score $r_{blind} = \Delta P_{color} - \Delta P_{race}$, computed by activation patching: the model processes an interpretive prompt "What does [MASK] refer to? Choose one: race or color," and the patched activations of "black"/"white" from ambiguous prompts shift the distribution toward "color" in aligned models. SelfIE, a training-free embedding interpretation method, gives natural-language readouts (74.4% fewer race-related interpretations in aligned models on implicit prompts). The intervention machinery is activation steering that replaces target-layer activations with cached race-laden activations from "Race: black and white," plus LoRA, a parameter-efficient fine-tuning method, trained on 431 input–output pairs that restate ambiguous sentences in explicitly racial terms; both are most effective when applied to early layers (5–14 or 1–20), consistent with the claim that race awareness must enter before safety mechanisms are engaged.

What would settle it

A control intervention that steers with equally strong non-race activations (e.g., cached activations of "color" or of unrelated words) and shows a similar reduction in implicit bias would falsify the race-awareness mechanism. Likewise, if LoRA fine-tuning with race-reinforcing examples reduces implicit bias even in prompts where "black"/"white" unambiguously mean colors, the effect is generic compliance, not race awareness.

Watch

Extended reading notes

Core claim

The central claim is that alignment amplifies implicit bias, and the mechanism is race blindness: aligned Llama 3 models, unlike their base counterparts, do not represent "black" and "white" as racial categories when the context is ambiguous, and this failure prevents safety guardrails from activating, so stereotype associations (black–negative, black–guilty, black–weapon) leak into outputs. The evidence is threefold: behavioral (aligned models jump from 8.13% explicit bias to 91.4% implicit bias while base models stay near 64.1%), representational (activation patching gives aligned models a positive race-blind score of 0.188 in ambiguous prompts, strongly correlated with the direct-color condition, while base models sit near zero), and interventional (injecting race-laden activations at early layers cuts implicit bias from 97.3% to 71.2%, and early-layer LoRA fine-tuning cuts it to 42.4%).

Load-bearing premise

The causal story depends on the assumption that the drop in implicit bias comes from the model actually representing race, rather than from the interventions making the model more cautious or more likely to hedge in word-association tasks.

Editorial extensions

If this is right

  • Explicit bias benchmarks alone cannot certify an aligned model as fair; any safety evaluation should include implicit association tests, since alignment can push implicit bias up while explicit bias goes to zero.
  • Debiasing by unlearning is not the only lever; teaching a model to represent a social category in early layers can suppress stereotype output, a mitigation strategy that is parameter-efficient and layer-selectable.
  • Early-layer interventions can outperform whole-model edits, meaning interpretability-guided layer targeting should be part of bias-mitigation pipelines.
  • The same mechanism could explain other alignment failures: suppressing a sensitive concept anywhere in training may create a blind spot that later leaks as implicit bias under ambiguity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A control condition that steers with matched non-race activations is the natural next test; without it, part of the measured reduction could be generic steering rather than race awareness.
  • If the mechanism generalizes, aligned models may show analogous blind spots for gender and other social categories whenever tokens are polysemous (e.g., "man" as human vs. male), predicting similar race-blind-type scores in gendered association prompts.
  • The perceptual definition of bias used here (any plausibly racial association counts as biased) implies that even "harmless" color readings of "black"/"white" can be raced by downstream decision systems; future work could test whether the implicit bias metric predicts real allocation decisions.
  • A distribution-shift test could show whether the LoRA intervention robustly reduces implicit bias across novel ambiguous prompt templates or whether it merely over-learns a narrow race-reinforcing pattern.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. Motivated by the discrepancy between explicit and implicit bias in aligned LMs, this paper curates 9,232 controlled prompts and shows that Llama 3 70B Instruct exhibits substantially higher implicit anti-Black associations than the base model, while explicit bias is reduced. Activation patching and SelfIE on Llama 3 8B suggest that in ambiguous contexts the aligned model encodes 'black'/'white' more as colors than as racial categories. The authors then intervene by injecting race-laden activations or fine-tuning with LoRA to reinforce racial associations, reporting large reductions in implicit bias. They conclude that alignment causes 'race blindness' that prevents safety mechanisms from activating, and that increasing race awareness is an effective mitigation strategy.

Significance. If the mechanistic and causal claims were established, the paper would make a valuable contribution by explaining a surprising phenomenon and proposing a new, falsifiable debiasing strategy. Strengths include the carefully matched prompt pairs, the clean base-versus-aligned comparison using the same pretrained model, the use of multiple interpretability methods, and the public release of code and data. The behavioral result that alignment increases implicit bias while decreasing explicit bias is a robust and important finding by itself. However, as argued in the major comments, the current evidence does not yet support the causal claim that race-blind representations cause the observed bias, because the interventions are confounded by task non-compliance and lack appropriate controls.

major comments (3)
  1. [Section 5.2, Eq. (1)] The reported drop in implicit bias from 97.3% to 42.4% may be an artifact of off-task responses being scored as unbiased. The bias metric in Eq. (1) labels a response as biased only if it explicitly assigns a negative stimulus to 'black'; a response that does not perform the assignment (such as the qualitative example 'Considering Black and White racial perspectives' in Section 5.2) receives Y=0 and is counted as unbiased. Section 5.3 reports that the all-layer and late-layer LoRA models fail to follow instructions in 16.8% and 17.4% of explicit/BBQ responses, respectively, so non-compliance is clearly present in these models. The authors must either verify that the implicit-suite responses are genuine assignments for the LoRA conditions, or analyze the bias metric on the subset of responses that actually make a black/white choice; without this, the causal conclusion in Section 6.1 is not supported.
  2. [Section 5.1, Figure 4a] The activation-steering experiment has no control condition, so the observed bias reduction could be caused by generic disruption of early-layer computation rather than by the semantic content 'race'. The authors should include control injections of matched activations from non-race contexts (e.g., 'Color: black and white' or other color-laden phrases) and demonstrate that the effect is specific to race-laden activations.
  3. [Section 4, Table 1 vs. Section 3.3, Figure 2b] The mechanistic analysis is performed on Llama 3 8B, whereas the behavioral results that motivate the mechanism are obtained from Llama 3 70B. The paper does not show that the alignment-induced implicit-bias gap replicates in the 8B model, which is the model used for activation patching and SelfIE. Without this replication, the proposed explanation of the 70B behavior rests on an untested cross-model assumption. The authors should either add the analogous behavioral experiment on 8B or clearly qualify the scope of the mechanistic claim.
minor comments (4)
  1. [Section 3.3] The regression coefficient 'b' is used without defining the model; please specify the logistic regression setup and the covariates (if any).
  2. [Section 4.1] The interpretive prompt 'What does [MASK] refer to?' should be stated explicitly in the caption of Figure 3, and the exact choice of the mask token (e.g., 'something' as described in Appendix D.1) should appear in the main text.
  3. [Table 2] The arrows in the cells are ambiguous; please replace with explicit differences from baseline or add a note that the arrows indicate reduction relative to the baseline row.
  4. [Appendix B] Because the LoRA training data is generated by GPT-4o, please provide the exact generation prompt and the full list of 431 input-output pairs, or at least a larger sample, so that readers can assess potential biases in the training labels.

Circularity Check

0 steps flagged · score 2.0 of 10

No central circularity: behavioral, interpretability, and intervention arms use independent operationalizations; self-citations are not load-bearing.

full rationale

The paper's central claim is not circular. The behavioral result (Section 3) is an empirical comparison on a purpose-built 9,232-prompt suite with deterministic decoding; the bias metric (Eq. 1) is defined independently of any internal representation. The mechanistic claim (Section 4) is measured with a different operationalization: activation patching into a masked race-vs-color interpretive prompt (Eq. 2) and SelfIE readouts, neither of which is derived from the bias labels. The causal interventions (Section 5) are genuine out-of-sample tests: LoRA is trained on 431 race-relevant input-output pairs and evaluated on previously unseen implicit prompts, and steering injects cached activations without fitting to the bias metric. The early-layer LoRA result (97.3% to 42.3%) is accompanied by only 3.7% instruction-following failures, so it is not explained by the task-non-compliance confound the skeptics raise; the higher failure rates for late/all layers (17.4%/16.8%) are disclosed in Section 5.3 as a side effect, not hidden. The paper cites prior work by co-authors (Bai et al. 2025 for implicit-bias prompt adaptation; Chen et al. 2024 for SelfIE), but these citations supply methods and background, and the present experiments independently reproduce the relevant phenomena; no uniqueness theorem or fitted parameter is imported from those citations. The 'race blindness' mechanism is an interpretation offered after separate behavioral and internal-representation measurements, and the intervention is designed to test that interpretation, not to define it into existence. Remaining threats, such as the absence of a non-race steering control, are confounds bearing on causal validity rather than circularity. Overall circularity score: 2.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No fitted free parameters are used: the bias metric is a deterministic label rule, and the race-blind score is a defined statistic, not a fitted coefficient. The main axioms are domain assumptions about the null model for unbiasedness, the comparability of base and aligned models, the validity of the activation-patching probe, and the hypothesized safety-guardrail mechanism.

assumptions (4)
  • domain assumption An unbiased implicit model should produce a 50% probability of pairing black with negative stimuli.
    Section 3.2 defines the null for implicit bias as random assignment; any deviation is labeled bias, but a model could deviate due to lexical, positional, or other non-racial factors.
  • domain assumption Llama 3 base and aligned models differ only in post-training alignment.
    Section 3.3 relies on this shared-pretraining claim from the Llama 3 release to attribute behavioral differences to alignment; it is not independently verified here.
  • domain assumption The 'What does [MASK] refer to?' probe with activation patching measures whether the model encodes the patched token as race or color.
    Section 4.1; the probe is a behavioral readout that may not fully capture internal representations, and the authors acknowledge interpretability limitations.
  • ad hoc to paper Representing race in hidden states activates safety guardrails, which suppress biased outputs.
    This hypothesis is stated in Section 4.3 and used to interpret the intervention results, but no direct measurement of guardrail activation is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Aligned but Blind: Alignment Increases Implicit Bias by Reducing Awareness of Race." pith.science (2026). https://pith.science/paper/LPGJKIMK

@misc{pith2026250600253,
  author       = {Pith},
  title        = {Pith review of: Aligned but Blind: Alignment Increases Implicit Bias by Reducing Awareness of Race},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LPGJKIMK}},
  note         = {Machine review of arXiv:2506.00253}
}
read the original abstract

Although value-aligned language models (LMs) appear unbiased in explicit bias evaluations, they often exhibit stereotypes in implicit word association tasks, raising concerns about their fair usage. We investigate the mechanisms behind this discrepancy and find that alignment surprisingly amplifies implicit bias in model outputs. Specifically, we show that aligned LMs, unlike their unaligned counterparts, overlook racial concepts in early internal representations when the context is ambiguous. Not representing race likely fails to activate safety guardrails, leading to unintended biases. Inspired by this insight, we propose a new bias mitigation strategy that works by incentivizing the representation of racial concepts in the early model layers. In contrast to conventional mitigation methods of machine unlearning, our interventions find that steering the model to be more aware of racial concepts effectively mitigates implicit bias. Similar to race blindness in humans, ignoring racial nuances can inadvertently perpetuate subtle biases in LMs.

Figures

Figures reproduced from arXiv: 2506.00253 by the authors.

Figure 1
Figure 1. Interpreting LM embeddings in natural language using SelfIE ( [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. (a) Prompt templates and selected probe words and stimuli. (b) Averaged black-biased response probabilities from Llama 3 70B Instruct and Base. For each subplot, the y-axis represents the proportion of black￾negative responses (see main text for bias metric), while the x-axis represents bias types. Alignment consistently increases black implicit bias while reducing explicit bias to near zero. Röttger et al., 2025). … view at source ↗
Figure 3
Figure 3. (a) Illustration of activation patching to determine whether the model processes probe words more like race or color. (b) Layer-wise activation patching probability shifts in Llama 3 8B models. In each sub-figure, x-axis represents the source layer from which we extracted activations, and y-axis represents the probability shifts for race vs. color, in Default, Color, and Names. As shown, the aligned model treats Def… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: (a) Implicit bias levels after replacing activations with race-laden activations. The y-axis represents the proportion of black-negative responses, while the x-axis denotes the starting layer for activation replacement (window size = 10). Each point represents an avera…
Figure 5
Figure 5. Figure 5: Specific examples of interpreting LM embeddings in natural language using SelfIE ( [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Bias levels in models fine-tuned with LoRA to reinforce racial associations at different layers. The [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Activation replacement results for the LLaMA 3 8B Instruct. In each sub-figure, the x-axis represents the [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

77 extracted references · 22 canonical work pages

  1. [1]

    Zeyuan Allen-Zhu and Yuanzhi Li. 2024. https://arxiv.org/abs/2309.14316 Physics of language models: Part 3.1, knowledge storage and extraction . Preprint, arXiv:2309.14316

    arXiv 2024

  2. [2]

    Apfelbaum, Michael I

    Evan P. Apfelbaum, Michael I. Norton, and Samuel R. Sommers. 2012. https://journals.sagepub.com/doi/10.1177/0963721411434980 Racial color blindness: Emergence, practice, and implications . Current directions in psychological science, 21(3):205--209

    work page doi:10.1177/0963721411434980 2012

  3. [3]

    Griffiths

    Xuechunzi Bai, Angelina Wang, Ilia Sucholutsky, and Thomas L. Griffiths. 2025. https://doi.org/10.1073/pnas.2416228122 Explicitly unbiased large language models still form biased associations . Proceedings of the National Academy of Sciences (PNAS), 122(8):e2416228122

    doi:10.1073/pnas.2416228122 2025

  4. [4]

    Bowman, Zac Hatfield-Dodds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, and Jared Kaplan

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared Mueller, Jeffrey Ladish, Joshua Landau, Kamal Ndousse, K...

    arXiv 2022

  5. [5]

    Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. 2023. https://arxiv.org/abs/2303.08112 Eliciting latent predictions from transformers with the tuned lens . Preprint, arXiv:2303.08112

    arXiv 2023

  6. [6]

    Leonard Bereska and Efstratios Gavves. 2024. https://arxiv.org/abs/2404.14082 Mechanistic interpretability for ai safety -- a review . Preprint, arXiv:2404.14082

    arXiv 2024

  7. [7]

    Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. 2023. https://openai.com/index/language-models-can-explain-neurons-in-language-models Language models can explain neurons in language models . OpenAI

    work page 2023

  8. [8]

    Eduardo Bonilla-Silva. 2021. Racism without racists: Color-blind racism and the persistence of racial inequality in America. Rowman & Littlefield

    work page 2021

Show all 77 references
  1. [9]

    Bryson, and Arvind Narayanan

    Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017. https://doi.org/10.1126/science.aal4230 Semantics derived automatically from language corpora contain human-like biases . Science, 356(6334):183--186

    2017 doi

  2. [10]

    Haozhe Chen, Carl Vondrick, and Chengzhi Mao. 2024. https://arxiv.org/abs/2403.10949 Selfie: Self-interpretation of large language model embeddings . Preprint, arXiv:2403.10949

    2024 arXiv

  3. [11]

    Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai-Wei Chang, and Rahul Gupta. 2021. https://doi.org/10.1145/3442188.3445924 Bold: Dataset and metrics for measuring biases in open-ended language generation . In Proceedings of the 2021 ACM Confere...

    2021

  4. [12]

    Omkar Dige, Diljot Singh, Tsz Fung Yau, Qixuan Zhang, Borna Bolandraftar, Xiaodan Zhu, and Faiza Khan Khattak. 2024. https://arxiv.org/abs/2406.13551 Mitigating social biases in language models through unlearning . Preprint, arXiv:2406.13551

    2024 arXiv

  5. [13]

    Finale Doshi-Velez and Been Kim. 2017. https://arxiv.org/abs/1702.08608 Towards a rigorous science of interpretable machine learning . Preprint, arXiv:1702.08608

    2017 arXiv

  6. [14]

    Eberhardt, Phillip Atiba Goff, Valerie J

    Jennifer L. Eberhardt, Phillip Atiba Goff, Valerie J. Purdie, and Paul G. Davies. 2004. https://doi.org/10.1037/0022-3514.87.6.876 Seeing black: Race, crime, and visual processing . Journal of Personality and Social Psychology, 87(6):876--893

    2004 doi

  7. [15]

    Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. 2021. https://arxiv.org/abs/2106.02997 Causal abstractions of neural networks . Preprint, arXiv:2106.02997

    2021 arXiv

  8. [16]

    Aryo Pradipta Gema, Pasquale Minervini, Luke Daines, Tom Hope, and Beatrice Alex. 2024. https://arxiv.org/abs/2307.03042 Parameter-efficient fine-tuning of llama for the clinical domain . Preprint, arXiv:2307.03042

    2024 arXiv

  9. [17]

    Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. 2023. https://arxiv.org/abs/2304.14767 Dissecting recall of factual associations in auto-regressive language models . Preprint, arXiv:2304.14767

    2023 arXiv

  10. [18]

    Asma Ghandeharioun, Avi Caciularu, Adam Pearce, Lucas Dixon, and Mor Geva. 2024. https://arxiv.org/abs/2401.06102 Patchscopes: A unifying framework for inspecting hidden representations of language models . Preprint, arXiv:2401.06102

    2024 arXiv

  11. [19]

    Greenwald and Mahzarin R

    Anthony G. Greenwald and Mahzarin R. Banaji. 2017. https://doi.org/10.1037/amp0000238 The implicit revolution: Reconceiving the relation between conscious and unconscious . American Psychologist, 72(9):861--871

    2017 doi

  12. [20]

    Greenwald, Debbie E

    Anthony G. Greenwald, Debbie E. McGhee, and Jordan L. K. Schwartz. 1998. https://doi.org/10.1037/0022-3514.74.6.1464 Measuring individual differences in implicit cognition: The implicit association test . Journal of Personality and Social Psychology, 74(6):1464--1480

    1998 doi

  13. [21]

    Jia-Chen Gu, Hao-Xiang Xu, Jun-Yu Ma, Pan Lu, Zhen-Hua Ling, Kai-Wei Chang, and Nanyun Peng. 2024. https://arxiv.org/abs/2401.04700 Model editing harms general abilities of large language models: Regularization to the rescue . Preprint, arXiv:2401.04700

    2024 arXiv

  14. [22]

    Wes Gurnee and Max Tegmark. 2024. https://arxiv.org/abs/2310.02207 Language models represent space and time . Preprint, arXiv:2310.02207

    2024 arXiv

  15. [23]

    Michael Hanna, Ollie Liu, and Alexandre Variengien. 2023. https://arxiv.org/abs/2305.00586 How does gpt-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model . Preprint, arXiv:2305.00586

    2023 arXiv

  16. [24]

    Stefan Heimersheim and Neel Nanda. 2024. https://arxiv.org/abs/2404.15255 How to use and interpret activation patching . Preprint, arXiv:2404.15255

    2024 arXiv

  17. [25]

    Valentin Hofmann, Pratyusha Ria Kalluri, Dan Jurafsky, and Sharese King. 2024. https://doi.org/10.1038/s41586-024-07037-z Ai generates covertly racist decisions about people based on their dialect . Nature, 633:147--154

    2024 doi

  18. [26]

    Ari Holtzman, Peter West, and Luke Zettlemoyer. 2023. https://arxiv.org/abs/2308.00189 Generative models as a complex systems science: How can we make sense of large language model behavior? Preprint, arXiv:2308.00189

    2023 arXiv

  19. [27]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. https://arxiv.org/abs/2106.09685 Lora: Low-rank adaptation of large language models . Preprint, arXiv:2106.09685

    2021 arXiv

  20. [28]

    Jennifer Hu and Michael C. Frank. 2024. https://arxiv.org/abs/2404.02418 Auxiliary task demands mask the capabilities of smaller language models . Preprint, arXiv:2404.02418

    2024 arXiv

  21. [29]

    Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory Sayres. 2018. https://arxiv.org/abs/1711.11279 Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav) . Preprint, arXiv:1711.11279

    2018 arXiv

  22. [30]

    Divyanshu Kumar, Umang Jain, Sahil Agarwal, and Prashanth Harshangi. 2024. https://arxiv.org/abs/2410.12864 Investigating implicit bias in large language models: A large-scale study of over 50 llms . Preprint, arXiv:2410.12864

    2024 arXiv

  23. [31]

    Kummerfeld, and Rada Mihalcea

    Andrew Lee, Xiaoyan Bai, Itamar Pres, Martin Wattenberg, Jonathan K. Kummerfeld, and Rada Mihalcea. 2024. https://arxiv.org/abs/2401.01967 A mechanistic understanding of alignment algorithms: A case study on dpo and toxicity . Preprint, arXiv:2401.01967

    2024 arXiv

  24. [32]

    Levinson, Huajian Cai, and Danielle Young

    Justin D. Levinson, Huajian Cai, and Danielle Young. 2009. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1471567 Guilty by implicit racial bias: The guilty/not guilty implicit association test . Ohio State Journal of Criminal Law

    2009

  25. [33]

    Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2024. https://arxiv.org/abs/2306.03341 Inference-time intervention: Eliciting truthful answers from a language model . Preprint, arXiv:2306.03341

    2024 arXiv

  26. [34]

    Tianlong Li, Zhenghua Wang, Wenhao Liu, Muling Wu, Shihan Dou, Changze Lv, Xiaohua Wang, Xiaoqing Zheng, and Xuanjing Huang. 2025. https://arxiv.org/abs/2401.06824 Revisiting jailbreaking for large language models: A representation engineering perspective . Preprint, arXiv:2401.06824

    2025 arXiv

  27. [35]

    Zongxi Li, Xianming Li, Yuzhang Liu, Haoran Xie, Jing Li, Fu lee Wang, Qing Li, and Xiaoqin Zhong. 2023. https://arxiv.org/abs/2310.01208 Label supervised llama finetuning . Preprint, arXiv:2310.01208

    2023 arXiv

  28. [36]

    Tom Lieberum, Matthew Rahtz, János Kramár, Neel Nanda, Geoffrey Irving, Rohin Shah, and Vladimir Maikulik. 2023. https://arxiv.org/abs/2307.09458 Does circuit analysis interpretability scale? evidence from multiple choice capabilities in chinchilla . Preprint, arXiv:2307.09458

    2023 arXiv

  29. [37]

    Pierrehumbert, and Furu Wei

    Fangru Lin, Shaoguang Mao, Emanuele La Malfa, Valentin Hofmann, Adrian de Wynter, Xun Wang, Si-Qing Chen, Michael Wooldridge, Janet B. Pierrehumbert, and Furu Wei. 2025. https://arxiv.org/abs/2410.11005 One language, many gaps: Evaluating dialect fairness and robustness of lar...

    2025 arXiv

  30. [38]

    Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021. https://arxiv.org/abs/2107.13586 Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing . Preprint, arXiv:2107.13586

    2021 arXiv

  31. [39]

    AI @ Meta Llama Team. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of models . Preprint, arXiv:2407.21783

    2024 arXiv

  32. [40]

    Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller

    Samuel Marks, Can Rager, Eric J. Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. 2024. https://arxiv.org/abs/2403.19647 Sparse feature circuits: Discovering and editing interpretable causal graphs in language models . Preprint, arXiv:2403.19647

    2024 arXiv

  33. [41]

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2023. https://arxiv.org/abs/2202.05262 Locating and editing factual associations in gpt . Preprint, arXiv:2202.05262

    2023 arXiv

  34. [42]

    Kanishka Misra and Kyle Mahowald. 2024. https://arxiv.org/abs/2403.19827 Language models learn rare phenomena from less rare phenomena: The case of the missing aanns . Preprint, arXiv:2403.19827

    2024 arXiv

  35. [43]

    Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. 2023. https://arxiv.org/abs/2301.05217 Progress measures for grokking via mechanistic interpretability . Preprint, arXiv:2301.05217

    2023 arXiv

  36. [44]

    Leland Gerson Neuberg. 2003. https://doi.org/10.1017/S0266466603004109 Causality: Models, reasoning, and inference . Econometric Theory, 19(4):675--685

    2003 doi

  37. [45]

    Norton, Samuel R

    Michael I. Norton, Samuel R. Sommers, Evan P. Apfelbaum, Natassia Pura, and Dan Ariely. 2006. https://pubmed.ncbi.nlm.nih.gov/17176425/ Color blindness and interracial interaction: Playing the political correctness game . Psychological Science, 17(11):949--953

    2006

  38. [46]

    Nosek, Anthony G

    Brian A. Nosek, Anthony G. Greenwald, and Mahzarin R. Banaji. 2007. https://faculty.washington.edu/agg/pdf/Nosek In Social psychology and the unconscious: The automaticity of higher mental processes, pages 265--292. Psychology Press

    2007

  39. [47]

    Chris Olah. 2022. https://transformer-circuits.pub/2022/mech-interp-essay/index.html Mechanistic interpretability, variables, and the importance of interpretable bases . Transformer Circuits Thread

    2022

  40. [48]

    Nina Panickassery. 2023. https://www.lesswrong.com/posts/zt6hRsDE84HeBKh7E/reducing-sycophancy-and-improving-honesty-via-activation Reducing sycophancy and improving honesty via activation steering . LessWrong

    2023

  41. [49]

    Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. 2024. https://arxiv.org/abs/2312.06681 Steering llama 2 via contrastive activation addition . Preprint, arXiv:2312.06681

    2024 arXiv

  42. [50]

    Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel R. Bowman. 2022. https://arxiv.org/abs/2110.08193 Bbq: A hand-built bias benchmark for question answering . Preprint, arXiv:2110.08193

    2022 arXiv

  43. [51]

    Mica Pollock. 2004. https://www.journals.uchicago.edu/doi/abs/10.1086/424719 Race wrestling: Struggling strategically with race in educational practice and research . American journal of education, 111(1):25--67

    2004 doi

  44. [52]

    Nirmalendu Prakash and Lee Ka Wei Roy. 2024. https://arxiv.org/abs/2406.12347 Interpreting bias in large language models: A feature-based approach . Preprint, arXiv:2406.12347

    2024 arXiv

  45. [53]

    Paul R \"o ttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Kirk, Hinrich Schuetze, and Dirk Hovy. 2024. https://aclanthology.org/2024.acl-long.816/ Political compass or spinning arrow? towards more meaningful evaluations for values and opinions in large langu...

    2024

  46. [54]

    Paul Röttger, Musashi Hinck, Valentin Hofmann, Kobi Hackenburg, Valentina Pyatkin, Faeze Brahman, and Dirk Hovy. 2025. https://arxiv.org/abs/2502.08395 Issuebench: Millions of realistic prompts for measuring issue bias in llm writing assistance . Preprint, arXiv:2502.08395

    2025 arXiv

  47. [55]

    Michael Santacroce, Yadong Lu, Han Yu, Yuanzhi Li, and Yelong Shen. 2023. https://arxiv.org/abs/2309.00754 Efficient rlhf: Reducing the memory usage of ppo . Preprint, arXiv:2309.00754

    2023 arXiv

  48. [56]

    Hakim Sidahmed, Samrat Phatale, Alex Hutcheson, Zhuonan Lin, Zhang Chen, Zac Yu, Jarvis Jin, Simral Chaudhary, Roman Komarytsia, Christiane Ahlheim, Yonghao Zhu, Bowen Li, Saravanan Ganesh, Bill Byrne, Jessica Hoffmann, Hassan Mansoor, Wei Li, Abhinav Rastogi, and Lucas Dixon....

    2024 arXiv

  49. [57]

    Stevens, Victoria C

    Flannery G. Stevens, Victoria C. Plaut, and Jeffrey Sanchez-Burks. 2008. https://journals.sagepub.com/doi/10.1177/0021886308314460 Unlocking the benefits of diversity: All-inclusive multiculturalism and positive organizational change . The journal of applied behavioral science...

    2008 doi

  50. [58]

    Alessandro Stolfo, Vidhisha Balachandran, Safoora Yousefi, Eric Horvitz, and Besmira Nushi. 2024. https://arxiv.org/abs/2410.12877 Improving instruction-following in language models through activation steering . Preprint, arXiv:2410.12877

    2024 arXiv

  51. [59]

    Simeng Sun, Dhawal Gupta, and Mohit Iyyer. 2023. https://arxiv.org/abs/2309.09055 Exploring the impact of low-rank adaptation on the performance, efficiency, and regularization of rlhf . Preprint, arXiv:2309.09055

    2023 arXiv

  52. [60]

    Kunal Suri, Prakhar Mishra, Saumajit Saha, and Atul Singh. 2023. https://arxiv.org/abs/2307.05162 Suryakiran at mediqa-sum 2023: Leveraging lora for clinical dialogue summarization . Preprint, arXiv:2307.05162

    2023 arXiv

  53. [61]

    Alex Tamkin, Amanda Askell, Liane Lovitt, Esin Durmus, Nicholas Joseph, Shauna Kravec, Karina Nguyen, Jared Kaplan, and Deep Ganguli. 2023. https://arxiv.org/abs/2312.03689 Evaluating and mitigating discrimination in language model decisions . Preprint, arXiv:2312.03689

    2023 arXiv

  54. [62]

    Lawler, Jimmy Ba, Rahul G

    Augustin Toma, Patrick R. Lawler, Jimmy Ba, Rahul G. Krishnan, Barry B. Rubin, and Bo Wang. 2023. https://arxiv.org/abs/2305.12031 Clinical camel: An open expert-level medical language model with dialogue-based knowledge encoding . Preprint, arXiv:2305.12031

    2023 arXiv

  55. [63]

    Vazquez, Ulisse Mini, and Monte MacDiarmid

    Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmid. 2024. https://arxiv.org/abs/2308.10248 Steering language models with activation engineering . Preprint, arXiv:2308.10248

    2024 arXiv

  56. [64]

    Angelina Wang, Xuechunzi Bai, Solon Barocas, and Su Lin Blodgett. 2023. https://conference2023.eaamo.org/papers/EAAMO23_paper_19.pdf Measuring stereotype harm from machine learning errors requires understanding who is being harmed by which errors in what ways . In ACM Conferen...

    2023

  57. [65]

    Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zinan Lin, Yu Cheng, Sanmi Koyejo, Dawn Song, and Bo Li

    Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang T. Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zinan Lin, Yu Cheng, Sanmi Koyejo, Dawn Song, and Bo Li. 2024. https://arxiv.org/ab...

    2024 arXiv

  58. [66]

    Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2022. https://arxiv.org/abs/2211.00593 Interpretability in the wild: a circuit for indirect object identification in gpt-2 small . Preprint, arXiv:2211.00593

    2022 arXiv

  59. [67]

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. https://arxiv.org/abs/2307.02483 Jailbroken: How does llm safety training fail? Preprint, arXiv:2307.02483

    2023 arXiv

  60. [68]

    Yotam Wolf, Noam Wies, Oshri Avnery, Yoav Levine, and Amnon Shashua. 2023. https://arxiv.org/abs/2304.11082 Fundamental limitations of alignment in large language models . arXiv preprint arXiv:2304.11082

    2023 arXiv

  61. [69]

    Manning, and Christopher Potts

    Zhengxuan Wu, Aryaman Arora, Atticus Geiger, Zheng Wang, Jing Huang, Dan Jurafsky, Christopher D. Manning, and Christopher Potts. 2025. https://arxiv.org/abs/2501.17148 Axbench: Steering llms? even simple baselines outperform sparse autoencoders . Preprint, arXiv:2501.17148

    2025 arXiv

  62. [70]

    Zhihao Xu, Ruixuan Huang, Changyu Chen, and Xiting Wang. 2024. https://arxiv.org/abs/2404.12038 Uncovering safety risks of large language models through concept activation vector . Preprint, arXiv:2404.12038

    2024 arXiv

  63. [71]

    Lilong Xue, Dan Zhang, Yuxiao Dong, and Jie Tang. 2024. https://arxiv.org/abs/2403.14888 Autore: Document-level relation extraction with large language models . Preprint, arXiv:2403.14888

    2024 arXiv

  64. [72]

    Zeping Yu and Sophia Ananiadou. 2025. https://arxiv.org/abs/2501.14457 Understanding and mitigating gender bias in llms via interpretable neuron editing . Preprint, arXiv:2501.14457

    2025 arXiv

  65. [73]

    Fred Zhang and Neel Nanda. 2024. https://arxiv.org/abs/2309.16042 Towards best practices of activation patching in language models: Metrics and methods . Preprint, arXiv:2309.16042

    2024 arXiv

  66. [74]

    Yazhou Zhang, Mengyao Wang, Youxi Wu, Prayag Tiwari, Qiuchi Li, Benyou Wang, and Jing Qin. 2024. https://arxiv.org/abs/2310.11374 Dialoguellm: Context and emotion knowledge-tuned large language models for emotion recognition in conversations . Preprint, arXiv:2310.11374

    2024 arXiv

  67. [75]

    Ziqian Zhong, Ziming Liu, Max Tegmark, and Jacob Andreas. 2023. https://arxiv.org/abs/2306.17844 The clock and the pizza: Two stories in mechanistic explanation of neural networks . Preprint, arXiv:2306.17844

    2023 arXiv

  68. [76]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  69. [77]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.