Pith. sign in

REVIEW 3 major objections 300 references

LLM-human agreement peaks with one to three demographic attributes and falls when the full set is used.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 10:36 UTC pith:ZC63ACZO

load-bearing objection Solid combinatorial study: more demographic attributes in the prompt reliably hurt LLM–human agreement after 1–3 high-signal ones; the rest is useful diagnostics with known correlational limits. the 3 major comments →

arxiv 2607.10590 v1 pith:ZC63ACZO submitted 2026-07-12 cs.CL

Demographic Prompting at Scale: When More Attributes Hurt LLM--Human Agreement

Mahammed Kamruzzaman , Shrabon Kumar Das , Gene Louis Kim This is my paper
classification cs.CL
keywords demographic promptingLLM-human alignmentover-specificationdirectional coherenceneuron probingsubjective NLP taskspersona prompts
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks how much demographic detail you should put into a prompt when you want a language model to match a particular group's judgments on subjective tasks such as toxicity, sentiment, politeness, offensiveness, and emotion. The authors run every combination of available demographic attributes across five open-source models and five datasets. They find that agreement with human labels is usually highest when the prompt names only one to three high-signal attributes; stuffing in the full demographic profile reliably hurts. Simply knowing which attributes most influence human labels is not enough to choose the right ones. What matters is whether each attribute carries a learnable and directionally coherent lexical signal that a single persona can exploit. Neuron-level measurements further show that specialized activation tracks alignment gains only when that signal is coherent; more activated neurons alone do not make a model more steerable. The practical takeaway is that demographic prompting is not a one-size-fits-all lever: less is often more, and attribute quality matters more than quantity.

Core claim

Alignment between LLM predictions and human annotations consistently peaks with one to three high-signal demographic attributes and degrades under the full attribute set, establishing a clear over-specification threshold. The magnitude of an attribute's influence on human labels does not predict alignment gains; both learnability of word-demographic patterns and the directional coherence of those patterns must be considered jointly. Specialized neuron activation correlates with better alignment only when the underlying signal is coherent, and activation volume alone does not imply steerability.

What carries the argument

The three-level diagnostic of attribute signal quality: magnitude (SHAP importance of demographics for human labels), learnability (LinearSVC kappa on word-by-demographic interaction features), and directional coherence (Fisher-averaged Spearman correlation of subgroup lexical weights). This framework, together with combinatorial enumeration of every attribute subset and specialized-neuron probing against a no-persona baseline, is what carries the argument that more demographic detail is not better.

Load-bearing premise

The paper assumes that the Fisher-averaged rank correlation of word-demographic interaction weights is a valid measure of the kind of directional coherence a persona prompt can actually exploit, and that this measure generalizes beyond the five English datasets and five mid-size open models tested.

What would settle it

Run the same full combinatorial prompting and coherence analysis on a held-out subjective task or a substantially larger proprietary model; if full-attribute prompts then outperform the one-to-three-attribute peak, or if high-coherence attributes no longer predict alignment gains, the over-specification threshold and the coherence claim fail.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. The paper studies how supplying annotator demographic attributes as prompt cues affects LLM–human agreement on five subjective tasks (toxicity, sentiment, politeness, offensiveness, emotion), using five open-source models (3B–12B). It enumerates essentially all attribute combinations from single-attribute through full-attribute prompts, and reports three findings: (i) agreement typically peaks with one to three high-signal attributes and degrades under the full set (an over-specification threshold); (ii) raw demographic influence on human labels (SHAP) does not predict which attributes help LLMs, whereas jointly considering lexical learnability (LSVC word×demographic interactions) and directional coherence (Fisher-averaged Spearman ¯ρ of subgroup weight vectors) better organizes when prompting helps or hurts; (iii) specialized-neuron activation proportion correlates with alignment gains only under coherent signals, and high activation volume (notably DeepSeek) does not imply steerability. Alignment is measured primarily by quadratic-weighted Cohen’s κ (accuracy for emotion), with bootstrap tests and three prompt paraphrases.

Significance. If the over-specification pattern holds more broadly, the work supplies a concrete, actionable constraint for persona-style demographic prompting: more attributes are not better, and full-attribute prompts are often actively harmful. The exhaustive combinatorial design across five tasks and five models is a genuine advance over prior single-attribute or all-vs-none comparisons, and the three-level dataset-side diagnostic (magnitude / learnability / coherence) plus the first application of specialized-neuron probing to demographic alignment are useful contributions for both practitioners and interpretability work. Strengths include paired bootstrap CIs (B=10,000), multi-paraphrase averaging, explicit significance markers in the main tables, an honest limitations section (English-only, small models, small-n rank correlations, correlational neurons, distillation caveat for DeepSeek), and practical task/model recommendations in Appendix J. The central combinatorial claim is well supported within the stated scope; the explanatory RQ2/RQ3 framework is more provisional but still informative.

major comments (3)
  1. Figures 1–5 and the accompanying narrative report the best κ (or accuracy) among all combinations of a given size k. For tasks with many attributes (e.g., Toxicity, n=8), C(8,3)=56 and C(8,4)=70, so the intermediate-k peaks are maxes over large candidate sets, while the full-attribute point is a single configuration. This selection asymmetry can inflate the apparent “peak at 1–3” even if the degradation of the full set is real. Please either (a) also report mean/median (and quantiles) of κ over all configs of size k, or (b) apply a multiple-comparison-aware procedure when declaring a size-k optimum, and state clearly that the over-specification claim rests primarily on full-set degradation vs. compact optima rather than on the precise location of the max.
  2. §4.2 / Tables 1–2: the Spearman correlations that underwrite the three-level framework are computed over only 5–9 attributes per task (as few as five for Politeness and Offensiveness). At this n, rank swaps move ρ substantially; several “significant” cells rest on very small samples. The Limitations section already flags this, but the Abstract and §5 still present learnability+coherence as a principal finding on equal footing with the combinatorial result. Please either aggregate evidence more robustly (e.g., task-pooled or model-pooled tests, bootstrap of the rank correlations themselves) or demote the language so that RQ2 is framed as a diagnostic hypothesis supported by consistent directional patterns, not as a firmly established predictor.
  3. §4.2 and Appendix G: two concrete cases sit awkwardly with the joint learnability+coherence story. On Offensiveness, race has low Fisher ¯ρ (+0.058) yet is the only attribute that significantly improves any model (Qwen); on Emotion, age drives the largest gains for Mistral/Qwen despite ranking below country/field_of_study on both LSVC accuracy and Fisher ¯ρ. These counterexamples do not refute the framework, but they show that coherence+learnability is neither necessary nor sufficient in every model–task cell. The main text should discuss these cases explicitly and state what residual factors (architecture, baseline strength, subgroup granularity) remain after the three-level account.

Circularity Check

0 steps flagged

Empirical combinatorial study; no derivation reduces to its own inputs by construction.

full rationale

The paper’s three principal findings are observational results from exhaustive prompt enumeration, dataset-side classifiers, and neuron activation probes—not first-principles derivations. Alignment (quadratic-weighted Cohen’s κ / accuracy) is computed against held-out human labels under every attribute combination; the reported peak at 1–3 attributes and degradation under the full set (Figures 1–5, Tables 12–16) is therefore a measured outcome, not a quantity forced by a fitted parameter or a definitional identity. SHAP importance, LSVC κ on word×demographic interactions, and Fisher-averaged ¯ρ are diagnostic correlates used only to interpret which attributes help; none reappears as a ‘prediction’ of the same quantity that was fitted. Neuron specialization proportion p is likewise a measured activation statistic correlated with alignment, not a self-defining claim. Self-citations (Kamruzzaman et al. 2024/2025) appear only as related-work context on persona bias and do not underwrite the combinatorial threshold, the coherence diagnostic, or the neuron results. No uniqueness theorem, ansatz smuggled via citation, or renaming of a known closed-form result is present. The work is self-contained against its external human-annotation benchmarks within the stated English 3B–12B scope.

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 0 invented entities

The paper is an empirical measurement study. Its load-bearing choices are experimental design decisions (which datasets, which models, k=10, top-200 words, Fisher averaging) rather than free physical constants or invented theoretical entities. No new particles, forces, or conserved quantities are postulated.

free parameters (2)
  • Key-neuron top-k = 10
    Chosen as k=10 after an ablation over {5,10,15,...,100}; the main neuron–alignment correlations depend on this threshold.
  • Top-200 interaction words per attribute = 200
    Vocabulary size used to compute pairwise Spearman correlations that become Fisher ¯ρ; different cutoffs could alter coherence rankings.
axioms (3)
  • domain assumption Quadratic-weighted Cohen’s κ (or accuracy for Emotion) is an appropriate primary measure of LLM–human alignment on these ordinal/categorical labels.
    Stated in §3.4; alternative metrics (macro/micro accuracy) are reported but the narrative rests on κ.
  • domain assumption Specialized-neuron proportion p = |N_demo \ N_baseline| / |N_demo| indexes internal engagement with a demographic cue.
    Adapted from Ying et al. (2025); treated as a testable hypothesis rather than proven mechanism (§3.6, §4.3).
  • ad hoc to paper Fisher z-averaged Spearman correlation of LSVC word×demographic weights measures directional coherence exploitable by a single persona prompt.
    Defined in §3.5; the second principal finding depends on this operationalization.

pith-pipeline@v1.1.0-grok45 · 36747 in / 2525 out tokens · 26519 ms · 2026-07-14T10:36:12.318209+00:00 · methodology

0 comments
read the original abstract

We investigate how annotator demographic attributes, supplied as prompt cues, shape the alignment between large language model (LLM) predictions and human annotations across five tasks. Using five open-source LLMs, we systematically vary the number and composition of demographic components in the prompt, spanning every combination from single-attribute through full-attribute configurations. Our experiments reveal three principal findings. First, alignment consistently peaks with one to three high-signal attributes and degrades under the full attribute set, establishing a clear over-specification threshold. Second, the overall magnitude of demographic influence on human annotations does not predict which attributes improve LLM alignment; instead, both the learnability and the directional coherence of each attribute's annotation signal need to be considered jointly. Third, neuron probing reveals that specialized activation correlates with alignment gains only under coherent annotation signals, and that activation volume alone does not imply steerability. Together, these results demonstrate that demographic prompting is not a monolithic intervention: its utility is highly context-dependent, shaped by attribute signal quality, task characteristics, and model architecture.

Figures

Figures reproduced from arXiv: 2607.10590 by Gene Louis Kim, Mahammed Kamruzzaman, Shrabon Kumar Das.

Figure 1
Figure 1. Figure 1: Best quadratic-weighted Cohen’s κ on the Toxicity task by the number of demographic prompt components across five LLMs. Specialized neuron set. Each base text appears under both a baseline (no-demographic) prompt and one or more identity-conditioned prompts. For a matched pair sharing the same base text, the spe￾cialized neuron set is S = Ndemo \ Nbaseline, i.e., the neurons active under the demographic pr… view at source ↗
Figure 5
Figure 5. Figure 5: Best accuracy on the [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 4
Figure 4. Figure 4: Best quadratic-weighted Cohen’s κ on the Of￾fensiveness task by the number of demographic prompt components across five LLMs. ery single-attribute prompt exceeding its baseline, though combining attributes degrades these gains. On Offensiveness, race is the only attribute that can selectively improve alignment: Qwen peaks with race alone, and race appears in every top Mistral configuration (see [PITH_FULL… view at source ↗
Figure 6
Figure 6. Figure 6: Ablation study for Toxicity and Sentiment Task [PITH_FULL_IMAGE:figures/full_fig_p024_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

300 extracted references · 7 canonical work pages

  1. [1]

    Publications Manual , year = "1983", publisher =

    1983

  2. [2]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243

    doi:10.1145/322234.322243 1981

  3. [3]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

  4. [4]

    Dan Gusfield , title =. 1997

    1997

  5. [5]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    2015

  6. [6]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =

  7. [7]

    2024 , eprint=

    Prompting Techniques for Reducing Social Bias in LLMs through System 1 and System 2 Cognitive Processes , author=. 2024 , eprint=

    2024

  8. [8]

    StereoSet: Measuring stereotypical bias in pretrained language models , author=. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , pages=

  9. [9]

    and Kim, Gene

    Kamruzzaman, Mahammed and Shovon, Md. and Kim, Gene. Investigating Subtler Biases in LLM s: Ageism, Beauty, Institutional, and Nationality Bias in Generative Models. Findings of the Association for Computational Linguistics ACL 2024. 2024

    2024

  10. [10]

    Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages=

    CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models , author=. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages=

    2020

  11. [11]

    Proceedings of the ACM collective intelligence conference , pages=

    Gender bias and stereotypes in large language models , author=. Proceedings of the ACM collective intelligence conference , pages=

  12. [12]

    Global is Good, Local is Bad?

    " Global is Good, Local is Bad?": Understanding Brand Bias in LLMs , author=. arXiv preprint arXiv:2406.13997 , year=

    Pith/arXiv arXiv

  13. [13]

    Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    French CrowS-pairs: Extending a challenge dataset for measuring social bias in masked language models to a language other than English , author=. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  14. [14]

    Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    IndiBias: A Benchmark Dataset to Measure Social Biases in Language Models for Indian Context , author=. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    2024

  15. [15]

    Evaluation Campaign of Natural Language Processing and Speech Tools for Italian , year=

    Haspeede 2@ evalita2020: Overview of the evalita 2020 hate speech detection task , author=. Evaluation Campaign of Natural Language Processing and Speech Tools for Italian , year=

    2020

  16. [16]

    Proceedings of the Fifth Arabic Natural Language Processing Workshop , pages=

    AraWEAT: Multidimensional Analysis of Biases in Arabic Word Embeddings , author=. Proceedings of the Fifth Arabic Natural Language Processing Workshop , pages=

  17. [17]

    BanMANI: A Dataset to Identify Manipulated Social Media News in Bangla , author=. Proceedings of the Workshop on Computational Terminology in NLP and Translation Studies (ConTeNTS) Incorporating the 16th Workshop on Building and Using Comparable Corpora (BUCC) , pages=

  18. [18]

    arXiv preprint arXiv:2407.03536 , year=

    Social Bias in Large Language Models For Bangla: An Empirical Study on Gender and Religious Bias , author=. arXiv preprint arXiv:2407.03536 , year=

    Pith/arXiv arXiv

  19. [19]

    Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP) , pages=

    Toward cultural bias evaluation datasets: The case of Bengali gender, religious, and national identity , author=. Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP) , pages=

  20. [20]

    Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP) , pages=

    An Empirical Study of Gendered Stereotypes in Emotional Attributes for Bangla in Multilingual Large Language Models , author=. Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP) , pages=

  21. [21]

    Stereotyping Norwegian salmon: An inventory of pitfalls in fairness benchmark datasets , author=. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , pages=

  22. [22]

    arXiv preprint arXiv:2303.08774 , year=

    Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=

    Pith/arXiv arXiv

  23. [23]

    Advances in neural information processing systems , volume=

    Language models are few-shot learners , author=. Advances in neural information processing systems , volume=

  24. [24]

    arXiv preprint arXiv:2307.09288 , year=

    Llama 2: Open foundation and fine-tuned chat models , author=. arXiv preprint arXiv:2307.09288 , year=

    Pith/arXiv arXiv

  25. [25]

    arXiv preprint arXiv:2310.06825 , year=

    Mistral 7B , author=. arXiv preprint arXiv:2310.06825 , year=

    Pith/arXiv arXiv

  26. [26]

    arXiv preprint arXiv:2312.11805 , year=

    Gemini: a family of highly capable multimodal models , author=. arXiv preprint arXiv:2312.11805 , year=

    Pith/arXiv arXiv

  27. [27]

    arXiv preprint arXiv:2408.00118 , year=

    Gemma 2: Improving open language models at a practical size , author=. arXiv preprint arXiv:2408.00118 , year=

    Pith/arXiv arXiv

  28. [28]

    arXiv preprint arXiv:2311.04892 , year=

    Bias runs deep: Implicit reasoning biases in persona-assigned llms , author=. arXiv preprint arXiv:2311.04892 , year=

    Pith/arXiv arXiv

  29. [29]

    `` Global is Good, Local is Bad? '' : Understanding Brand Bias in LLM s

    Kamruzzaman, Mahammed and Nguyen, Hieu and Kim, Gene. `` Global is Good, Local is Bad? '' : Understanding Brand Bias in LLM s. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024

    2024

  30. [30]

    G lobal V oices, Local Biases: Socio-Cultural Prejudices across Languages

    Mukherjee, Anjishnu and Raj, Chahat and Zhu, Ziwei and Anastasopoulos, Antonios. G lobal V oices, Local Biases: Socio-Cultural Prejudices across Languages. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023. doi:10.18653/v1/2023.emnlp-main.981

    doi:10.18653/v1/2023.emnlp-main.981 2023

  31. [31]

    Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Prompting

    Beck, Tilman and Schuff, Hendrik and Lauscher, Anne and Gurevych, Iryna. Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Prompting. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 2024

    2024

  32. [32]

    Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLM s

    Sun, Huaman and Pei, Jiaxin and Choi, Minje and Jurgens, David. Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLM s. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers). 2025. d...

    doi:10.18653/v1/2025.naacl-short.71 2025

  33. [33]

    arXiv preprint arXiv:2411.08977 , year=

    Robustness and Confounders in the Demographic Alignment of LLMs with Human Perceptions of Offensiveness , author=. arXiv preprint arXiv:2411.08977 , year=

    Pith/arXiv arXiv

  34. [34]

    arXiv preprint arXiv:2410.08820 , year=

    Which Demographics do LLMs Default to During Annotation? , author=. arXiv preprint arXiv:2410.08820 , year=

    Pith/arXiv arXiv

  35. [35]

    arXiv preprint arXiv:2505.11795 , year=

    The Effects of Demographic Instructions on LLM Personas , author=. arXiv preprint arXiv:2505.11795 , year=

    Pith/arXiv arXiv

  36. [36]

    arXiv preprint arXiv:2503.23243 , year=

    Evaluating how LLM annotations represent diverse views on contentious topics , author=. arXiv preprint arXiv:2503.23243 , year=

    Pith/arXiv arXiv

  37. [37]

    Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attribution

    Plaza-del-Arco, Flor Miriam and Cercas Curry, Amanda and Curry, Alba and Abercrombie, Gavin and Hovy, Dirk. Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attribution. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024. doi:10.18653/v1/2024.acl-long.415

    doi:10.18653/v1/2024.acl-long.415 2024

  38. [38]

    , author=

    Evidence for universality and cultural variation of differential emotion response patterning. , author=. Journal of personality and social psychology , volume=. 1994 , publisher=

    1994

  39. [39]

    Seventeenth Symposium on Usable Privacy and Security (SOUPS 2021) , pages=

    Designing toxic content classification for a diversity of perspectives , author=. Seventeenth Symposium on Usable Privacy and Security (SOUPS 2021) , pages=

    2021

  40. [40]

    Proceedings of the 2018 chi conference on human factors in computing systems , pages=

    Addressing age-related bias in sentiment analysis , author=. Proceedings of the 2018 chi conference on human factors in computing systems , pages=

    2018

  41. [41]

    Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops , month =

    Aflalo, Amit and Bagon, Shai and Kashti, Tamar and Eldar, Yonina , title =. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops , month =. 2023 , pages =

    2023

  42. [42]

    Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing - Natural Language Processing in the Generative AI era , month =

    Kamruzzaman, Mahammed and Kim, Gene Louis , title =. Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing - Natural Language Processing in the Generative AI era , month =. 2025 , address =

    2025

  43. [43]

    , author=

    Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , author=. Psychological bulletin , volume=. 1968 , publisher=

    1968

  44. [44]

    biometrics , pages=

    The measurement of observer agreement for categorical data , author=. biometrics , pages=. 1977 , publisher=

    1977

  45. [45]

    Advances in Neural Information Processing Systems , volume=

    How do large language models handle multilingualism? , author=. Advances in Neural Information Processing Systems , volume=

  46. [46]

    arXiv preprint arXiv:2504.07440 , year=

    Model utility law: Evaluating llms beyond performance through mechanism interpretable metric , author=. arXiv preprint arXiv:2504.07440 , year=

    Pith/arXiv arXiv

  47. [47]

    Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models

    Tang, Tianyi and Luo, Wenyang and Huang, Haoyang and Zhang, Dongdong and Wang, Xiaolei and Zhao, Xin and Wei, Furu and Wen, Ji-Rong. Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024. doi:10.18653/v1...

    doi:10.18653/v1/2024.acl-long.309 2024

  48. [48]

    Intrinsic Test of Unlearning Using Parametric Knowledge Traces

    Hong, Yihuai and Yu, Lei and Yang, Haiqin and Ravfogel, Shauli and Geva, Mor. Intrinsic Test of Unlearning Using Parametric Knowledge Traces. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. doi:10.18653/v1/2025.emnlp-main.985

    doi:10.18653/v1/2025.emnlp-main.985 2025

  49. [49]

    Disentangling Language and Culture for Evaluating Multilingual Large Language Models

    Ying, Jiahao and Tang, Wei and Zhao, Yiran and Cao, Yixin and Rong, Yu and Zhang, Wenxuan. Disentangling Language and Culture for Evaluating Multilingual Large Language Models. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. doi:10.18653/v1/2025.acl-long.1082

    doi:10.18653/v1/2025.acl-long.1082 2025

  50. [50]

    2015 , publisher=

    Best-worst scaling: Theory, methods and applications , author=. 2015 , publisher=

    2015

  51. [51]

    Robustness and Confounders in the Demographic Alignment of LLM s with Human Perceptions of Offensiveness

    Alipour, Shayan and Sen, Indira and Samory, Mattia and Mitra, Tanu. Robustness and Confounders in the Demographic Alignment of LLM s with Human Perceptions of Offensiveness. Findings of the Association for Computational Linguistics: ACL 2025. 2025. doi:10.18653/v1/2025.findings-acl.1136

    doi:10.18653/v1/2025.findings-acl.1136 2025

  52. [52]

    Which Demographics do LLM s Default to During Annotation?

    Sch. Which Demographics do LLM s Default to During Annotation?. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. doi:10.18653/v1/2025.acl-long.848

    work page doi:10.18653/v1/2025.acl-long.848 2025

  53. [53]

    Proceedings of the National Academy of Sciences , volume=

    ChatGPT outperforms crowd workers for text-annotation tasks , author=. Proceedings of the National Academy of Sciences , volume=. 2023 , publisher=

    2023

  54. [54]

    arXiv preprint arXiv:2304.06588 , year=

    Chatgpt-4 outperforms experts and crowd workers in annotating political twitter messages with zero-shot learning , author=. arXiv preprint arXiv:2304.06588 , year=

    Pith/arXiv arXiv

  55. [55]

    From Anger to Joy: How Nationality Personas Shape Emotion Attribution in Large Language Models

    Kamruzzaman, Mahammed and Al Monsur, Abdullah and Kim, Gene Louis and Chhabra, Anshuman. From Anger to Joy: How Nationality Personas Shape Emotion Attribution in Large Language Models. Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computationa...

    2025

  56. [56]

    A Woman is More Culturally Knowledgeable than A Man?

    " A Woman is More Culturally Knowledgeable than A Man?": The Effect of Personas on Cultural Norm Interpretation in LLMs , author=. arXiv preprint arXiv:2409.11636 , year=

    Pith/arXiv arXiv

  57. [57]

    arXiv preprint arXiv:2303.18223 , volume=

    A survey of large language models , author=. arXiv preprint arXiv:2303.18223 , volume=

    Pith/arXiv arXiv

  58. [58]

    Transactions of the Association for Computational Linguistics , volume=

    Dealing with disagreements: Looking beyond the majority vote in subjective annotations , author=. Transactions of the Association for Computational Linguistics , volume=. 2022 , publisher=

    2022

  59. [59]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=

    The ecological fallacy in annotation: Modeling human label variation goes beyond sociodemographics , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=

  60. [60]

    Steering Llama 2 via Contrastive Activation Addition

    Rimsky, Nina and Gabrieli, Nick and Schulz, Julian and Tong, Meg and Hubinger, Evan and Turner, Alexander. Steering Llama 2 via Contrastive Activation Addition. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024. doi:10.18653/v1/2024.acl-long.828

    doi:10.18653/v1/2024.acl-long.828 2024

  61. [61]

    arXiv preprint arXiv:2406.10130 , year=

    The devil is in the neurons: Interpreting and mitigating social biases in pre-trained language models , author=. arXiv preprint arXiv:2406.10130 , year=

    Pith/arXiv arXiv

  62. [62]

    arXiv preprint arXiv:2501.14457 , year=

    Understanding and mitigating gender bias in llms via interpretable neuron editing , author=. arXiv preprint arXiv:2501.14457 , year=

    Pith/arXiv arXiv

  63. [63]

    Advances in neural information processing systems , volume=

    A unified approach to interpreting model predictions , author=. Advances in neural information processing systems , volume=

  64. [64]

    Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages=

    Transformer Feed-Forward Layers Are Key-Value Memories , author=. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages=

    2021

  65. [65]

    Social Norms in Cinema: A Cross-Cultural Analysis of Shame, Pride and Prejudice

    Rai, Sunny and Zaveri, Khushang and Havaldar, Shreya and Nema, Soumna and Ungar, Lyle and Guntuku, Sharath Chandra. Social Norms in Cinema: A Cross-Cultural Analysis of Shame, Pride and Prejudice. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume ...

    2025

  66. [66]

    arXiv preprint arXiv:2206.07682 , year=

    Emergent abilities of large language models , author=. arXiv preprint arXiv:2206.07682 , year=

    Pith/arXiv arXiv

  67. [67]

    2022 , institution=

    Aligned with whom? Direct and social goals for AI systems , author=. 2022 , institution=

    2022

  68. [68]

    AI & SOCIETY , pages=

    The problem of alignment , author=. AI & SOCIETY , pages=. 2024 , publisher=

    2024

  69. [69]

    Minds and machines , volume=

    Artificial intelligence, values, and alignment , author=. Minds and machines , volume=. 2020 , publisher=

    2020

  70. [70]

    Desired behaviors

    “Desired behaviors”: alignment and the emergence of a machine learning ethics , author=. AI & SOCIETY , pages=. 2025 , publisher=

    2025

  71. [71]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Order matters: Investigate the position bias in multi-constraint instruction following , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  72. [72]

    Judging the judges: A systematic study of position bias in llm-as-a-judge , author=. Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics , pages=

  73. [73]

    arXiv preprint arXiv:1803.05457 , year=

    Think you have solved question answering? try arc, the ai2 reasoning challenge , author=. arXiv preprint arXiv:1803.05457 , year=

    Pith/arXiv arXiv

  74. [74]

    Proceedings of the Computational S anskrit & Digital Humanities: Selected papers presented at the 18th World S anskrit Conference. 2023

    2023

  75. [75]

    Neural Approaches for Data Driven Dependency Parsing in S anskrit

    Krishna, Amrith and Gupta, Ashim and Garasangi, Deepak and Sandhan, Jeevnesh and Satuluri, Pavankumar and Goyal, Pawan. Neural Approaches for Data Driven Dependency Parsing in S anskrit. Proceedings of the Computational S anskrit & Digital Humanities: Selected papers presented at the 18th World S anskrit Conference. 2023

    2023

  76. [76]

    Evaluating Neural Word Embeddings for S anskrit

    Sandhan, Jivnesh and Paranjay, Om Adideva and Digumarthi, Komal and Behra, Laxmidhar and Goyal, Pawan. Evaluating Neural Word Embeddings for S anskrit. Proceedings of the Computational S anskrit & Digital Humanities: Selected papers presented at the 18th World S anskrit Conference. 2023

    2023

  77. [77]

    Validation and Normalization of DCS corpus and Development of the S anskrit Heritage Engine ' s Segmenter

    Sriram, Krishnan and Kulkarni, Amba and Huet, G \'e rard. Validation and Normalization of DCS corpus and Development of the S anskrit Heritage Engine ' s Segmenter. Proceedings of the Computational S anskrit & Digital Humanities: Selected papers presented at the 18th World S anskrit Conference. 2023

    2023

  78. [78]

    Pre-annotation Based Approach for Development of a S anskrit Named Entity Recognition Dataset

    Sujoy, Sarkar and Krishna, Amrith and Goyal, Pawan. Pre-annotation Based Approach for Development of a S anskrit Named Entity Recognition Dataset. Proceedings of the Computational S anskrit & Digital Humanities: Selected papers presented at the 18th World S anskrit Conference. 2023

    2023

  79. [79]

    Disambiguation of Instrumental, Dative and Ablative Case suffixes in S anskrit

    Maity, Malay and Panchal, Sanjeev and Kulkarni, Amba. Disambiguation of Instrumental, Dative and Ablative Case suffixes in S anskrit. Proceedings of the Computational S anskrit & Digital Humanities: Selected papers presented at the 18th World S anskrit Conference. 2023

    2023

  80. [80]

    Creation of a Digital Rig V edic Index (Anukramani) for Computational Linguistic Tasks

    Mahesh, A V S D S and Bhattacharya, Arnab. Creation of a Digital Rig V edic Index (Anukramani) for Computational Linguistic Tasks. Proceedings of the Computational S anskrit & Digital Humanities: Selected papers presented at the 18th World S anskrit Conference. 2023

    2023

Showing first 80 references.