REVIEW 5 major objections 5 minor 30 references
Probing Social Identity Bias in Chinese LLMs with Gendered Pronouns and Social Groups
T0 review · 5 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read Chinese LLMs consistently write more warmly about 'we' than 'they'—and live chats make it worse.
desk verdict A useful extension of social-identity framing tests to Chinese, but the unvalidated sentiment API and missing toxicity analysis keep it from landing the quantitative claims. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The measurement device is a minimal prompt contrast: '我们是/我们通常/我们相信' (we are/typically/believe) pitted against '他们是/她们是' (they are) as outgroup forms, exploiting Mandarin's written sex distinction in third-person plural pronouns—a distinction English lacks. The resulting sentence completions, roughly 297,600 generated texts, are labeled by a commercial Chinese sentiment classifier, and logistic regression converts the label distributions into odds ratios for ingroup solidarity and outgroup hostility. The odds ratio is the carrying identity: values above 1 are interpreted as systematic bias, and the gendered pronoun split is what lets the authors isolate a bias pattern not observable in Engli
What would settle it
Take a stratified random sample of the generated 'we' versus 'they' sentences and have human annotators rate valence without knowing the prompt; if human ratings show no systematic ingroup/outgroup difference—or if the gap tracks topic categories such as crime, politics, or misfortune rather than the pronoun—the odds-ratio result fails.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that ten Chinese LLMs—both base and instruction-tuned—consistently generate more positive sentiment when asked to continue sentences starting with '我们' (we) and more negative sentiment when starting with '他们' or '她们' (they), with odds ratios greater than 1 across all models. The asymmetry appears in simple sentence-completion prompts, across 240 social groups salient in Chinese society, and again—more strongly—in 4,079 real Chinese sentences from a large user–chatbot conversation corpus, where assistant responses show ingroup solidarity and outgroup hostility with odds ratios near 2. The gendered pronoun comparison adds a Mandarin-specific layer: fe
Load-bearing premise
The whole measurement rests on the assumption that a single commercial sentiment classifier labels Chinese completions correctly enough, and that the sentiment difference between 'we' and 'they' prompts reflects social identity bias rather than differences in topic, register, or word associations.
Editorial extensions
If this is right
- Chinese-facing LLMs in production are likely to reproduce ingroup-favoring, outgroup-denigrating language, making bias mitigation a concrete safety issue for chatbots and content moderation.
- Instruction tuning reduces but does not eliminate the asymmetry: pretrained models show stronger outgroup hostility, so alignment partially tempers rather than removes the effect.
- Feminine 'they' references are especially vulnerable in pretrained models, implying that gender bias and social identity bias can compound.
- Naturalistic interactions show the effect can be larger than synthetic prompts indicate, so lab benchmarks may understate real-deployment bias.
- If these findings hold, English-only bias audits miss a class of Mandarin-specific biases that require language-aware evaluation.
Reading between the lines
- Because the paper relies on a single sentiment classifier, an immediate test is human annotation of a stratified sample; if human raters do not see the same valence gap, the odds ratios may be an artifact of the classifier's own associations.
- The 'we' versus 'they' contrast may partly reflect distributional differences—'we' completions could drift to solidarity topics and 'they' to news-like negative topics—so topic-controlled prompts would sharpen the causal claim.
- Mandarin's pronoun distinction could serve as a general probe for gender bias in other Chinese NLP tasks, such as toxicity classification, translation, and dialogue safety, not just sentiment.
- The naturalistic finding is based largely on Western-model responses in Chinese; collecting equivalent dialogues from Chinese-native models would test whether the amplification is a property of the model family or of conversational context.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper investigates social identity framing in Chinese LLMs. In Study 1, the authors prompt ten Chinese LLMs with ingroup (“我们”, “we”) and outgroup (“他们”/“她们”, “they”) sentence starters across 240 Chinese social groups, then use Aliyun sentiment labels as the outcome in logistic regressions to estimate ingroup-solidarity and outgroup-hostility odds ratios. In Study 2, they extract keyword-marked sentences from the WildChat corpus to test whether the same asymmetries appear in naturalistic user–chatbot interaction. The reported findings are: odds ratios greater than 1 across models, with base models showing stronger outgroup hostility than instruction-tuned models; greater negativity toward feminine “她们” than masculine “他” outgroups in base models; and odds ratios around 2 in assistant responses from naturalistic dialogue, interpreted as amplification of bias in real interactions. The paper claims this is the first systematic examination of ingroup–outgroup framing in Chinese LLMs and that Mandarin's gendered third-person plural pronouns reveal bias patterns not directly observable in English.
Significance. If the measurements are valid, the contribution is meaningful: it is a large-scale language-aware evaluation spanning ten Chinese LLMs, it exploits the 他们/她们 distinction in a way that genuinely goes beyond English-only studies, the directional effect is consistent across all ten models, and the authors transparently report survival rates and regression specifications with controls. However, the strength of the claim is currently disproportionate to the evidence: the entire quantitative edifice rests on a single, unvalidated sentiment classifier, the abstract promises a toxicity analysis that is absent from the body, and the naturalistic-dialogue comparison conflates model family with setting. The design has the right shape, and the weaknesses are addressable, so the paper could become a solid contribution after substantial revision; as written, the central measurement remains unvalidated.
major comments (5)
- [Abstract; §3.3] The abstract promises a two-tiered framework that “assesses both sentiment and toxicity,” and states that “toxicity gaps remain more persistent” and that the feminine plural pronoun is “associated with higher toxicity.” No toxicity measure, classifier, or result appears anywhere in the body: Sections 3–5 define only PosSent/NegSent from Aliyun sentiment labels. This is a missing-support issue in an advertised central finding. The authors must either add the toxicity analysis or substantially revise the abstract and the contribution claims to describe a sentiment-only study.
- [§3.3; Limitations] Every reported odds ratio (Figures 1–3, 5, and Appendix E) is computed from a single Aliyun sentiment API call, with no human validation, no second classifier, and no per-condition error analysis. This is load-bearing because completions in the two conditions differ in topic and register (“我们相信…” vs “她们是…”), so the relevant failure mode is systematic label bias correlated with the prompt contrast, which would make every OR>1 an artifact. The Limitations paragraph concedes “method-specific biases” but offers no calibration. A revision must report inter-annotator/API agreement on a labeled sample (several hundred sentences per condition), per-condition error rates, and re-estimated odds ratios under validated or second-classifier labels.
- [§3.1, Table 6] Survival filtering is strongly differential: Qwen3-8B retains only 15.0% of “we” outputs versus 22.9% of “male they” outputs, with retention ranging from 15% to 90% across models. The paper also never checks whether retained sentences actually begin with the required starter; the refusal example in §3.1 (“Hello! It looks like you may have forgotten…”) exceeds the 10-character filter and would be retained even though it does not start with “我们是”. Non-compliant outputs analyzed under the wrong condition could bias the odds ratios. I ask for per-condition starter-compliance rates and a sensitivity analysis restricted to or reweighted for compliant generations.
- [§4.3, §5] The “amplification in interactive settings” claim is not supported by the design. Study 2 uses WildChat, i.e., GPT-3.5/GPT-4 responses, not the ten Chinese LLMs from Study 1, so the assistant-response odds ratios near 2 compare a different model family and a different task. User versus assistant sentences also differ in every other respect, and a sentence containing “我们” or “他们” is not necessarily an identity expression, since Chinese first-person plural has inclusive and discourse-anaphoric uses. The §5 conclusion that “these asymmetries are amplified in interactive settings” should be substantially softened or replaced with an analysis that controls for model and selection; the final-paragraph ecological-validity caveat understates this problem.
- [§4.2.1, Fig. 2] The headline “all models” claim is stated inconsistently. §4.2.1 says “Across both gendered forms, all models yield odds ratios greater than 1.0,” but the same sentence immediately acknowledges “the exception of Baichuan2-7B-Base,” whose ingroup-solidarity odds ratio for the female framing falls below 1. The abstract and §4.1 make the same unqualified “all models/systematic” claim. Since cross-model consistency is the central empirical assertion, the text should report the actual count of exceptions and the confidence interval for the Baichuan estimate.
minor comments (5)
- [§3.1 vs Appendix C] Sample-size inconsistency: §3.1 states 20 sentences per {template, group} (total 57,600 for 240 groups × 12 templates), while Appendix C states 50 continuations per pair (144,000). Please reconcile the numbers.
- [Eqs. (1)–(2)] The derivation of the “outgroup hostility” odds ratio is not defined. With InG coded as an ingroup indicator and outgroup as the reference, only the ingroup coefficient is directly identified; please state explicitly whether the outgroup-hostility OR is obtained by inverting the ingroup OR or by re-estimating with a reversed reference/outcome.
- [§4 headings] The section numbering is disordered: §4.1 “Study 1” is followed by §4.1.1, then §4.2 “Ingroup vs. Outgroup Prompts,” §4.2.1–§4.2.3, and §4.3 “Study 2.” Renumber for readability.
- [Table 7] Table 7 lists six model versions, but gpt-3.5-turbo-0125 contributes only 14 sentences (0.3%) and several rows are versions of the same underlying model family. Clarify whether the random intercept (1|model) in Eq. (2) treats these as distinct groups.
- [Data availability] No code or data availability statement is provided. Releasing the prompts, raw model outputs, and analysis scripts would materially support reproducibility, especially given the reliance on a proprietary API classifier.
Circularity Check
No circularity: the empirical odds-ratio results come from external sentiment labels and fresh LLM generations, not from a fitted parameter or a self-citation chain.
full rationale
This paper is an empirical measurement study, not a derivation from first principles. The central claim—that Chinese LLMs produce more positive sentiment under ingroup ('我们') prompts and more negative sentiment under outgroup ('他们'/'她们') prompts—is obtained by generating sentences, labeling them with Alibaba's Aliyun sentiment classifier (§3.3), and fitting logistic regressions (Eqs. 1–2). PosSent and NegSent are defined from the external classifier's output, and InG is the prompt condition; the odds ratios are descriptive estimates from the data, not fitted parameters renamed as predictions. The methodology is explicitly adopted from Hu et al. (2025), which is an external, non-overlapping citation, so no self-citation chain supports the result. The paper's self-citations (Liu et al. 2025a,b) appear only in related work and for the social-group taxonomy; they do not carry the empirical claim. The Limitations section candidly states that relying on a single classifier 'may introduce method-specific biases,' which is an acknowledged construct-validity/correctness risk, not a circularity: the outcome variable is not defined in terms of the prompt contrast. Differential survival rates (Table 6) are a sample-selection threat, not a definitional one. No equation or inference step reduces to its own input, so the circularity score is 0.
Assumptions & free parameters
free parameters (3)
- Sentence filtering thresholds =
min 10 Chinese chars; min 5 words; exclude 5-gram overlap
- Context selection thresholds =
quality ≥0.9; length 5-100 chars
- Generations per condition =
2,000 per starter (generic); 20 or 50 per group-template in social-group setting
assumptions (5)
- domain assumption Aliyun classifier's positive/negative labels are valid sentiment measures for Chinese LLM outputs.
- domain assumption The we/they (我们/他们/她们) starter contrast isolates social identity bias rather than topic or lexical differences.
- domain assumption Retained sentences after survival filtering are representative of model outputs.
- domain assumption The 240 hand-picked groups are salient Chinese social categories and translations are faithful.
- domain assumption WildChat assistant responses from GPT models can serve as evidence about Chinese LLM behavior in real interactions.
Cite this review
Pith. "Pith review of Probing Social Identity Bias in Chinese LLMs with Gendered Pronouns and Social Groups." pith.science (2026). https://pith.science/paper/JNNWZMSH
@misc{pith2026251006974,
author = {Pith},
title = {Pith review of: Probing Social Identity Bias in Chinese LLMs with Gendered Pronouns and Social Groups},
year = {2026},
howpublished = {\url{https://pith.science/paper/JNNWZMSH}},
note = {Machine review of arXiv:2510.06974}
}
read the original abstract
Large language models (LLMs) are increasingly deployed in user-facing applications, raising concerns that they may reflect and amplify social biases. We investigate social identity biases in Chinese LLMs using Mandarin-specific prompts across ten representative models. Our evaluation compares ingroup ("We") and outgroup ("They") framings across 240 social groups salient in the Chinese context, using a two-tiered measurement framework that assesses both sentiment and toxicity. The prompt design explicitly accounts for linguistic properties of Mandarin, including the distinction between the default gender-neutral plural pronoun and its explicitly feminine counterpart, enabling a controlled comparison of social identity framing effects. Across models, we observe systematic ingroup-outgroup asymmetries, although their expression differs across measurement dimensions. In particular, instruction tuning often reduces sentiment asymmetries, while toxicity gaps remain more persistent. Moreover, the feminine-marked plural pronoun is associated with higher toxicity than the default gender-neutral plural in several models. Our study introduces a language-aware evaluation framework for Chinese LLMs and shows that (i) social identity biases previously documented in English also manifest in Chinese and that (ii) Mandarin-specific linguistic structure can reveal bias patterns that are not directly observable in English-only settings.
Figures
Reference graph
Works this paper leans on
-
[1]
Jianghao Chen, Pu Jian, Tengxiao Xi, Dongyi Yi, Qianlong Du, Chenglin Ding, Guibo Zhu, Chengqing Zong, Jinqiao Wang, and Jiajun Zhang. 2023. https://arxiv.org/abs/2311.01149 Chinesewebtext: Large-scale high-quality chinese web text extracted with effective evaluation model . Preprint, arXiv:2311.01149
arXiv 2023
-
[2]
Zina Chkirbene, Ridha Hamila, Ala Gouissem, and Unal Devrim. 2024. https://doi.org/10.1109/HONET63146.2024.10822885 Large language models (llm) in industry: A survey of applications, challenges, and trends . In 2024 IEEE 21st International Conference on Smart Communities: Improving Quality of Life using AI, Robotics and IoT (HONET), pages 229--234
arXiv 2024
-
[3]
Marta Costa-juss \`a , Pierre Andrews, Eric Smith, Prangthip Hansanti, Christophe Ropers, Elahe Kalbassi, Cynthia Gao, Daniel Licht, and Carleigh Wood. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.874 Multilingual holistic bias: Extending descriptors and patterns to unveil demographic biases in languages at scale . In Proceedings of the 2023 Conferen...
-
[4]
Ameet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, and Karthik Narasimhan. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.88 Toxicity in chatgpt: Analyzing persona-assigned language models . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1236--1270, Singapore. Association for Computational Linguistics
-
[5]
Leon Fr \"o hling, Gianluca Demartini, and Dennis Assenmacher. 2025. https://aclanthology.org/2025.woah-1.43/ Personas with attitudes: Controlling LLM s for diverse data annotation . In Proceedings of the The 9th Workshop on Online Abuse and Harms (WOAH), pages 468--481, Vienna, Austria. Association for Computational Linguistics
2025
-
[6]
Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K. Ahmed. 2024. https://doi.org/10.1162/coli_a_00524 Bias and fairness in large language models: A survey . Computational Linguistics, 50(3):1097--1179
-
[7]
Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, and 1 others. 2022. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. arXiv preprint arXiv:2209.07858
arXiv 2022
-
[8]
Tiancheng Hu, Yara Kyrychenko, Steve Rathje, Nigel Collier, Sander van der Linden , and Jon Roozenbeek. 2025. https://doi.org/10.1038/s43588-024-00741-1 Generative language models exhibit social identity biases . Nature Computational Science, 5(1):65--75
Show all 30 references
-
[9]
C-T James Huang, Y-H Audrey Li, and Yafei Li. 2009. The syntax of chinese. (No Title)
2009
-
[10]
Leilei Jiang, Guixiang Zhu, Jianshan Sun, Jie Cao, and Jia Wu. 2025. Exploring the occupational biases and stereotypes of chinese large language models. Scientific Reports, 15(1):18777
2025
-
[11]
Hannah Rose Kirk, Bertie Vidgen, Paul R \"o ttger, and Scott A Hale. 2024. The benefits, risks and bounds of personalizing the alignment of large language models to individuals. Nature Machine Intelligence, 6(4):383--392
2024
-
[12]
Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019. https://doi.org/10.18653/v1/W19-3823 Measuring bias in contextualized word representations . In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, pages 166--172, Flor...
2019 doi
-
[13]
Michael Lepori. 2020. https://doi.org/10.18653/v1/2020.coling-main.151 Unequal representations: Analyzing intersectional biases in word embeddings using representational similarity analysis . In Proceedings of the 28th International Conference on Computational Linguistics, pag...
2020 doi
-
[14]
Charles N Li and Sandra A Thompson. 1989. Mandarin Chinese: A functional reference grammar. Univ of California Press
1989
-
[15]
Yanyang Li, Jianqiao Zhao, Duo Zheng, Zi-Yuan Hu, Zhi Chen, Xiaohui Su, Yongfeng Huang, Shijia Huang, Dahua Lin, Michael Lyu, and Liwei Wang. 2023. https://doi.org/10.18653/v1/2023.emnlp-demo.17 CLEVA : C hinese language models EVA luation platform . In Proceedings of the 2023...
2023 doi
-
[16]
Geng Liu, Carlo Alberto Bono, and Francesco Pierri. 2025 a . https://doi.org/10.7717/peerj-cs.2694 Comparing diversity, negativity, and stereotypes in chinese-language ai technologies: an investigation of baidu, ernie and qwen . PeerJ Computer Science, 11:e2694
2025 doi
-
[17]
Geng Liu, Li Feng, Carlo Alberto Bono, Songbo Yang, Mengxiao Zhu, and Francesco Pierri. 2025 b . Evaluating prompt-driven chinese large language models: The influence of persona assignment on stereotypes and safeguards. arXiv preprint arXiv:2506.04975
2025 arXiv
-
[18]
Bowman, and Rachel Rudinger
Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019. https://doi.org/10.18653/v1/N19-1063 On measuring social biases in sentence encoders . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational...
2019 doi
-
[19]
Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heintz, and Dan Roth. 2023. Recent advances in natural language processing via large pre-trained language models: A survey. ACM Computing Surveys, 56(2):1--40
2023
-
[20]
Moin Nadeem, Anna Bethke, and Siva Reddy. 2021. https://doi.org/10.18653/v1/2021.acl-long.416 S tereo S et: Measuring stereotypical bias in pretrained language models . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Inte...
2021 doi
-
[21]
Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R. Bowman. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.154 C row S -pairs: A challenge dataset for measuring social biases in masked language models . In Proceedings of the 2020 Conference on Empirical Methods in Na...
2020 doi
-
[22]
Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel Bowman. 2022. https://doi.org/10.18653/v1/2022.findings-acl.165 BBQ : A hand-built bias benchmark for question answering . In Findings of the Association for ...
2022 doi
-
[23]
Saddam Hossain Mukta, Kaniz Fatema, Nur Mohammad Fahad, Sadman Sakib, Most Marufatul Jannat Mim, Jubaer Ahmad, Mohammed Eunus Ali, and Sami Azam
Mohaimenul Azam Khan Raiaan, Md. Saddam Hossain Mukta, Kaniz Fatema, Nur Mohammad Fahad, Sadman Sakib, Most Marufatul Jannat Mim, Jubaer Ahmad, Mohammed Eunus Ali, and Sami Azam. 2024. https://doi.org/10.1109/ACCESS.2024.3365742 A review on large language models: Architectures...
2024
-
[24]
Mubashar Raza, Zarmina Jahangir, Muhammad Bilal Riaz, Muhammad Jasim Saeed, and Muhammad Awais Sattar. 2025. Industrial applications of large language models. Scientific Reports, 15(1):13755
2025
-
[25]
Henri Tajfel and John C Turner. 2004. The social identity theory of intergroup behavior. In Political psychology, pages 276--293. Psychology Press
2004
-
[26]
Yuemei Xu, Ling Hu, Jiayi Zhao, Zihan Qiu, Kexin Xu, Yuqi Ye, and Hanwen Gu. 2025. A survey on multilingual large language models: Corpora, alignment, and bias. Frontiers of Computer Science, 19(11):1911362
2025
- [27]
-
[28]
Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024. Wildchat: 1m chatgpt interaction logs in the wild. arXiv preprint arXiv:2405.01470
2024 arXiv
-
[29]
online" 'onlinestring :=
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
-
[30]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.