A Dual-Layered Evaluation of Geopolitical and Cultural Bias in LLMs

Hyuhng Joon Kim; Sean Kim

REVIEW 3 major objections 5 minor 28 references

A Dual-Layered Evaluation of Geopolitical and Cultural Bias in LLMs

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Language sways factual LLM answers; national bias leaks into disputes.

desk verdict Useful two-phase framework and dataset, but the central phase-dependent bias claim is confounded by the 'your country' prompt framing and needs major revision before the paper's headline result can be trusted. read the letter →

arxiv 2506.21881 v1 pith:AJZ5K657 submitted 2025-06-27 cs.CL

Sean Kim , Hyuhng Joon Kim This is my paper

classification cs.CL

keywords LLMbiasmultilingualevaluationgeopoliticalculturalmodelinferenceEastAsianterritorialdisputesfactualQA

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes that bias in multilingual LLMs has two separable sources: model bias, which comes from the model's training data, and inference bias, which is induced by the language of the query. The authors test this distinction with a two-phase evaluation on five models across Korean, Chinese, Japanese, and English. In Phase 1, factual questions with single verifiable answers, responses mainly track the query language. In Phase 2, geopolitically disputed questions about East Asian territorial and historical conflicts, the model's training context starts to dominate, especially for the Korean and Chinese models. The paper argues that bias is not static but emerges from an interaction between task type, query language, and training context.

What carries the argument

The central machinery is the two-phase evaluation design built on the model-bias/inference-bias distinction. Model bias is defined as alignment with the model's primary training language; inference bias as alignment with the query language. Two metrics, the Model Bias Rate (MBR) and Inference Bias Rate (IBR), quantify how often responses match each expected national stance, with responses matching both or neither treated as neutral. The Phase 2 dataset organizes each dispute into four question types—OPEN, PERSONA, TF, and CHOICE—to vary how much the prompt constrains the answer. This setup lets the paper attribute answer variation to query language versus training context rather than to model size or prompting style.

What would settle it

Run the Phase 2 disputes on Bllossom 8B with Japanese queries; the paper predicts Korea-aligned outputs except for a few TF cases. A consistent pattern of Japan-aligned responses across question types would contradict the model-bias claim for the Korean model.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the dominant source of bias switches with the nature of the question: for factual QA, inference bias (alignment with the query language) prevails across all evaluated models, whereas for disputable QA, model bias (alignment with the primary training language) becomes visible, particularly for the Korean Bllossom model and, to a lesser extent, the Chinese Qwen model. This is measured by two rates, MBR and IBR, computed from human and model-based evaluations of whether each response matches the model's own national stance or the query's national stance. The paper reads this as evidence that factual retrieval is language-adaptive, while politically sensitive reasoning activates culturally embedded training patterns. It also reports that open-ended questions produce evasive answers, while constrained formats like true/false and forced choice expose biases more clearly.

Load-bearing premise

The study assumes that each open-source model's primary training language and developer's country make it a faithful representative of that nation's viewpoint (e.g., Llama 3 stands for the US position); if that mapping is wrong, the measured split between model bias and inference bias loses its meaning.

Editorial extensions

If this is right

If a deployment asks an LLM factual questions, the language of the user is the dominant driver of the answer, and switching languages can change the answer even when the facts are fixed.
On disputed topics, models from countries involved in the dispute will tend to echo their home country's narrative even when prompted in another language.
Neutrality cannot be assumed from a model's refusal behavior: even models that aim for neutrality (US Llama, GPT-4) show topic-dependent leans, such as favoring Japan in the Senkaku/Diaoyu dispute.
Structured prompts such as true/false or forced choice are more effective than open-ended questions at detecting latent national alignment.
Evaluation of LLM bias should separate factual consistency from interpretive alignment, because the two phases exhibit opposite dominant biases.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension is to use models with precisely documented pretraining corpora or balanced multilingual data to confirm that the observed 'model bias' actually moves with the training-country data distribution rather than with model architecture or family.
The result suggests that LLM-as-judge evaluations are risky in geopolitically sensitive settings, since the judging model may carry its own training-country bias; the paper avoided model-based evaluation in Phase 2 for this reason.
The finding that constrained prompts surface bias could be extended to other formats such as chain-of-thought or counterfactual role-play to probe whether the revealed inclination is a stable model property or an artifact of question framing.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper gives the community something real: a clearly defined two-phase framework separating model bias from inference bias, a manually curated multilingual dataset covering four East Asian disputes, and four question types that probe bias in different ways. The dataset and the definitions are the genuine contributions, and the authors are honest about their limitations. The writing is clear and the citations are on point.

The problem is the central claim. The abstract says Phase 1 shows query-language alignment while Phase 2 shows a shift toward model bias. But every Phase 1 question is explicitly prefixed with 'your country's' (Section 4.1), while Phase 2 questions use neutral wording. Asking What is your country's name? forces the model to pick a country and makes the query language the obvious cue. Asking Which country does Dokdo/Takeshima belong to? gives no such cue, so the model must fall back on its own internal knowledge. The phase-dependent shift could therefore be caused entirely by the presence or absence of the anchor, independent of whether the topic is factual or disputable. The paper interprets the shift as a property of topic, but it never controls for prompt framing. This is a load-bearing confound, not a minor weakness.

The other concerns are real but secondary. Phase 2 rests on a small number of items per model, with many cells marked invalid, and no statistical tests or confidence intervals appear anywhere. The model-to-country mapping (Llama 3 as the US model, Qwen as the Chinese model) is acknowledged by the authors to be imprecise, and the reader's concern about that mapping is legitimate, though it is less decisive than the prompt-framing issue.

Who benefits from this paper? Researchers working on multilingual bias evaluation, especially those who want a compact, reproducible dataset for East Asian disputes. The dataset itself is useful and the framework is a reasonable starting point. The paper deserves a serious referee, but it needs major revision: both phases should use matched prompt framing, the authors should add significance tests or at least confidence intervals, and they should report inter-annotator agreement for the human evaluation. As written, the headline result should be treated as a hypothesis, not a finding.

Referee Report

3 major / 5 minor

Summary. The paper introduces a two-phase evaluation framework to separate model bias (alignment with a model's primary training language) from inference bias (alignment with the query language) in LLMs. Phase 1 uses 70 factual questions, all prefixed with the phrase "your country's," translated into Korean, Chinese, Japanese, and English; Phase 2 uses 16 disputable QA items about four East Asian territorial and historical disputes, each in four question formats (OPEN, PERSONA, TF, CHOICE). Five models are evaluated: Bllossom, Qwen1.5, Rakuten, Llama 3, and GPT-4. The authors report that Phase 1 is dominated by inference bias, while Phase 2 shows stronger model bias, especially for the Korean and Chinese models, and they interpret this as a phase-dependent shift from language-driven adaptation to training-driven cultural alignment.

Significance. The paper addresses an important and understudied problem: how LLMs behave in multilingual, geopolitically sensitive settings where factual correctness and national narratives interact. Its strengths include a manually curated multilingual dataset focused on East Asian disputes, a clear conceptual distinction between model bias and inference bias, and the release of data and code, which supports reproducibility. The inclusion of both open-source and proprietary models with different language emphases is also a useful design. However, the central empirical claim is currently under-supported: the phase comparison is confounded by prompt framing, and Phase 2 relies on very small samples without statistical quantification. If the confound is resolved and the results hold, the framework would be a valuable contribution to culturally aware LLM evaluation.

major comments (3)

[Section 4.1 vs. Section 5.1 / Table 6] The phase comparison conflates question content with prompt wording. Every Phase 1 prompt is explicitly prefixed with "your country's" (Section 4.1), while Phase 2 questions are neutrally framed (e.g., "What do you think about the 'Northeast Project dispute' between Republic of Korea and China?"). Thus, the observed shift from inference bias in Phase 1 to model bias in Phase 2 could be entirely due to the presence or absence of the country anchor, independent of whether questions are factual or disputable. The interpretation in Section 6(1) that "factual questions tend to elicit language-adapted responses... politically sensitive topics activate culturally embedded patterns" is not supported unless the framing is controlled. Please add a control condition: either run Phase 1 questions without the "your country" prefix, or add a comparable anchor to Phase 2 questions, or restrict the analysis to a matched subset where the wording is identical in both phases.
[Section 5.3 / Appendix E] The Phase 2 analysis rests on only 16 core items per model (4 disputes × 4 question types), and many responses are labeled "invalid." For example, Table 7 shows that Bllossom (KR) yields invalid responses for nearly all OPEN questions; the claimed model-bias pattern is based on a handful of valid cells in the PERSONA, TF, and CHOICE formats. No confidence intervals, significance tests, or effect sizes are provided for any Phase 2 percentage, including the 83.3% reported in Figure 4, which is calculated from 12 responses. This makes it impossible for the reader to assess whether the differences between models and between bias types are robust. Please report per-cell counts, exact binomial confidence intervals, and statistical comparisons (e.g., a test of whether IBR exceeds MBR across models).
[Section 3.2 / Limitations] The model-to-country mapping is a load-bearing assumption for the entire model-bias analysis. Treating Llama 3 as the "US model" and Qwen as the "CN model" assumes that these general-purpose models encode national perspectives based on developer affiliation and primary training language. The paper acknowledges this imprecision in the Limitations, but the acknowledgement does not address the resulting threat to construct validity: if this mapping is incorrect, the distinction between model bias and inference bias becomes unmeasurable. At minimum, the authors should provide a manipulation check, such as using the Phase 1 factual answers to verify that each model aligns with its assigned country more often than with others, or restrict the model-bias claims to models that explicitly target a national market (e.g., Bllossom for Korea, Rakuten for Japan).

minor comments (5)

[Throughout] There is an inconsistent spelling of the Korean model name: the paper uses "Bllossom" in most places, but the reference to Choi et al. (2024) and the GitHub URL use "Blossom" (the URL reads "LLM-Bias-Evaluation" but the model name is potentially "Bllossom"). Please verify the official spelling and use it consistently.
[Tables 3–5] The column headers "M" and "I" are not expanded in the captions; although the text defines MBR and IBR, the captions should explicitly state "M = model bias rate (MBR), I = inference bias rate (IBR)" to make the tables self-contained.
[Section 4.2] The statement "GPT-4o was chosen over GPT-4 to avoid bias, as GPT-4 was among the evaluated models" is confusing because Table 5 evaluates GPT-4, not GPT-4o. Rephrase to clarify that GPT-4o was used only as an evaluator and never as a subject in Phase 1.
[Section 4.3 / Appendix D] The phrase "which states that it lacks a nationality" appears to be a verbatim quote from a model response, but the exact response is not reproduced. Please include the actual model-generated sentence in the text or an appendix so that the reader can verify the neutrality behavior described.
[Tables 12–13] The captions state that questions "unanswered by more than three models" are excluded, but the table reports percentages that do not clearly indicate how many of the 70 questions remain. Please report the number of remaining questions for each exclusion criterion so that the reader can compare the tables with Table 3 and Table 4.

Circularity Check

1 steps flagged · score 7.0 of 10

Phase 1 'inference bias' is built into the dataset: the answer key is the query-language country, so the central phase-shift claim reduces to prompt design.

self definitional [Section 4.1, Table 1, Section 4.3, Equation 2]
"All prompts are explicitly prefixed with 'your country's' to anchor responses within each model's national context. ... Each question is paired with culturally appropriate, country-specific answers from KR, CN, JP, and US. ... Model-based evaluation revealed that IBR is consistently higher across all models."

In Phase 1, every factual question includes 'your country's' (e.g., 'What is your country's name?'), and the reference answer for a Korean query is the Korean answer, the reference answer for a Chinese query is the Chinese answer, etc. The IBR metric (Eq. 2) scores a response as query-language aligned exactly when it matches this query-language answer key. Therefore, any model that correctly follows the prompt and provides the factual answer will be classified as exhibiting 'inference bias' by construction. The paper's central Phase 1 finding — 'Phase 1 exhibits query language induced alignment' — is not an empirical discovery but a tautology of dataset design: the answers are defined per query language.

full rationale

The paper defines inference bias as alignment with the query language and constructs Phase 1 so that the expected answer for each question is the country associated with the query language (Table 1). Because the prompt explicitly asks about 'your country' and the answer key is keyed to the query language, IBR measures whether the model returns the supplied answer, not an independent 'bias.' The dominant inference-bias result in Phase 1 and the claimed contrast with Phase 2 therefore reduce to the dataset's construction. Phase 2 results (model bias in KR/CN models) are not forced by the same mechanism, and the paper's manual annotation and model-to-country mapping are acknowledged limitations rather than circular moves. Nevertheless, the abstract's strongest claim — the phase-dependent shift from query-language alignment to training-language alignment — is substantially an artifact of prompt design, warranting a score of 7 on the circularity scale.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new entities and fits no parameters; its claims rest on assumptions about model-country correspondence and the validity of the manually labeled answer keys.

assumptions (3)

domain assumption Each selected LLM represents the national viewpoint of its developer's country (Bllossom=KR, Qwen=CN, Rakuten=JP, Llama=US).
Section 3.2: 'We infer each model's cultural orientation based on its developers' national affiliations and intended user base.' This assumption underlies the definition of model bias.
domain assumption The expected answers in Phase 1 (from Wikipedia-style official sources) and the expected stances in Phase 2 (the authors' categorization of dominant national perspectives) are valid ground truth.
Sections 4.1 and 5.1 state answers are drawn from official/Wikipedia content and from 'the stance most commonly represented in public, political, or historical discourse.' Label validity is not independently verified.
domain assumption Translated questions are semantically and culturally equivalent across the four languages, so any answer difference is attributable to language, not translation artifacts.
Section 1: 'All questions are designed to maintain semantic consistency across languages.' Manual verification of translations by the authors is the only support.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Dual-Layered Evaluation of Geopolitical and Cultural Bias in LLMs." pith.science (2026). https://pith.science/paper/AJZ5K657

@misc{pith2026250621881,
  author       = {Pith},
  title        = {Pith review of: A Dual-Layered Evaluation of Geopolitical and Cultural Bias in LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AJZ5K657}},
  note         = {Machine review of arXiv:2506.21881}
}

read the original abstract

As large language models (LLMs) are increasingly deployed across diverse linguistic and cultural contexts, understanding their behavior in both factual and disputable scenarios is essential, especially when their outputs may shape public opinion or reinforce dominant narratives. In this paper, we define two types of bias in LLMs: model bias (bias stemming from model training) and inference bias (bias induced by the language of the query), through a two-phase evaluation. Phase 1 evaluates LLMs on factual questions where a single verifiable answer exists, assessing whether models maintain consistency across different query languages. Phase 2 expands the scope by probing geopolitically sensitive disputes, where responses may reflect culturally embedded or ideologically aligned perspectives. We construct a manually curated dataset spanning both factual and disputable QA, across four languages and question types. The results show that Phase 1 exhibits query language induced alignment, while Phase 2 reflects an interplay between the model's training context and query language. This paper offers a structured framework for evaluating LLM behavior across neutral and sensitive topics, providing insights for future LLM deployment and culturally aware evaluation practices in multilingual contexts.

Figures

Figures reproduced from arXiv: 2506.21881 by the authors.

**Figure 2.** Conceptual illustration of model bias and in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Distribution (%) of factual question topics in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Example distribution(%) of the KR model re [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

28 extracted references · 7 canonical work pages

[1]

Abubakar Abid, Maheen Farooqi, and James Zou. 2021. Persistent anti-muslim bias in large language models. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 298--306

2021
[2]

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

arXiv 2023
[3]

culture

Muhammad Farid Adilazuarda, Sagnik Mukherjee, Pradhyumna Lavania, Siddhant Singh, Alham Fikri Aji, Jacki O'Neill, Ashutosh Modi, and Monojit Choudhury. 2024. Towards measuring and modeling" culture" in llms: A survey. arXiv preprint arXiv:2403.15412

arXiv 2024
[4]

Alham Fikri Aji, Jessica Zosa Forde, Alyssa Marie Loo, Lintang Sutawika, Skyler Wang, Genta Indra Winata, Zheng Xin Yong, Ruochen Zhang, A Seza Do g ru \"o z, Yin Lin Tan, et al. 2023. Current status of nlp in south east asia with insights from multilingualism and language diversity. In Proceedings of the 13th International Joint Conference on Natural Lan...

work page 2023
[5]

Arnav Arora, Lucie-Aim \'e e Kaffee, and Isabelle Augenstein. 2022. Probing pre-trained language models for cross-cultural differences in values. arXiv preprint arXiv:2203.13722

arXiv 2022
[6]

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng X...

arXiv 2023
[7]

Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610--623

2021
[8]

Yong Cao, Li Zhou, Seolhwa Lee, Laura Cabello, Min Chen, and Daniel Hershcovich. 2023. Assessing cross-cultural alignment between chatgpt and human societies: An empirical study. arXiv preprint arXiv:2303.17466

arXiv 2023

Show all 28 references

[9]

ChangSu Choi, Yongbin Jeong, Seoyoon Park, InHo Won, HyeonSeok Lim, SangMin Kim, Yejee Kang, Chanhyuk Yoon, Jaewan Park, Yiseul Lee, HyeJin Lee, Younggyun Hahm, Hansaem Kim, and KyungTae Lim. 2024. Optimizing language augmentation for multilingual large language models: A case...

2024 arXiv
[10]

Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. 2023. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair nlp models. arXiv preprint arXiv:2305.08283

2023 arXiv
[11]

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

2024 arXiv
[12]

Jing Huang and Diyi Yang. 2023. Culturally aware natural language inference. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7591--7609

2023
[13]

Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

2024 arXiv
[14]

Grgur Kova c , Masataka Sawayama, R \'e my Portelas, C \'e dric Colas, Peter Ford Dominey, and Pierre-Yves Oudeyer. 2023. Large language models as superpositions of cultural perspectives. arXiv preprint arXiv:2307.07870

2023 arXiv
[15]

Bryan Li, Samar Haider, and Chris Callison-Burch. 2024 a . This land is your, my land: Evaluating geopolitical bias in language models through territorial disputes. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguis...

2024
[16]

Cheng Li, Mengzhuo Chen, Jindong Wang, Sunayana Sitaram, and Xing Xie. 2024 b . Culturellm: Incorporating cultural differences into large language models. Advances in Neural Information Processing Systems, 37:84799--84838

2024
[17]

Chen Cecilia Liu, Iryna Gurevych, and Anna Korhonen. 2024. Culturally aware and adapted nlp: A taxonomy and a survey of the state of the art. arXiv preprint arXiv:2406.03930

2024 arXiv
[18]

Tarek Naous, Michael J Ryan, Alan Ritter, and Wei Xu. 2023. Having beer after prayer? measuring cultural bias in large language models. arXiv preprint arXiv:2305.14456

2023 arXiv
[19]

Jirui Qi, Raquel Fern \'a ndez, and Arianna Bisazza. 2023. Cross-lingual consistency of factual knowledge in multilingual language models. arXiv preprint arXiv:2310.10378

2023 arXiv
[20]

Rakuten Group, Inc. , Aaron Levine, Connie Huang, Chenguang Wang, Eduardo Batista, Ewa Szymanska, Hongyi Ding, Hou Wei Chou, Jean-François Pessiot, Johanes Effendi, Justin Chiu, Kai Torben Ohlhus, Karan Chopra, Keiji Shinzato, Koji Murakami, Lee Xiong, Lei Chen, Maki Kubota, M...

2024 arXiv
[21]

Aida Ramezani and Yang Xu. 2023. Knowledge of cultural moral norms in large language models. arXiv preprint arXiv:2306.01857

2023 arXiv
[22]

Lukas Struppek, Dom Hintersdorf, Felix Friedrich, Patrick Schramowski, Kristian Kersting, et al. 2023. Exploiting cultural biases via homoglyphs in text-to-image synthesis. Journal of Artificial Intelligence Research, 78:1017--1068

2023
[23]

Yan Tao, Olga Viberg, Ryan S Baker, and Ren \'e F Kizilcec. 2024. Cultural bias and cultural alignment of large language models. PNAS nexus, 3(9):pgae346

2024
[24]

Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805

2023 arXiv
[25]

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

2023 arXiv
[26]

Seunguk Yu, Juhwan Choi, and Youngbin Kim. 2025. Delving into multilingual ethical bias: The msqad with statistical hypothesis tests for large language models. arXiv preprint arXiv:2505.19121

2025 arXiv
[27]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
[28]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Abubakar Abid, Maheen Farooqi, and James Zou. 2021. Persistent anti-muslim bias in large language models. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 298--306

2021

[2] [2]

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

arXiv 2023

[3] [3]

culture

Muhammad Farid Adilazuarda, Sagnik Mukherjee, Pradhyumna Lavania, Siddhant Singh, Alham Fikri Aji, Jacki O'Neill, Ashutosh Modi, and Monojit Choudhury. 2024. Towards measuring and modeling" culture" in llms: A survey. arXiv preprint arXiv:2403.15412

arXiv 2024

[4] [4]

Alham Fikri Aji, Jessica Zosa Forde, Alyssa Marie Loo, Lintang Sutawika, Skyler Wang, Genta Indra Winata, Zheng Xin Yong, Ruochen Zhang, A Seza Do g ru \"o z, Yin Lin Tan, et al. 2023. Current status of nlp in south east asia with insights from multilingualism and language diversity. In Proceedings of the 13th International Joint Conference on Natural Lan...

work page 2023

[5] [5]

Arnav Arora, Lucie-Aim \'e e Kaffee, and Isabelle Augenstein. 2022. Probing pre-trained language models for cross-cultural differences in values. arXiv preprint arXiv:2203.13722

arXiv 2022

[6] [6]

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng X...

arXiv 2023

[7] [7]

Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610--623

2021

[8] [8]

Yong Cao, Li Zhou, Seolhwa Lee, Laura Cabello, Min Chen, and Daniel Hershcovich. 2023. Assessing cross-cultural alignment between chatgpt and human societies: An empirical study. arXiv preprint arXiv:2303.17466

arXiv 2023