Pith. sign in

REVIEW 2 major objections 5 minor 123 references

People Are Not Just Their Countries. Disentangling Social Determinants of LLM Value Alignment Across Europe

T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read Ten commercial LLMs align more closely with the stated values of Europe's richer, more educated, and more secular respondents, and a respondent's country of residence explains as much of the mismatch as all socio-demographics combined.

desk verdict Careful cross-national study of LLM value alignment by socio-demographics and country, but English-only prompting leaves a real confound that could explain part of the country and SES gradients. read the letter →

arxiv 2608.07367 v1 pith:THVIARTW submitted 2026-08-07 cs.AI

classification cs.AI
keywords LLMvaluealignmentEuropeanSocialSurveysocio-demographicgroupscountryofresidencepluralisticWEIRDpopulationsinversepropensityweightingprompting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that evaluating how well LLMs reflect human values by country alone is misleading: the same ten commercial models agree much more with the stated values of some socio-demographic groups, such as people with higher education, higher income, professional occupations, and no religion, than with others, and these gaps are comparable in size to country-level gaps. Using Wave 11 of the European Social Survey, the authors prompt ten models with value-laden survey questions and compute, for each of 50,116 respondents, an alignment score based on the average normalized distance between the respondent's answer and the model's majority answer. The authors claim that country of residence and socio-demographic background are complementary rather than interchangeable: reweighting countries to a common socio-demographic mix barely reduces between-country alignment differences, and a country-only model explains as much variance as the full set of 15 demographic variables. If the claim is right, LLM-based advice, moderation, and synthetic public opinion tools in Europe are systematically skewed toward the values of privileged and secular groups, making group-level representativeness a necessary target for alignment work.

What carries the argument

The load-bearing object is the per-person, per-model alignment score $A_{p,m,q} = 1 - \frac{|a_{p,q} - a_{m,q}|}{|R_q|}$, which turns a Likert-scale answer into a normalized similarity between a survey respondent and a model's majority vote, averaged over questions to give $A_{p,m,\mathcal{Q}}$. This score is what lets the authors compare groups: for each socio-demographic group and country they compute the mean deviation from the population average across all models, and they verify the robustness of these deviations with bootstraps that include model-answer variation. Two further pieces of machinery carry the disentangling: inverse propensity weighting, which reweights each country's respondents so countries share a common socio-demographic distribution, and a variance-decomposition comparison of OLS regressions with gradient-boosted trees over country-only, socio-demographics-only, and combined covariate sets, which separates additive from interaction-driven contributions.

What would settle it

Re-run the ESS alignment protocol with the same ten models prompted in each country's main survey languages and recompute the group and country deviations; if the education-income-religion gradient and the Sweden-versus-Bulgaria gap shrink to near zero, the paper's central finding is a prompting-language artifact rather than a property of the models' value alignment.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that value alignment between LLMs and humans is structured by social stratification within countries, not only by national borders. Across ten commercially deployed models, the average deviation of a group's alignment from the population mean rises with educational level, income decile, subjective financial comfort, and occupational class, and it is sharply lower for Muslims, Eastern Orthodox respondents, the unemployed, and people with no interest in politics. At the individual level, a respondent's country alone explains between roughly 7% and 28% of the variance in alignment scores depending on the model, which is on par with or above the variance explained by the full set of socio-demographic variables, while the combination of the two explains up to 42.8%, pointing to complementarity. The authors also show that reweighting countries so that they share the same socio-demographic composition does not remove country differences, and that the relative weight of country versus socio-demographics depends on whether values are measured with broader opinion questions or with Schwartz's abstract Portrait Value Questionnaire items.

Load-bearing premise

The core comparison assumes that prompting every model in English, while humans answered in their own languages, does not systematically shift model answers in ways that create the observed country, education, income, and religion gaps.

Editorial extensions

If this is right

  • Standard alignment evaluations that aggregate by country hide a systematic within-country skew: higher socio-economic status groups are better represented by commercial LLMs.
  • Country of residence cannot be replaced by socio-demographics, or vice versa; both covariate sets are needed to predict individual alignment.
  • Between-country differences in alignment are not an artifact of different demographic compositions, since inverse propensity weighting leaves the spread of country means largely unchanged.
  • The question set defines the result: on broad value-laden opinion questions countries explain a large share of variance, whereas on Schwartz Portrait Value Questionnaire items the country share shrinks for most models, so alignment claims should state which notion of values they use.
  • If LLMs continue to be used as general-purpose information and advice tools, the values of already advantaged groups will be reinforced.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension left implicit by the English-only design is prompting the same models in each respondent's native language: if the education, income, and religion gradients shrink or vanish, the central finding is partly a language artifact.
  • The same alignment-score machinery could be applied to regional surveys outside Europe, such as the Afrobarometer or Latinobarometro; a repeat of the high-socio-economic-status alignment gradient there would show the WEIRD-alignment pattern generalizes beyond Europe.
  • Because the country-only and combined models are largely additive, intersectional interaction effects appear to add little beyond main effects, which suggests simple, transparent audit models could be sufficient for monitoring representativeness in production systems.
  • Refusal rates differ sharply by model and by sensitive topic, so alignment scores on contested questions are partly measurements of what models are willing to state; a refusal-aware score would treat declined answers differently from explicit disagreement.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper studies value alignment between 10 commercial LLMs and ESS Wave 11 respondents (50,116 people across 29 countries and Israel), using 47 value-laden survey questions. It constructs a per-person, per-model alignment score as a normalized Likert-distance to the model's majority-vote answer, then analyzes cross-model deviations by country and 15 socio-demographic variables. The main empirical contributions are: (i) group-level alignment gaps, especially by education, income, occupation, and religion; (ii) an inverse-propensity-weighting analysis suggesting that between-country alignment differences are not explained by socio-demographic composition; and (iii) a variance decomposition showing that country of residence alone explains roughly as much individual-level alignment variance as the full set of socio-demographics, with the two being complementary. The paper includes extensive appendices with bootstrap confidence intervals, a joint bootstrap over model answer variability, IPW balance checks, extremeness analyses, and train/test R^2 tables.

Significance. If the central findings are taken at face value, this is a valuable and timely contribution: it extends the 'WEIRD LLM' finding from country-level aggregates to actual socio-demographic groups within Europe, uses a fresh ESS wave rather than the potentially contaminated WVS, and provides a transparent, reproducible pipeline (code is public; bootstrap, IPW, and cross-validation details are documented). The measurement object is a descriptive alignment score with no fitted parameters, and the paper carefully separates sampling uncertainty from model-answer variability. The main weakness is that the entire comparison is made with English-only prompts, and the authors' response to this threat is an acknowledged limitation rather than a demonstrated invariance. Because the paper's strongest claims concern which European people and groups the LLMs 'represent', the language confound is load-bearing and needs either additional robustness evidence or a substantially softened interpretation.

major comments (2)
  1. [3.1, 5] The central comparison pairs English-prompted LLM answers ("Models are only prompted in English", §3.1) with human answers collected in each country's native language. If English prompts shift LLM stances toward Anglophone, higher-SES, secular value profiles, then the observed gradients in Figures 1, 2, and 4—higher alignment for more educated, richer, less religious, and Nordic/Central European respondents—could arise even for a model that is equally 'aligned' across groups in its native-language behavior. The invariance evidence cited in the Discussion (Davidov et al. 2008; Alemán and Woods 2016) concerns the human ESS/WVS instruments across languages; it does not establish that an English-prompted LLM's answer distribution is invariant to the respondent's linguistic or cultural context. This is a stated limitation, but it is not resolved. Concretely, the authors should either (a) prompt models in each country's language (or a representative subset of languages) and show that the cross-group and cross-country deviation patterns persist, or (b) measure the English-prompt versus native-prompt discrepancy for a subset of countries and bound its effect on the reported deviations. Without such evidence, the abstract's claim that LLMs are 'unequally aligned to the values of different socio-demographic groups' conflates value alignment with language-mediated response similarity.
  2. [3.1, 4.1, A.1] The restricted question set Q excludes 8 questions that at least one model refused to answer, and the excluded items are disproportionately 'controversial topics or specific institutions' (§3.1), with four coinciding with the highest human non-response rates. Because group differences on LGB tolerance, gender equality, and left-right placement are likely to be large, restricting to Q removes the very items on which education/income/religion alignment gaps might be strongest. Model refusal is itself a value-relevant behavior and discarding it may bias the measured alignment gradients toward convergence. The authors should report the main group-deviation and R^2 analyses on the maximal set of questions for models with low refusal rates (e.g., the DeepSeek/Mistral models that answer nearly all questions), or otherwise show that conclusions are stable to alternative refusal treatments. The joint bootstrap in Appendix A.5 only varies the question set within Q; it does not address the systematic exclusion of refused items.
minor comments (5)
  1. [Table 1] The caption uses O, Q, and Q̃ without defining them in the main text; the distinction between the full 53-item set and the restricted 45-item set should be stated where Table 1 is introduced.
  2. [§3.1] The text says '47 questions' while the appendix lists 53 items, with a parenthetical that 9 items belong to 3 conceptual questions. This is confusing on first reading; please state the 53/47 convention earlier and consistently.
  3. [Appendix A.1] The legend 'Green: Questions all LLMs answered (Q)set; underlined :Reduced PVQ setset' contains a typo and unclear formatting; the table would benefit from a cleaner visual encoding of Q and the PVQ subset.
  4. [§4.1] The paragraph on generations mentions a U-shape but the supporting text says it results from diverging model-specific patterns; this should be flagged directly in the figure discussion to avoid over-reading the aggregate pattern.
  5. [Appendix A.7] The IPW results are presented with clipping at 0.01 in the main text, while the drop-based robustness checks show non-trivial differences for individual countries (e.g., Israel). Please add a sentence in §4.2 explicitly interpreting this country-level instability for the claim that 'country differences remain after reweighting'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the alignment metric is a parameter-free distance against external ESS data, and the variance decompositions are out-of-sample empirical measurements.

full rationale

The paper's derivation chain is self-contained against external data. The alignment score A_{p,m,q} = 1 - |a_{p,q}-a_{m,q}|/|R_q| is a parameter-free normalized distance, then averaged over questions; no parameter is fitted to the ESS responses. The group deviations d_{G,M,Q} are arithmetic summaries of these scores. The IPW reweighting, propensity score estimation, and OLS/XGBoost variance decomposition are all empirical measurements of those fixed scores against external ESS respondent attributes; the R2 values are out-of-sample via 10-fold cross-validation, so they are not fitted values renamed as predictions. The midpoint-model analysis in Appendix A.6 is an explicit artifact check rather than a fitted input. The measurement-invariance citations (Davidov et al. 2008; Alemán and Woods 2016) are external independent sources, not self-citations, and are used only to support comparability of the human survey, not to define any LLM quantity. The English-only prompting limitation is a genuine external-validity threat but does not make any central result equivalent to its inputs by construction. No circular step can be exhibited.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claims rest on measurement choices and survey assumptions rather than fitted constants. The main dependencies are Likert equidistance, English-only prompting with cross-national comparability, majority-vote aggregation, survey weighting, and the hand-chosen IPW clipping threshold. No new theoretical entities are introduced.

free parameters (2)
  • Propensity score clipping threshold = 0.01
    Hand-chosen threshold in IPW reweighting (Appendix A.7); robustness checks with dropping at 0.01 and 0.05 show main conclusions unchanged.
  • Question subset Q (restricted set) = 45 of 53 questions (8 excluded)
    Post-hoc exclusion of questions where at least one LLM refused all 20 calls, to enable cross-model comparison; affects alignment score composition (Section 3.1, Appendix A.1).
assumptions (4)
  • domain assumption Equal spacing of Likert scale categories in alignment score
    Alignment score A_{p,m,q} assumes equal distances across scale categories (Section 3.2); acknowledged by authors as an implicit assumption.
  • domain assumption Cross-language and scalar measurement invariance of value questions
    English-prompted LLM answers are compared to native-language survey responses across 30 countries; paper relies on Davidov et al. 2008 for measurement invariance (Section 3.1, Limitations).
  • domain assumption Majority vote over 20 calls summarizes the model's stance
    Analysis uses majority vote for each question-model pair, with robustness via joint bootstrap (Section 3.1, Appendix A.5).
  • domain assumption Survey weights (pspwght) correct for non-response
    Post-stratification weights used 'where possible' to ensure demographic representativeness (Section 3.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of People Are Not Just Their Countries. Disentangling Social Determinants of LLM Value Alignment Across Europe." pith.science (2026). https://pith.science/paper/THVIARTW

@misc{pith2026260807367,
  author       = {Pith},
  title        = {Pith review of: People Are Not Just Their Countries. Disentangling Social Determinants of LLM Value Alignment Across Europe},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/THVIARTW}},
  note         = {Machine review of arXiv:2608.07367}
}
read the original abstract

As Large Language Models (LLMs) are increasingly used as a primary source of information and advice, understanding their alignment to humans in terms of values becomes a pressing concern. A growing literature has leveraged large scale surveys to investigate to what extent LLMs' and humans' stated values and opinions align. With limited exceptions, studied populations have been defined country borders or cultural bounds. Yet, this focus neglects the role that socio-demographic divides may play for value alignment disparities. Relying on the European Social Survey, we address this knowledge gap by considering value alignment displayed with respect to 10 prominent commercial LLMs in terms of 15 socio-demographic variables as well as country of residence. Our analyses reveal that LLMs are indeed unequally aligned to the values of different socio-demographic groups, notably those defined by education, income, occupation and religion. When examining alignment at the individual level, a respondent's country, taken as a stand-alone variable, explains a substantial amount of variation that is on par with the full set of considered socio-demographics. Further disentangling the respective role of country-level and socio-demographic factors, we find they are complementary in explaining value alignment patterns, with their relative weights varying across the subset of questions considered.

Figures

Figures reproduced from arXiv: 2608.07367 by the authors.

Figure 1
Figure 1. Cross-model mean deviation from the mean pop [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Cross-model mean deviation from the mean popu [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Cross-model mean deviations pre (white) and post [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (18 more)
Figure 4
Figure 4. Figure 4: Mean test R2 over 10-fold cross validation with standard deviations, for five methods predicting individual alignment scores. The upper chart shows scores on Q, the set of questions answered by all LLMs; the lower chart shows scores on PVQ questions. Covariates vary be…
Figure 5
Figure 5. Figure 5: Number of value-laden questions answered per respondent [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Refusal to answer rates by question and topic [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Average answer to survey question across the whole population indicated by the star, and LLM majority vote answers. [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: Refusals/invalid answers for each question per model. The emphasised questions are excluded for questions at least [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Pairwise Pearson correlations between models’ answers and alignment scores. The upper triangle shows correla [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 10
Figure 10. Figure 10: Cross-model mean alignment deviation by socio-demographic subgroup and country. The left panel shows each [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: Bootstrap mean alignment scores (95% CIs, n = 5,000 resamples) for each LLM, shown for countries. [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 12
Figure 12. Figure 12: Bootstrap mean alignment scores (95% CIs, n = 5,000 resamples) for each LLM, shown for socio-demographic [PITH_FULL_IMAGE:figures/full_fig_p023_12.png]
Figure 13
Figure 13. Figure 13: The three figures show some results of the joint bootstrapped as described in Algorithm [PITH_FULL_IMAGE:figures/full_fig_p026_13.png]
Figure 14
Figure 14. Figure 14: Alignment score plotted against response extremeness for each LLM model. Semi-transparent dots show individual [PITH_FULL_IMAGE:figures/full_fig_p027_14.png]
Figure 15
Figure 15. Figure 15: Bootstrap estimates of alignment deviation by socio-demographic group for a synthetic midpoint model. Positive [PITH_FULL_IMAGE:figures/full_fig_p028_15.png]
Figure 16
Figure 16. Figure 16: Histograms of estimated propensity scores by country. Red: distribution of scores among respondents in the country; [PITH_FULL_IMAGE:figures/full_fig_p030_16.png]
Figure 17
Figure 17. Figure 17: Mean absolute standardised mean difference (SMD) across the 30 countries (averaged across socio-demographic [PITH_FULL_IMAGE:figures/full_fig_p031_17.png]
Figure 18
Figure 18. Figure 18: Mean absolute standardised mean difference (SMD) across the socio-demographic variables considered, pre- and [PITH_FULL_IMAGE:figures/full_fig_p032_18.png]
Figure 19
Figure 19. Figure 19: Country means after propensity weighting, ob [PITH_FULL_IMAGE:figures/full_fig_p033_19.png]
Figure 21
Figure 21. Figure 21: Country means after propensity weighting, ob [PITH_FULL_IMAGE:figures/full_fig_p033_21.png]
Figure 22
Figure 22. Figure 22: Train vs. test R2 for five prediction methods across LLM models (10-fold CV, post-stratification weighted). Solid bars = train R2 ; hatched bars = test R2 ; error bars = SD across folds. Methods combine country fixed effects and/or socio￾demographic predictors, fitted…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

123 extracted references · 29 canonical work pages

  1. [1]

    International Conference on Machine Learning , pages=

    Whose opinions do language models reflect? , author=. International Conference on Machine Learning , pages=. 2023 , organization=

  2. [2]

    Llama meets

    Chalkidis, Ilias and Brandl, Stephanie , month = mar, year =. Llama meets. doi:10.48550/arXiv.2403.13592 , abstract =

  3. [3]

    Algorithmic Fidelity of Large Language Models in Generating Synthetic G erman Public Opinions: A Case Study

    Ma, Bolei and Yoztyurk, Berk and Haensch, Anna-Carolina and Wang, Xinpeng and Herklotz, Markus and Kreuter, Frauke and Plank, Barbara and A enmacher, Matthias. Algorithmic Fidelity of Large Language Models in Generating Synthetic G erman Public Opinions: A Case Study. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics....

  4. [4]

    Social Science Computer Review , author =

    Vox. Social Science Computer Review , author =. 2025 , pages =. doi:10.1177/08944393251337014 , abstract =

  5. [5]

    Schäfer, Johannes and Combs, Aidan and Bagdon, Christopher and Li, Jiahui and Probol, Nadine and Greschner, Lynn and Papay, Sean and Resendiz, Yarik Menchaca and Velutharambath, Aswathy and Wührl, Amelie and Weber, Sabine and Klinger, Roman , month = may, year =. Which. doi:10.48550/arXiv.2410.08820 , abstract =

  6. [6]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    On the alignment of large language models with global human opinion , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  7. [7]

    Benchmarking

    Ju, Chengyi and Shi, Weijie and Liu, Chengzhong and Ji, Jiaming and Zhang, Jipeng and Zhang, Ruiyuan and Xu, Jiajie and Yang, Yaodong and Han, Sirui and Guo, Yike , file =. Benchmarking

  8. [8]

    Weeber, Franziska and Ceron, Tanise and Padó, Sebastian , month = aug, year =. Do. doi:10.48550/arXiv.2508.05553 , abstract =

Show all 123 references
  1. [9]

    Sociodemographic

    Sun, Huaman and Pei, Jiaxin and Choi, Minje and Jurgens, David , file =. Sociodemographic

  2. [10]

    Sukiennik, Nicholas and Gao, Chen and Xu, Fengli and Li, Yong , month = apr, year =. An. doi:10.48550/arXiv.2504.08863 , abstract =

  3. [11]

    Proceedings of the Eighth AAAI/ACM Conference on AI, Ethics, and Society (AIES 2025) , year =

    Batzner, Jan and Stocker, Volker and Schmid, Stefan and Kasneci, Gjergji , title =. Proceedings of the Eighth AAAI/ACM Conference on AI, Ethics, and Society (AIES 2025) , year =

  4. [12]

    He, Zihao and Guo, Siyi and Rao, Ashwin and Lerman, Kristina , month = jun, year =. Whose. doi:10.48550/arXiv.2402.11114 , abstract =

  5. [13]

    Questioning the

    Dominguez-Olmedo, Ricardo and Hardt, Moritz and Mendler-Dünner, Celestine , file =. Questioning the

  6. [14]

    Political

    Röttger, Paul and Hofmann, Valentin and Pyatkin, Valentina and Hinck, Musashi and Kirk, Hannah Rose and Schütze, Hinrich and Hovy, Dirk , month = jun, year =. Political. doi:10.48550/arXiv.2402.16786 , abstract =

  7. [15]

    PNAS Nexus , author =

    Cultural bias and cultural alignment of large language models , volume =. PNAS Nexus , author =. 2024 , keywords =. doi:10.1093/pnasnexus/pgae346 , abstract =

  8. [16]

    Investigating Cultural Alignment of Large Language Models

    AlKhamissi, Badr and ElNokrashy, Muhammad and Alkhamissi, Mai and Diab, Mona. Investigating Cultural Alignment of Large Language Models. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. 2024. doi:10.18653/v1/2024.acl-long.671

  9. [17]

    Durmus, Esin and Nguyen, Karina and Liao, Thomas I. and Schiefer, Nicholas and Askell, Amanda and Bakhtin, Anton and Chen, Carol and Hatfield-Dodds, Zac and Hernandez, Danny and Joseph, Nicholas and Lovitt, Liane and McCandlish, Sam and Sikder, Orowa and Tamkin, Alex and Thamk...

  10. [18]

    Randomness,

    Khan, Ariba and Casper, Stephen and Hadfield-Menell, Dylan , month = jun, year =. Randomness,. Proceedings of the 2025. doi:10.1145/3715275.3732147 , abstract =

  11. [19]

    Arora, Arnav and Kaffee, Lucie-Aimée and Augenstein, Isabelle , year =. Probing

  12. [20]

    Proceedings of the first workshop on cross-cultural considerations in NLP (C3NLP) , pages=

    Assessing cross-cultural alignment between ChatGPT and human societies: An empirical study , author=. Proceedings of the first workshop on cross-cultural considerations in NLP (C3NLP) , pages=

  13. [21]

    Tjuatja, Lindia and Chen, Valerie and Wu, Sherry Tongshuang and Talwalkar, Ameet and Neubig, Graham , month = feb, year =. Do. doi:10.48550/arXiv.2311.04076 , abstract =

  14. [22]

    Survey of

    Pawar, Siddhesh and Park, Junyeong and Jin, Jiho and Arora, Arnav and Myung, Junho and Yadav, Srishti and Haznitrama, Faiz Ghifari and Song, Inhwa and Oh, Alice and Augenstein, Isabelle , month = oct, year =. Survey of. doi:10.48550/arXiv.2411.00860 , abstract =

  15. [23]

    and Ren, Richard and Phan, Long and Mu, Norman and Khoja, Adam and Zhang, Oliver and Hendrycks, Dan , month = feb, year =

    Mazeika, Mantas and Yin, Xuwang and Tamirisa, Rishub and Lim, Jaehyuk and Lee, Bruce W. and Ren, Richard and Phan, Long and Mu, Norman and Khoja, Adam and Zhang, Oliver and Hendrycks, Dan , month = feb, year =. Utility. doi:10.48550/arXiv.2502.08640 , abstract =

  16. [24]

    and Liu, Ziquan and Ferianc, Martin and Treleaven, Philip and Rodrigues, Miguel , month = may, year =

    Masoud, Reem I. and Liu, Ziquan and Ferianc, Martin and Treleaven, Philip and Rodrigues, Miguel , month = may, year =. Cultural. doi:10.48550/arXiv.2309.12342 , abstract =

  17. [25]

    Are Large Language Models Consistent over Value-laden Questions?

    Moore, Jared and Deshpande, Tanvi and Yang, Diyi. Are Large Language Models Consistent over Value-laden Questions?. Findings of the Association for Computational Linguistics: EMNLP 2024. 2024. doi:10.18653/v1/2024.findings-emnlp.891

  18. [26]

    and Kim, Hyunwoo and Santy, Sebastin and Sorensen, Taylor and Lin, Bill Yuchen and Dziri, Nouha and Ren, Xiang and Choi, Yejin , month = aug, year =

    Li, Huihan and Jiang, Liwei and Hwang, Jena D. and Kim, Hyunwoo and Santy, Sebastin and Sorensen, Taylor and Lin, Bill Yuchen and Dziri, Nouha and Ren, Xiang and Choi, Yejin , month = aug, year =. doi:10.48550/arXiv.2404.10199 , abstract =

  19. [27]

    and Dave, Shachi and Prabhakaran, Vinodkumar and Dev, Sunipa

    Jha, Akshita and Davani, Aida and Reddy, Chandan K. and Dave, Shachi and Prabhakaran, Vinodkumar and Dev, Sunipa. S ee GULL : A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Models. Proceedings of the 61st Annual Meeting of the Association for Com...

  20. [28]

    Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems

    Wan, Yixin and Zhao, Jieyu and Chadha, Aman and Peng, Nanyun and Chang, Kai-Wei. Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems. Findings of the Association for Computational Linguistics: EMNLP 2023. 2023. doi:10.18653/v1/2023...

  21. [29]

    Towards Measuring and Modeling ``Culture'' in LLM s: A Survey

    Adilazuarda, Muhammad Farid and Mukherjee, Sagnik and Lavania, Pradhyumna and Singh, Siddhant Shivdutt and Aji, Alham Fikri and O ' Neill, Jacki and Modi, Ashutosh and Choudhury, Monojit. Towards Measuring and Modeling ``Culture'' in LLM s: A Survey. Proceedings of the 2024 Co...

  22. [30]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values? , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

  23. [31]

    Adilazuarda, Muhammad Farid and Mukherjee, Sagnik and Lavania, Pradhyumna and Singh, Siddhant and Aji, Alham Fikri and O'Neill, Jacki and Modi, Ashutosh and Choudhury, Monojit , month = sep, year =. Towards. doi:10.48550/arXiv.2403.15412 , abstract =

  24. [32]

    doi:10.48550/arXiv.2501.01031 , abstract =

    Seo, Wonduk and Yuan, Zonghao and Bu, Yi , month = may, year =. doi:10.48550/arXiv.2501.01031 , abstract =

  25. [33]

    and Narayanan, Arvind , month = apr, year =

    Veselovsky, Veniamin and Argin, Berke and Stroebl, Benedikt and Wendler, Chris and West, Robert and Evans, James and Griffiths, Thomas L. and Narayanan, Arvind , month = apr, year =. Localized. doi:10.48550/arXiv.2504.10191 , abstract =

  26. [34]

    Aligning

    Hwang, EunJeong and Majumder, Bodhisattwa Prasad and Tandon, Niket , month = may, year =. Aligning. doi:10.48550/arXiv.2305.14929 , abstract =

  27. [35]

    doi:10.48550/arXiv.2505.14422 , abstract =

    Mao, Xutao and Tao, Ezra Xuanru and Wang, Leyao , month = nov, year =. doi:10.48550/arXiv.2505.14422 , abstract =

  28. [36]

    Beyond Probabilities: Unveiling the Misalignment in Evaluating Large Language Models

    Lyu, Chenyang and Wu, Minghao and Aji, Alham. Beyond Probabilities: Unveiling the Misalignment in Evaluating Large Language Models. Proceedings of the 1st Workshop on Towards Knowledgeable Language Models (KnowLLM 2024). 2024. doi:10.18653/v1/2024.knowllm-1.10

  29. [37]

    Evaluating the

    Scherrer, Nino and Shi, Claudia and Feder, Amir and Blei, David M , file =. Evaluating the

  30. [38]

    Proceedings of the AAAI Conference on Artificial Intelligence , author =

    Value. Proceedings of the AAAI Conference on Artificial Intelligence , author =. 2024 , pages =. doi:10.1609/aaai.v38i18.29970 , abstract =

  31. [39]

    Chen, Kai and He, Zihao and Yan, Jun and Shi, Taiwei and Lerman, Kristina , month = jun, year =. How. doi:10.48550/arXiv.2402.11725 , abstract =

  32. [40]

    Values in the

    Huang, Saffron and Durmus, Esin and McCain, Miles and Handa, Kunal and Tamkin, Alex and Hong, Jerry and Stern, Michael and Somani, Arushi and Zhang, Xiuruo and Ganguli, Deep , month = apr, year =. Values in the. doi:10.48550/arXiv.2504.15236 , abstract =

  33. [41]

    SSRN Electronic Journal , author =

    The political ideology of conversational. SSRN Electronic Journal , author =. 2023 , file =. doi:10.2139/ssrn.4316084 , language =

  34. [42]

    Humanities and Social Sciences Communications , author =

    “. Humanities and Social Sciences Communications , author =. 2025 , pages =. doi:10.1057/s41599-025-04465-z , abstract =

  35. [43]

    and Shwartz, Vered , month = may, year =

    Liu, Zhuozhuo Joy and Samir, Farhan and Bhatia, Mehar and Nelson, Laura K. and Shwartz, Vered , month = may, year =. Is. doi:10.48550/arXiv.2505.18322 , abstract =

  36. [44]

    and Held, William and Yang, Diyi , month = jun, year =

    Ryan, Michael J. and Held, William and Yang, Diyi , month = jun, year =. Unintended. doi:10.48550/arXiv.2402.15018 , abstract =

  37. [45]

    Proceedings of the 9th Widening NLP Workshop , pages=

    ValueCompass: A framework for measuring contextual value alignment between human and LLMs , author=. Proceedings of the 9th Widening NLP Workshop , pages=

  38. [46]

    Hase, Peter and Diab, Mona and Celikyilmaz, Asli and Li, Xian and Kozareva, Zornitsa and Stoyanov, Veselin and Bansal, Mohit and Iyer, Srinivasan , month = nov, year =. Do. doi:10.48550/arXiv.2111.13654 , abstract =

  39. [47]

    Aakanksha and Ahmadian, Arash and Ermis, Beyza and Goldfarb-Tarrant, Seraphina and Kreutzer, Julia and Fadaee, Marzieh and Hooker, Sara , month = jul, year =. The. doi:10.48550/arXiv.2406.18682 , abstract =

  40. [48]

    doi:10.48550/arXiv.2405.13974 , abstract =

    Pistilli, Giada and Leidinger, Alina and Jernite, Yacine and Kasirzadeh, Atoosa and Luccioni, Alexandra Sasha and Mitchell, Margaret , month = may, year =. doi:10.48550/arXiv.2405.13974 , abstract =

  41. [49]

    Towards Identifying Social Bias in Dialog Systems: Framework, Dataset, and Benchmark

    Zhou, Jingyan and Deng, Jiawen and Mi, Fei and Li, Yitong and Wang, Yasheng and Huang, Minlie and Jiang, Xin and Liu, Qun and Meng, Helen. Towards Identifying Social Bias in Dialog Systems: Framework, Dataset, and Benchmark. Findings of the Association for Computational Lingui...

  42. [50]

    Advances in Neural Information Processing Systems , volume=

    The PRISM alignment dataset: What participatory, representative and individualised human feedback reveals about the subjective and multicultural alignment of large language models , author=. Advances in Neural Information Processing Systems , volume=

  43. [51]

    doi:10.48550/arXiv.2410.18906 , abstract =

    Azzopardi, Leif and Moshfeghi, Yashar , month = nov, year =. doi:10.48550/arXiv.2410.18906 , abstract =

  44. [52]

    doi:10.48550/arXiv.2309.03876 , abstract =

    Haller, Patrick and Aynetdinov, Ansar and Akbik, Alan , month = sep, year =. doi:10.48550/arXiv.2309.03876 , abstract =

  45. [53]

    Sociological Methods & Research , author =

    Machine. Sociological Methods & Research , author =. 2025 , pages =. doi:10.1177/00491241251330582 , abstract =

  46. [54]

    Williams, Tristan and Weeber, Franziska and Padó, Sebastian and Akbik, Alan , month = jan, year =. Beyond. doi:10.48550/arXiv.2601.15755 , abstract =

  47. [55]

    Missing the

    Sen, Indira and Lutz, Marlene and Rogers, Elisa and Garcia, David and Strohmaier, Markus , file =. Missing the

  48. [56]

    Feng, Shangbin and Park, Chan Young and Liu, Yuhan and Tsvetkov, Yulia , file =. From

  49. [57]

    Cultivating

    Zhang, Lily Hong and Milli, Smitha and Jusko, Karen and Smith, Jonathan and Amos, Brandon and Bouaziz, Wassim and Revel, Manon and Kussman, Jack and Sheynin, Yasha and Titus, Lisa and Radharapu, Bhaktipriya and Yu, Jane and Sarma, Vidya and Rose, Kris and Nickel, Maximilian , ...

  50. [58]

    Journal of Cross-Cultural Psychology , author =

    Whence. Journal of Cross-Cultural Psychology , author =. 2011 , pages =. doi:10.1177/0022022110381429 , abstract =

  51. [59]

    arXiv preprint arXiv:2411.10109 , volume=

    Generative agent simulations of 1,000 people , author=. arXiv preprint arXiv:2411.10109 , volume=

  52. [60]

    Nature Machine Intelligence , volume=

    The benefits, risks and bounds of personalizing the alignment of large language models to individuals , author=. Nature Machine Intelligence , volume=. 2024 , publisher=

  53. [61]

    Humanities and Social Sciences Communications , author =

    Why human–. Humanities and Social Sciences Communications , author =. 2025 , pages =. doi:10.1057/s41599-025-04532-5 , language =

  54. [62]

    and Plank, Barbara and Kreuter, Frauke

    Ma, Bolei and Wang, Xinpeng and Hu, Tiancheng and Haensch, Anna-Carolina and Hedderich, Michael A. and Plank, Barbara and Kreuter, Frauke. The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Models. Findings of the Association for Compu...

  55. [63]

    Fairness in

    Abeliuk, Andrés and Gaete, Vanessa and Bro, Naim , month = jan, year =. Fairness in. doi:10.48550/arXiv.2501.15351 , abstract =

  56. [64]

    Kennedy, Molly and Parker, Ali and Liu, Yihong and Schütze, Hinrich , year =. Left,. doi:10.48550/ARXIV.2601.05835 , abstract =

  57. [65]

    Russo, Giuseppe and Nozza, Debora and Röttger, Paul and Hovy, Dirk , file =. The

  58. [66]

    Persona-driven

    Kreutner, Maximilian and Lutz, Marlene and Strohmaier, Markus , file =. Persona-driven

  59. [67]

    Nadeem, Afrozah and Seth, Agrima and Nasim, Mehwish and Naseem, Usman , month = feb, year =. Bias. doi:10.48550/arXiv.2601.23001 , abstract =

  60. [68]

    Shetty, Anudeex and Naseem, Usman and Aletras, Nikolaos and Dras, Mark and Ji, Heng and Nakov, Preslav , month = mar, year =. Towards. doi:10.20944/preprints202603.1876.v1 , language =

  61. [69]

    doi:10.48550/arXiv.2601.04885 , abstract =

    Sun, Ao and Wang, Xiaoyu and Tan, Zhe and Li, Yu and Zhu, Jiachen and Su, Shu and Jia, Yuheng , month = jan, year =. doi:10.48550/arXiv.2601.04885 , abstract =

  62. [70]

    Cultural

    Liemt, Erin van and Shelby, Renee and Smart, Andrew and Kumbale, Sinchana and Zhang, Richard and Dixit, Neha and Rashid, Qazi Mamunur and Smith-Loud, Jamila , month = mar, year =. Cultural. doi:10.48550/arXiv.2603.05723 , abstract =

  63. [71]

    Operationalizing

    Ali, Dalia and Zhao, Dora and Koenecke, Allison and Papakyriakopoulos, Orestis , file =. Operationalizing

  64. [72]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    AlignSurvey: A Comprehensive Benchmark for Human Preferences Alignment in Social Surveys , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  65. [73]

    Zhao, Weixiang and Li, Haozhen and Zhao, Yanyan and Liu, Haixiao and Li, Biye and Liu, Ting and Qin, Bing , file =

  66. [74]

    Rethinking

    AlKhamissi, Mai and Xiao, Yunze and AlKhamissi, Badr and Diab, Mona , file =. Rethinking

  67. [75]

    Pan, Weihang and Yu, Zhengxu , file =

  68. [76]

    Proceedings of the 2026

    Yun, Bhada and Su, Renn and Wang, April Yi , month = apr, year =. Proceedings of the 2026. doi:10.1145/3772318.3790566 , abstract =

  69. [77]

    AI and Ethics , author =

    Towards friendly. AI and Ethics , author =. 2026 , pages =. doi:10.1007/s43681-026-01070-x , abstract =

  70. [78]

    Jiang, Liwei and Sorensen, Taylor and Levine, Sydney and Choi, Yejin , file =. Can

  71. [79]

    Guan, Jian and Wu, Junfei and Li, Jia-Nan and Cheng, Chuanqi and Wu, Wei , file =. A

  72. [80]

    Mihalcea, Rada and Ignat, Oana and Bai, Longju and Borah, Angana and Chiruzzo, Luis and Jin, Zhijing and Kwizera, Claude and Nwatu, Joan and Poria, Soujanya and Solorio, Thamar , booktitle=. Why

  73. [81]

    Language

    Suh, Joseph and Jahanparast, Erfan and Moon, Suhong and Kang, Minwoo and Chang, Serina , file =. Language

  74. [82]

    Orlikowski, Matthias and Pei, Jiaxin and Röttger, Paul and Cimiano, Philipp and Jurgens, David and Hovy, Dirk , file =. Beyond

  75. [83]

    Analyze the

    Fedzechkina, Masha and Gualdoni, Eleonora and Williamson, Sinead and Metcalf, Katherine and Seto, Skyler and Theobald, Barry-John , file =. Analyze the

  76. [84]

    Steering into

    Sundar, Anirudh and Williamson, Sinead and Metcalf, Katherine and Theobald, Barry-John and Seto, Skyler and Fedzechkina, Masha , file =. Steering into

  77. [85]

    Information Processing & Management , author =

    Towards realistic evaluation of cultural value alignment in large language models:. Information Processing & Management , author =. 2025 , pages =. doi:10.1016/j.ipm.2025.104099 , language =

  78. [86]

    Bulte, Bram and Terryn, Ayla Rigouts , file =

  79. [87]

    LLM Tropes: Revealing Fine-Grained Values and Opinions in Large Language Models

    Wright, Dustin and Arora, Arnav and Borenstein, Nadav and Yadav, Srishti and Belongie, Serge and Augenstein, Isabelle. LLM Tropes: Revealing Fine-Grained Values and Opinions in Large Language Models. Findings of the Association for Computational Linguistics: EMNLP 2024. 2024. ...

  80. [88]

    Journal of Cross-Cultural Psychology , author =

    The. Journal of Cross-Cultural Psychology , author =. 2011 , pages =. doi:10.1177/0022022110362757 , abstract =

  81. [89]

    Sorensen, Taylor and Moore, Jared and Fisher, Jillian and Gordon, Mitchell and Mireshghallah, Niloofar and Rytting, Christopher Michael and Ye, Andre and Jiang, Liwei and Lu, Ximing and Dziri, Nouha and Althoff, Tim and Choi, Yejin , month = aug, year =. A. doi:10.48550/arXiv....

  82. [90]

    Benchmarking Distributional Alignment of Large Language Models

    Meister, Nicole and Guestrin, Carlos and Hashimoto, Tatsunori. Benchmarking Distributional Alignment of Large Language Models. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologie...

  83. [91]

    Comparative Political Studies , author =

    Value. Comparative Political Studies , author =. 2016 , pages =. doi:10.1177/0010414015600458 , abstract =

  84. [92]

    Journal of Cross-Cultural Psychology , author =

    Clashing. Journal of Cross-Cultural Psychology , author =. 2020 , pages =. doi:10.1177/0022022120956716 , abstract =

  85. [93]

    Evaluating

    Zhong, Qishuai and Li, Zongmin and Fan, Siqi and Sun, Aixin , month = may, year =. Evaluating. doi:10.48550/arXiv.2505.21362 , abstract =

  86. [94]

    Social Sciences & Humanities Open , author =

    Do demographic predictors of personal values vary by context?. Social Sciences & Humanities Open , author =. 2022 , pages =. doi:10.1016/j.ssaho.2022.100264 , abstract =

  87. [95]

    Journal of Cross-Cultural Psychology , author =

    Sociodemographic. Journal of Cross-Cultural Psychology , author =. 2014 , pages =. doi:10.1177/0022022113513402 , abstract =

  88. [96]

    Public Opinion Quarterly , author =

    Bringing. Public Opinion Quarterly , author =. 2008 , pages =. doi:10.1093/poq/nfn035 , language =

  89. [97]

    Behaviour & Information Technology , author =

    Challenges and future directions for integration of large language models into socio-technical systems , issn =. Behaviour & Information Technology , author =. 2024 , pages =. doi:10.1080/0144929X.2024.2431068 , abstract =

  90. [98]

    Proceedings of the National Academy of Sciences , author =

    The simulation of judgment in. Proceedings of the National Academy of Sciences , author =. 2025 , pages =. doi:10.1073/pnas.2518443122 , abstract =

  91. [99]

    States of knowledge , pages=

    Ordering knowledge, ordering society , author=. States of knowledge , pages=. 2004 , publisher=

  92. [100]

    2025 , howpublished =

  93. [101]

    Biometrika , volume=

    The central role of the propensity score in observational studies for causal effects , author=. Biometrika , volume=. 1983 , publisher=

  94. [102]

    Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages=

    Xgboost: A scalable tree boosting system , author=. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages=

  95. [103]

    Dolata, Mateusz and Feuerriegel, Stefan and Schwabe, Gerhard , journal=. A. 2022 , publisher=

  96. [104]

    Fairness and

    Selbst, Andrew D and Boyd, Danah and Friedler, Sorelle A and Venkatasubramanian, Suresh and Vertesi, Janet , booktitle=. Fairness and

  97. [105]

    Journal of communication , volume=

    Users as research inventions: How research categories perpetuate inequities , author=. Journal of communication , volume=. 1989 , publisher=

  98. [106]

    Systems Research and Behavioral Science , volume =

    Kroes, Peter and Franssen, Maarten and Poel, Ibo van de and Ottens, Maarten , title =. Systems Research and Behavioral Science , volume =. doi:https://doi.org/10.1002/sres.703 , url =. https://onlinelibrary.wiley.com/doi/pdf/10.1002/sres.703 , year =

  99. [107]

    Language Model Alignment in Multilingual Trolley Problems , url =

    Jin, Zhijing and Kleiman-Weiner, Max and Piatti, Giorgio and Levine, Sydney and Liu, Jiarui and Gonzalez Adauto, Fernando and Ortu, Francesco and Strausz, Andr\'. Language Model Alignment in Multilingual Trolley Problems , url =. International Conference on Learning Representa...

  100. [108]

    On the algorithmic bias of aligning large language models with

    Xiao, Jiancong and Li, Ziniu and Xie, Xingyu and Getzen, Emily and Fang, Cong and Long, Qi and Su, Weijie J , journal=. On the algorithmic bias of aligning large language models with. 2025 , publisher=

  101. [109]

    European Journal of Political Research , volume=

    The theory of human development: A cross-cultural analysis , author=. European Journal of Political Research , volume=. 2003 , publisher=

  102. [110]

    2005 , publisher=

    Modernization, Cultural Change, and Democracy: The Human Development Sequence , author=. 2005 , publisher=

  103. [111]

    Advances in Experimental Social Psychology , volume=

    Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries , author=. Advances in Experimental Social Psychology , volume=. 1992 , publisher=

  104. [112]

    1980 , address =

    Hofstede, Geert , title =. 1980 , address =

  105. [113]

    , author=

    Age and gender differences in human values: A 20-nation study. , author=. Psychology and aging , volume=. 2020 , publisher=

  106. [114]

    Comparative Political Studies , volume=

    Misconceptions of measurement equivalence: Time for a paradigm shift , author=. Comparative Political Studies , volume=. 2016 , publisher=

  107. [115]

    Friedman , journal =

    Jerome H. Friedman , journal =. Greedy Function Approximation: A Gradient Boosting Machine , urldate =

  108. [116]

    Missing the Margins : A Systematic Literature Review on the Demographic Representativeness of LLM s

    Sen, Indira and Lutz, Marlene and Rogers, Elisa and Garcia, David and Strohmaier, Markus. Missing the Margins : A Systematic Literature Review on the Demographic Representativeness of LLM s. Findings of the Association for Computational Linguistics: ACL 2025. 2025. doi:10.1865...

  109. [117]

    arXiv preprint arXiv:2404.08382 , year=

    Look at the text: Instruction-tuned language models are more robust multiple choice selectors than you think , author=. arXiv preprint arXiv:2404.08382 , year=

  110. [118]

    nationology

    On “nationology”: The gravitational field of national culture , author=. Journal of Cross-Cultural Psychology , volume=. 2021 , publisher=

  111. [119]

    Rethinking

    Xu, Yingwei Wayne and Chi, Christina G and Gursoy, Dogan and Cai, Ruiying Raine , journal=. Rethinking. 2025 , publisher=

  112. [120]

    Be friendly, not friends: How

    Sun, Yuan and Wang, Ting , booktitle=. Be friendly, not friends: How

  113. [121]

    Frontiers in Psychology , volume=

    Outsourcing cognition: the psychological costs of AI-era convenience , author=. Frontiers in Psychology , volume=. 2025 , publisher=

  114. [122]

    AJOB neuroscience , volume=

    Anthropomorphism in AI , author=. AJOB neuroscience , volume=. 2020 , publisher=

  115. [123]

    AI and Ethics , volume=

    Anthropomorphism in AI: hype and fallacy , author=. AI and Ethics , volume=. 2024 , publisher=

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.