{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2023:BWKHUD37M6BSRM6FACJIVV42PL","short_pith_number":"pith:BWKHUD37","schema_version":"1.0","canonical_sha256":"0d947a0f7f678328b3c500928ad79a7adc7f43eb29475951988da37ce5282ffb","source":{"kind":"arxiv","id":"2306.16388","version":2},"attestation_state":"computed","paper":{"title":"Towards Measuring the Representation of Subjective Global Opinions in Language Models","license":"http://creativecommons.org/licenses/by/4.0/","headline":"Large language models produce answers that match opinions from the United States and certain European and South American countries more closely than opinions from other nations.","cross_cats":["cs.AI"],"primary_cat":"cs.CL","authors_text":"Alex Tamkin, Amanda Askell, Anton Bakhtin, Carol Chen, Danny Hernandez, Deep Ganguli, Esin Durmus, Jack Clark, Janel Thamkul, Jared Kaplan, Karina Nguyen, Liane Lovitt, Nicholas Joseph, Nicholas Schiefer, Orowa Sikder, Sam McCandlish, Thomas I. Liao, Zac Hatfield-Dodds","submitted_at":"2023-06-28T17:31:53Z","abstract_excerpt":"Large language models (LLMs) may not equitably represent diverse global perspectives on societal issues. In this paper, we develop a quantitative framework to evaluate whose opinions model-generated responses are more similar to. We first build a dataset, GlobalOpinionQA, comprised of questions and answers from cross-national surveys designed to capture diverse opinions on global issues across different countries. Next, we define a metric that quantifies the similarity between LLM-generated survey responses and human responses, conditioned on country. With our framework, we run three experimen"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":true},"canonical_record":{"source":{"id":"2306.16388","kind":"arxiv","version":2},"metadata":{"license":"http://creativecommons.org/licenses/by/4.0/","primary_cat":"cs.CL","submitted_at":"2023-06-28T17:31:53Z","cross_cats_sorted":["cs.AI"],"title_canon_sha256":"2a9629f5efb27fae810debc0ee06bc748ce2f2830a71e275ed726aa837d9168a","abstract_canon_sha256":"56e02e29c283cdfb8325bad47d002bd7e49cfa011c01c414d24d796899052e7e"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-05-17T23:38:48.652203Z","signature_b64":"A7Enc+Mo+e7UfElwIoQzBk4KD9vsAn9MPLz8N+zTtXTJhvwGc8Zvj5i0QsdbzlZfPvV0oCqWzqNNtATi1+BqCQ==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"0d947a0f7f678328b3c500928ad79a7adc7f43eb29475951988da37ce5282ffb","last_reissued_at":"2026-05-17T23:38:48.651547Z","signature_status":"signed_v1","first_computed_at":"2026-05-17T23:38:48.651547Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"Towards Measuring the Representation of Subjective Global Opinions in Language Models","license":"http://creativecommons.org/licenses/by/4.0/","headline":"Large language models produce answers that match opinions from the United States and certain European and South American countries more closely than opinions from other nations.","cross_cats":["cs.AI"],"primary_cat":"cs.CL","authors_text":"Alex Tamkin, Amanda Askell, Anton Bakhtin, Carol Chen, Danny Hernandez, Deep Ganguli, Esin Durmus, Jack Clark, Janel Thamkul, Jared Kaplan, Karina Nguyen, Liane Lovitt, Nicholas Joseph, Nicholas Schiefer, Orowa Sikder, Sam McCandlish, Thomas I. Liao, Zac Hatfield-Dodds","submitted_at":"2023-06-28T17:31:53Z","abstract_excerpt":"Large language models (LLMs) may not equitably represent diverse global perspectives on societal issues. In this paper, we develop a quantitative framework to evaluate whose opinions model-generated responses are more similar to. We first build a dataset, GlobalOpinionQA, comprised of questions and answers from cross-national surveys designed to capture diverse opinions on global issues across different countries. Next, we define a metric that quantifies the similarity between LLM-generated survey responses and human responses, conditioned on country. With our framework, we run three experimen"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"By default, LLM responses tend to be more similar to the opinions of certain populations, such as those from the USA, and some European and South American countries, highlighting the potential for biases.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That the chosen cross-national survey responses serve as an unbiased and representative ground truth for each country's population-level opinions on the selected issues.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"LLMs default to responses more similar to opinions from the USA and some European and South American countries; prompting for a country shifts alignment but can introduce stereotypes, while translation does not reliably match language speakers.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Large language models produce answers that match opinions from the United States and certain European and South American countries more closely than opinions from other nations.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"aac285012dafa9d149fde2ab710877079922bb89c05f73a09c85926c9f5e265c"},"source":{"id":"2306.16388","kind":"arxiv","version":2},"verdict":{"id":"2db2b6b3-a4c3-4a9c-b679-b81b725e14f5","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-16T07:37:44.317773Z","strongest_claim":"By default, LLM responses tend to be more similar to the opinions of certain populations, such as those from the USA, and some European and South American countries, highlighting the potential for biases.","one_line_summary":"LLMs default to responses more similar to opinions from the USA and some European and South American countries; prompting for a country shifts alignment but can introduce stereotypes, while translation does not reliably match language speakers.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That the chosen cross-national survey responses serve as an unbiased and representative ground truth for each country's population-level opinions on the selected issues.","pith_extraction_headline":"Large language models produce answers that match opinions from the United States and certain European and South American countries more closely than opinions from other nations."},"references":{"count":97,"sample":[{"doi":"10.1145/3461702.3462624","year":2021,"title":"InProceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society(Virtual Event, USA)(AIES ’21)","work_id":"7f62359d-f2d6-4fcb-8c8b-770b094135b8","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2011,"title":"Subjective natural language problems: Motivations, applications, characterizations, and implications","work_id":"ae449bd3-929a-434d-9915-64f1831905db","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":null,"title":"Probing pre-trained language models for cross-cultural differences in values","work_id":"9e2a3ada-aae4-42e7-adb6-9f67bed7dacd","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2023,"title":"URL https://aclanthology.org/2023","work_id":"988509e6-9e6d-48e9-9e08-3137c834ae56","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2021,"title":"A general language assistant as a laboratory for alignment","work_id":"51b13307-1831-4a7b-bea8-559d663289df","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":97,"snapshot_sha256":"5b52fb416d23dfd459667638bf336cd3f29f7148e77099dad182eab6a8834939","internal_anchors":8},"formal_canon":{"evidence_count":2,"snapshot_sha256":"b4a476d2f5ca70bb77e33ee2ba11791bd9054f5443ea29cbe2733905de2f6103"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2306.16388","created_at":"2026-05-17T23:38:48.651634+00:00"},{"alias_kind":"arxiv_version","alias_value":"2306.16388v2","created_at":"2026-05-17T23:38:48.651634+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2306.16388","created_at":"2026-05-17T23:38:48.651634+00:00"},{"alias_kind":"pith_short_12","alias_value":"BWKHUD37M6BS","created_at":"2026-05-18T12:33:33.725879+00:00"},{"alias_kind":"pith_short_16","alias_value":"BWKHUD37M6BSRM6F","created_at":"2026-05-18T12:33:33.725879+00:00"},{"alias_kind":"pith_short_8","alias_value":"BWKHUD37","created_at":"2026-05-18T12:33:33.725879+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":23,"internal_anchor_count":23,"sample":[{"citing_arxiv_id":"2509.21267","citing_title":"Task-Dependent Evaluation of LLM Output Homogenization: A Taxonomy-Guided Framework","ref_index":6,"is_internal_anchor":true},{"citing_arxiv_id":"2510.22628","citing_title":"Sentra-Guard: A Real-Time Multilingual Defense Against Adversarial LLM Prompts","ref_index":20,"is_internal_anchor":true},{"citing_arxiv_id":"2512.05024","citing_title":"Model-Free Assessment of Simulator Fidelity via Quantile Curves","ref_index":9,"is_internal_anchor":true},{"citing_arxiv_id":"2402.05070","citing_title":"A Roadmap to Pluralistic Alignment","ref_index":264,"is_internal_anchor":true},{"citing_arxiv_id":"2603.11974","citing_title":"Normative Common Ground Replication (NormCoRe): Replication-by-Translation for Studying Norms in Multi-Agent AI","ref_index":19,"is_internal_anchor":true},{"citing_arxiv_id":"2605.10310","citing_title":"Positive Alignment: Artificial Intelligence for Human Flourishing","ref_index":50,"is_internal_anchor":true},{"citing_arxiv_id":"2605.13334","citing_title":"LLM-Based Persuasion Enables Guardrail Override in Frontier LLMs","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2604.03493","citing_title":"Cultural Authenticity: Comparing LLM Cultural Representations to Native Human Expectations","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2605.10442","citing_title":"StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs","ref_index":35,"is_internal_anchor":true},{"citing_arxiv_id":"2605.12199","citing_title":"Overtrained, Not Misaligned","ref_index":66,"is_internal_anchor":true},{"citing_arxiv_id":"2605.10843","citing_title":"Training-Free Cultural Alignment of Large Language Models via Persona Disagreement","ref_index":12,"is_internal_anchor":true},{"citing_arxiv_id":"2605.10442","citing_title":"StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs","ref_index":35,"is_internal_anchor":true},{"citing_arxiv_id":"2605.09875","citing_title":"Cross-Family Universality of Behavioral Axes via Anchor-Projected Representations","ref_index":14,"is_internal_anchor":true},{"citing_arxiv_id":"2605.09893","citing_title":"Pseudo-Deliberation in Language Models: When Reasoning Fails to Align Values and Actions","ref_index":13,"is_internal_anchor":true},{"citing_arxiv_id":"2604.23575","citing_title":"The Collapse of Heterogeneity in Silicon Philosophers","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2605.05662","citing_title":"XL-SafetyBench: A Country-Grounded Cross-Cultural Benchmark for LLM Safety and Cultural Sensitivity","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2604.22749","citing_title":"Representational Harms in LLM-Generated Narratives Against Global Majority Nationalities","ref_index":21,"is_internal_anchor":true},{"citing_arxiv_id":"2605.02601","citing_title":"SemEval-2026 Task 7: Everyday Knowledge Across Diverse Languages and Cultures","ref_index":29,"is_internal_anchor":true},{"citing_arxiv_id":"2604.21309","citing_title":"When Bigger Isn't Better: A Comprehensive Fairness Evaluation of Political Bias in Multi-News Summarisation","ref_index":1,"is_internal_anchor":true},{"citing_arxiv_id":"2604.21564","citing_title":"Measuring Opinion Bias and Sycophancy via LLM-based Persuasion","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2604.19301","citing_title":"Large Language Models Exhibit Normative Conformity","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2604.15675","citing_title":"C-Mining: Unsupervised Discovery of Seeds for Cultural Data Synthesis via Geometric Misalignment","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2604.20131","citing_title":"Whose Story Gets Told? Positionality and Bias in LLM Summaries of Life Narratives","ref_index":287,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":2,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/BWKHUD37M6BSRM6FACJIVV42PL","json":"https://pith.science/pith/BWKHUD37M6BSRM6FACJIVV42PL.json","graph_json":"https://pith.science/api/pith-number/BWKHUD37M6BSRM6FACJIVV42PL/graph.json","events_json":"https://pith.science/api/pith-number/BWKHUD37M6BSRM6FACJIVV42PL/events.json","paper":"https://pith.science/paper/BWKHUD37"},"agent_actions":{"view_html":"https://pith.science/pith/BWKHUD37M6BSRM6FACJIVV42PL","download_json":"https://pith.science/pith/BWKHUD37M6BSRM6FACJIVV42PL.json","view_paper":"https://pith.science/paper/BWKHUD37","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2306.16388&json=true","fetch_graph":"https://pith.science/api/pith-number/BWKHUD37M6BSRM6FACJIVV42PL/graph.json","fetch_events":"https://pith.science/api/pith-number/BWKHUD37M6BSRM6FACJIVV42PL/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/BWKHUD37M6BSRM6FACJIVV42PL/action/timestamp_anchor","attest_storage":"https://pith.science/pith/BWKHUD37M6BSRM6FACJIVV42PL/action/storage_attestation","attest_author":"https://pith.science/pith/BWKHUD37M6BSRM6FACJIVV42PL/action/author_attestation","sign_citation":"https://pith.science/pith/BWKHUD37M6BSRM6FACJIVV42PL/action/citation_signature","submit_replication":"https://pith.science/pith/BWKHUD37M6BSRM6FACJIVV42PL/action/replication_record"}},"created_at":"2026-05-17T23:38:48.651634+00:00","updated_at":"2026-05-17T23:38:48.651634+00:00"}