Pith Number

pith:BWKHUD37

pith:2023:BWKHUD37M6BSRM6FACJIVV42PL

not attested not anchored not stored refs resolved

Towards Measuring the Representation of Subjective Global Opinions in Language Models

Alex Tamkin, Amanda Askell, Anton Bakhtin, Carol Chen, Danny Hernandez, Deep Ganguli, Esin Durmus, Jack Clark, Janel Thamkul, Jared Kaplan, Karina Nguyen, Liane Lovitt, Nicholas Joseph, Nicholas Schiefer, Orowa Sikder, Sam McCandlish, Thomas I. Liao, Zac Hatfield-Dodds

Large language models produce answers that match opinions from the United States and certain European and South American countries more closely than opinions from other nations.

arxiv:2306.16388 v2 · 2023-06-28 · cs.CL · cs.AI

Open paper page JSON Open Graph Bundle Merged state What is a Pith Number?

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

By default, LLM responses tend to be more similar to the opinions of certain populations, such as those from the USA, and some European and South American countries, highlighting the potential for biases.

C2weakest assumption

That the chosen cross-national survey responses serve as an unbiased and representative ground truth for each country's population-level opinions on the selected issues.

C3one line summary

LLMs default to responses more similar to opinions from the USA and some European and South American countries; prompting for a country shifts alignment but can introduce stereotypes, while translation does not reliably match language speakers.

References

97 extracted · 97 resolved · 8 Pith anchors

[1] InProceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society(Virtual Event, USA)(AIES ’21) 2021 · doi:10.1145/3461702.3462624

[2] Subjective natural language problems: Motivations, applications, characterizations, and implications 2011

[3] Probing pre-trained language models for cross-cultural differences in values

[4] URL https://aclanthology.org/2023 2023

[5] A general language assistant as a laboratory for alignment 2021

Formal links

2 machine-checked theorem links

Cited by

23 papers in Pith

Task-Dependent Evaluation of LLM Output Homogenization: A Taxonomy-Guided Framework

Sentra-Guard: A Real-Time Multilingual Defense Against Adversarial LLM Prompts

Model-Free Assessment of Simulator Fidelity via Quantile Curves

A Roadmap to Pluralistic Alignment

Normative Common Ground Replication (NormCoRe): Replication-by-Translation for Studying Norms in Multi-Agent AI

Receipt and verification

First computed	2026-05-17T23:38:48.651547Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

0d947a0f7f678328b3c500928ad79a7adc7f43eb29475951988da37ce5282ffb

Aliases

arxiv: 2306.16388 · arxiv_version: 2306.16388v2 · doi: 10.48550/arxiv.2306.16388 · pith_short_12: BWKHUD37M6BS · pith_short_16: BWKHUD37M6BSRM6F · pith_short_8: BWKHUD37

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/BWKHUD37M6BSRM6FACJIVV42PL \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 0d947a0f7f678328b3c500928ad79a7adc7f43eb29475951988da37ce5282ffb

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "56e02e29c283cdfb8325bad47d002bd7e49cfa011c01c414d24d796899052e7e",
    "cross_cats_sorted": [
      "cs.AI"
    ],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2023-06-28T17:31:53Z",
    "title_canon_sha256": "2a9629f5efb27fae810debc0ee06bc748ce2f2830a71e275ed726aa837d9168a"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2306.16388",
    "kind": "arxiv",
    "version": 2
  }
}