A randomized audit of five LLM APIs finds verified survey-country metadata improves held-out response forecasts, while disclosing that a country label was randomly assigned does not reliably attenuate its influence.
Whose Norms? Disentangling Cultural and Personal Alignment in Large Language Models
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
Large language models are increasingly used for social decision-making situations that require balancing cultural norms with personal preferences. For example, a user preferring honesty might ask whether to correct a coworker publicly when local norms favor indirect feedback. Yet existing research studies cultural alignment and personalization largely separately. We introduce PACT, the Personal-Preference and Cultural-Norm Trade-off framework, which evaluates whether models choose to follow a cultural norm or allow personal preferences. We find that LLMs vary in how rigidly they enforce cultural norms, with behavior shifted more by country context (7.8%) than age (1%) and gender (0.7%) and shifting non-uniformly after instruction tuning. Furthermore, our five-country human study on PACT shows that culture-following in humans is mainly driven by scenario country, with the lowest agreement when participants judge their own cultural contexts, showing within-culture pluralism. Finally, human-LLM alignment experiments show that models can match majority choices, but fail to capture response distributions and uncertainty (with best correlations reaching only 0.24). Together, these findings motivate alignment evaluations that go beyond majority to capture cultural pluralism and disagreement in social judgment.
fields
cs.AI 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Signal or Spurious Cue? A Randomized Audit of Survey-Country Metadata in LLM Social Inference
A randomized audit of five LLM APIs finds verified survey-country metadata improves held-out response forecasts, while disclosing that a country label was randomly assigned does not reliably attenuate its influence.