pith. sign in

arxiv: 2604.09945 · v1 · submitted 2026-04-10 · 💻 cs.CV · cs.AI· cs.CL

Cross-Cultural Value Awareness in Large Vision-Language Models

Phillip Howard , Xin Su , Kathleen C. Fraser This is my paper

Pith reviewed 2026-05-10 16:44 UTC · model grok-4.3

classification 💻 cs.CV cs.AIcs.CL
keywords large vision-language modelscultural valuesMoral Foundations Theoryvalue judgmentscounterfactual imagesAI fairness
0
0 comments X

The pith

Large vision-language models adjust their judgments of a person's moral, ethical, and political values when the same individual appears in different cultural contexts.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests whether five popular LVLMs recognize and reflect cultural differences in the values they attribute to people shown in images. It does so by holding the depicted person constant while varying only the surrounding cultural signals, then measuring shifts in model outputs. The analysis applies Moral Foundations Theory to categorize the values, tracks lexical patterns in the responses, and checks how closely the generated values align with the cultural cues present. A reader would care because these models are increasingly used to interpret and respond to people from varied backgrounds, so any embedded cultural sensitivity shapes fairness in real applications.

Core claim

Counterfactual image sets that place the same person in different cultural settings produce measurable changes in the value judgments generated by LVLMs; these changes are detectable through Moral Foundations Theory categories, lexical analysis of the text, and direct comparison of outputs across the matched images.

What carries the argument

Counterfactual image sets that isolate cultural context by depicting the identical person across varied cultural backgrounds, allowing direct comparison of model-generated value statements.

Load-bearing premise

Changes in model outputs across the image variants reflect genuine awareness of cultural value differences rather than superficial visual patterns or prompt effects.

What would settle it

If the five LVLMs produce statistically identical value judgments and lexical profiles for every cultural variant of the same person, the claim of cultural sensitivity would not hold.

Figures

Figures reproduced from arXiv: 2604.09945 by Kathleen C. Fraser, Phillip Howard, Xin Su.

Figure 1
Figure 1. Figure 1: We prompt LVLMs with counterfactuals depicting different cultural contexts (here: (a) Christian church, (b) [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Frequency of MFT foundation value assignments by model and religious context [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Frequency of MFT foundation value assignments by model and religious context [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Frequency of MFT foundation value assignments by model and socioeconomic context [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Frequency of MFT foundation value assignments by model and national context [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗
read the original abstract

The rapid adoption of large vision-language models (LVLMs) in recent years has been accompanied by growing fairness concerns due to their propensity to reinforce harmful societal stereotypes. While significant attention has been paid to such fairness concerns in the context of social biases, relatively little prior work has examined the presence of stereotypes in LVLMs related to cultural contexts such as religion, nationality, and socioeconomic status. In this work, we aim to narrow this gap by investigating how cultural contexts depicted in images influence the judgments LVLMs make about a person's moral, ethical, and political values. We conduct a multi-dimensional analysis of such value judgments in five popular LVLMs using counterfactual image sets, which depict the same person across different cultural contexts. Our evaluation framework diagnoses LVLM awareness of cultural value differences through the use of Moral Foundations Theory, lexical analyses, and the sensitivity of generated values to depicted cultural contexts.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The manuscript proposes an evaluation framework for assessing cross-cultural value awareness in large vision-language models (LVLMs). It examines how images depicting different cultural contexts (religion, nationality, socioeconomic status) influence the models' judgments on a person's moral, ethical, and political values. The analysis employs counterfactual image sets showing the same individual across contexts, applied to five popular LVLMs, and diagnoses awareness using Moral Foundations Theory, lexical analyses, and output sensitivity to contexts.

Significance. Should the empirical results confirm that LVLMs exhibit differential value judgments based on cultural contexts in a manner consistent with Moral Foundations Theory and not attributable to visual artifacts, this would represent a meaningful advance in AI fairness research. It highlights potential cultural biases in multimodal models beyond traditional social stereotypes and provides a structured approach using psychological theory for evaluation. This could guide the development of more culturally aware and equitable vision-language systems.

major comments (1)
  1. [Evaluation Framework and Counterfactual Image Sets] The central claim that the framework diagnoses LVLM awareness of cultural value differences (abstract) relies on the assumption that value judgment shifts across counterfactual image sets reflect internalized cultural understanding. Depicting the same person in different cultural contexts necessarily alters low-level visual elements such as clothing, backgrounds, objects, and lighting. The manuscript provides no explicit controls (e.g., style-matched variants, feature ablation, or non-cultural visual perturbations) to isolate cultural effects from pattern matching on superficial cues. This is load-bearing for the sensitivity analysis and lexical/MFT-based diagnoses.
minor comments (2)
  1. [Abstract] The abstract describes the framework and approach but omits any quantitative results, statistical tests, or key findings from the five LVLMs. Including a brief summary of main outcomes would improve informativeness.
  2. Clarify the exact implementation of lexical analyses and Moral Foundations Theory mappings, including any specific dictionaries, questionnaires, or prompt templates used for value extraction.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their thoughtful and constructive review, as well as their positive assessment of the work's potential significance for AI fairness research. We address the single major comment below and describe the revisions we will make to strengthen the manuscript.

read point-by-point responses

  1. Referee: [Evaluation Framework and Counterfactual Image Sets] The central claim that the framework diagnoses LVLM awareness of cultural value differences (abstract) relies on the assumption that value judgment shifts across counterfactual image sets reflect internalized cultural understanding. Depicting the same person in different cultural contexts necessarily alters low-level visual elements such as clothing, backgrounds, objects, and lighting. The manuscript provides no explicit controls (e.g., style-matched variants, feature ablation, or non-cultural visual perturbations) to isolate cultural effects from pattern matching on superficial cues. This is load-bearing for the sensitivity analysis and lexical/MFT-based diagnoses.

    Authors: We appreciate the referee's identification of this methodological point. Our counterfactual sets are generated by holding the individual's core visual identity fixed (face, pose, expression, and skin tone) while varying only the cultural indicators (attire, religious or national symbols, background architecture, and socioeconomic cues). This design isolates the effect of cultural context from changes in personal identity. Nevertheless, we acknowledge that the manuscript does not include explicit controls such as non-cultural visual perturbations (e.g., lighting or style changes without cultural content) or feature ablations to rule out reliance on low-level patterns. We will add these controls in the revision: (1) a set of non-cultural visual perturbations applied to the same base images, (2) style-matched variants that preserve cultural elements while altering artistic style, and (3) ablation of specific visual regions (e.g., clothing vs. background). These additions will allow us to quantify how much of the observed value shifts persist after removing superficial visual cues, thereby strengthening the claim that the models exhibit sensitivity to cultural value differences. The MFT and lexical analyses will be re-run on the controlled outputs to confirm the diagnoses remain robust. revision: yes

Circularity Check

0 steps flagged

No circularity: purely empirical evaluation without derivations or self-referential reductions

full rationale

The paper conducts an empirical study of LVLM value judgments using counterfactual image sets, Moral Foundations Theory, and lexical analyses. No mathematical derivations, equations, fitted parameters, or predictions are present that could reduce to inputs by construction. Claims rest on observed output sensitivities to depicted contexts, supported by external psychological frameworks rather than self-citation chains or ansatzes. The methodology is self-contained and falsifiable via replication on the described image sets and analysis techniques.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The analysis rests on the assumption that Moral Foundations Theory provides a suitable cross-cultural lens for value judgments and that model output changes indicate cultural awareness.

axioms (1)
  • domain assumption Moral Foundations Theory provides a valid and comprehensive framework for categorizing moral, ethical, and political values across cultures
    Invoked to structure the diagnosis of LVLM value judgments and cultural differences.

pith-pipeline@v0.9.0 · 5450 in / 1138 out tokens · 51445 ms · 2026-05-10T16:44:15.833284+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

6 extracted references · 6 canonical work pages

  1. [1]

    InAdvances in experi- mental social psychology, volume 47, pages 55–130

    Moral foundations theory: The pragmatic va- lidity of moral pluralism. InAdvances in experi- mental social psychology, volume 47, pages 55–130. Elsevier. Jesse Graham, Jonathan Haidt, and Brian A Nosek

    work page

  2. [2]

    Jonathan Haidt and Craig Joseph

    Liberals and conservatives rely on different sets of moral foundations.Journal of personality and social psychology, 96(5):1029. Jonathan Haidt and Craig Joseph. 2004. Intuitive ethics: How innately prepared intuitions generate culturally variable virtues.Daedalus, 133(4):55–66. Siobhan Mackenzie Hall, Fernanda Gonçalves Abrantes, Hanwen Zhu, Grace Sodunk...

    work page 2004

  3. [3]

    In Ninth workshop on speech and language processing for assistive technologies (SLPAT-2022), pages 58– 65

    Applying the stereotype content model to as- sess disability bias in popular pre-trained NLP mod- els underlying AI-based assistive technologies. In Ninth workshop on speech and language processing for assistive technologies (SLPAT-2022), pages 58– 65. Phillip Howard, Kathleen C Fraser, Anahita Bhiwandi- walla, and Svetlana Kiritchenko. 2025. Uncovering b...

    work page arXiv 2022

  4. [4]

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee

    Religious affiliation and conceptions of the moral domain.Social Cognition, 39(1):139–165. Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024. Llava- next: Improved reasoning, ocr, and world knowledge. Tamim Mobayed. 2019. Religious differences across moral foundations. https://blogs. 5 lse.ac.uk/religionglobalso...

    work page 2024

  5. [5]

    Chahat Raj, Anjishnu Mukherjee, Aylin Caliskan, An- tonios Anastasopoulos, and Ziwei Zhu

    Comprehensive stereotype content dictionaries using a semi-automated method.European Journal of Social Psychology, 51(1):178–196. Chahat Raj, Anjishnu Mukherjee, Aylin Caliskan, An- tonios Anastasopoulos, and Ziwei Zhu. 2024. Bias- dora: Exploring hidden biased associations in vision- language models. InFindings of the Association for Computational Lingui...

    work page 2024

  6. [6]

    individualizing

    is a widely used social psychological frame- work which proposes that human morality is de- scribed by five (or in recent version, six) fundamen- tal moral foundations. The foundations are, briefly: Care/Harm(concern for the suffering of others), Fairness/Reciprocity(encompassing the concepts of justice and proportionality),Loyalty/Betrayal (loyalty to on...

    work page 2009