Pith. sign in

REVIEW 4 major objections 6 minor 35 references

Toward Socially Aware Vision-Language Models: Evaluating Cultural Competence Through Multimodal Story Generation

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Vision-language models change children's stories based on a child's nationality, but the standard auto-scorer praises the weakest model, so human judgment is required.

desk verdict Useful dataset and a real caution about CLIPScore, but the image-text confound undercuts the causal claim about cultural cue processing. read the letter →

arxiv 2508.16762 v1 pith:DTFDUJEB submitted 2025-08-22 cs.CL cs.CY

Arka Mukherjee , Shreya Ghosh This is my paper
classification cs.CLcs.CY
keywords culturalcompetencevision-languagemodelsmultimodalstorygenerationalignmentCLIPScorebiasHofstededimensionsWorldValuesSurveycross-modalevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to measure VLM cultural competence in an open-ended generative setting: writing a children's story for a child of a given nationality from a culturally relevant image. Across five open models and 42 countries it finds that nationality cues change stories measurably—names, family terms, foods, and landmarks vary far more between countries than across repeated rewrites. It also finds that this adaptation is shallow and uneven: correlations with Hofstede and World Values Survey distances are weak or inverted for several models, and all models lean on majority-ethnic names. Its sharpest claim is that CLIPScore, the automated image-text similarity metric, is architecturally biased, ranking the model humans judge worst as the best cross-modal performer. The authors conclude that cultural competence in VLMs should be assessed with generative tasks and human judgment, and they flag that their own test is limited to English children's stories, explicit nationality labels, and Western survey frameworks.

What carries the argument

The mechanism is a nationality perturbation built into a fixed story prompt—'Write a children's story about {concept} for a/an {identity} kid in English based on the image'—paired with a scraped, human-filtered image specific to each (concept, country). The measurement chain then runs through three instruments: word-level edit ratio and TF-IDF/BLEU for lexical adaptation; Kendall's tau between story similarity and Hofstede's six dimensions or WVS's 259 dimensions for value alignment; and CLIPScore-based within- versus cross-nationality recall for cross-modal coherence, checked against human ratings on ten dimensions. The key named mechanism is cross-modal recall: whether a story embedding re

What would settle it

Hold the input image fixed and vary only the nationality word in the prompt across the paper's 42 countries; if the between-nationality lexical variance drops to the within-nationality level, the reported effect was driven by the country-specific images rather than by cultural identity cues. A second check: recompute the cross-modal recall with a non-CLIP vision-language scorer; if SmolVLM2's advantage over Gemma3 disappears, the CLIPScore architectural bias is confirmed.

Watch

Extended reading notes

Core claim

The central claim is that cultural identity cues embedded in both prompt and image do change what a VLM generates: ANOVA F-values from 1540 to 8707 with p below 1e-48 show between-country lexical variance far exceeding within-country variance, and TF-IDF reveals country-specific names, kin terms, foods, and landmarks. At the same time, the stories do not reliably track cultural-value distances: correlations with Hofstede's dimensions are often negative for the Gemma models, Qwen shows the best but still weak positive alignment, and WVS correlations hover near zero. Cross-modal retrieval can pick the correct country's story about a third of the time in the best model (28.7% within-nationality

Load-bearing premise

The load-bearing assumption is that the nationality label in the prompt drives the measured differences; in the data collection (Section 3.1), the image and the label change together, so the tests cannot separate cultural adaptation from image-conditioned generation.

Editorial extensions

If this is right

  • Cultural competence in VLMs should be evaluated on open-ended generation, not only object recognition or multiple-choice VQA, because generative outputs reveal adaptation and stereotype patterns that recognition tasks miss.
  • Automated image-text similarity scores such as CLIPScore can rank models in the reverse order of culturally judged quality, so benchmarks built on them need human calibration or a different scorer.
  • No evaluated architecture is consistently culturally competent: some models show inverse alignment, where culturally distant countries receive more similar stories, and all favor majority-ethnic names.
  • Hofstede and WVS frameworks capture different aspects of narrative culture; only 31% of countries keep the same correlation sign across frameworks, so a single survey-based yardstick understates cultural competence.
  • Because models score higher on safety-oriented dimensions than on authenticity-oriented ones, current VLMs appear to prioritize non-offensiveness over culturally specific representation in generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the images and the nationality label change together in the dataset, the paper's large F-statistics likely overstate how much of the adaptation is driven by the cultural identity cue; a same-image, many-nationalities control would separate image-driven from cue-driven generation.
  • The CLIPScore inversion suggests a practical test for other multimodal cultural benchmarks: swap the vision encoder for a different contrastive model and see whether model rankings flip; if they do, the benchmark is measuring the encoder's cultural associations rather than the model's competence.
  • The tiered vocabulary pattern—names, then family terms, then foods and practices—could be turned into an automated proxy for cultural depth that tracks human ratings, allowing future studies to screen large corpora before human review.
  • Since all stories were generated in English, the apparent cultural adaptation may partly be a stereotype-recognition effect; generating in the target countries' languages or with code-switching would test whether models have deeper cultural competence or only surface markers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper proposes a benchmark and evaluation framework for cultural competence in vision-language models (VLMs), assessed through open-ended multimodal story generation. The authors construct 1,470 prompts by combining story concepts with 42 countries, scrape a culturally relevant image for each prompt, and ask five open-source VLMs to generate children's stories (73,500 stories total). Analyses include ANOVAs on lexical variance across vs. within nationalities (F-values 1540–8707, all p<1e-48), TF-IDF extraction of country-specific vocabulary, BLEU-based story similarity correlations against Hofstede Cultural Dimensions and World Values Survey, CLIPScore-based cross-modal retrieval (within-nationality Recall@1 up to 28.7% vs. cross-nationality 0.2%), and a human judgment study of 250 stories. The authors report that VLMs show significant nationality-driven lexical adaptation, that culture-specific vocabulary is rich but stereotypes are present, that HCD/WVS correlations are weak or inverse for several models, that CLIPScore contradicts human judgments for SmolVLM2, and that human evaluation is therefore indispensable. The dataset and code are released publicly.

Significance. If the central interpretation were fully supported, the paper would make a useful contribution: it moves VLM cultural evaluation from recognition/VQA tasks to an open-ended generative task, provides a large multilingual-country corpus with human ratings, and identifies a plausible CLIPScore bias when ranking model-generated cultural content. The scale (five models, 42 countries, 73.5k stories) and the authors' willingness to report negative results (weak WVS correlations, inverse HCD alignment in Gemma models) are strengths, as are the public release and the explicit limitations section. The main weakness is that the experiment does not isolate the cultural identity cue in the text from the culturally specific image content, which undermines the paper's headline causal claim (RQ1) and parts of the cross-modal recall analysis. With that confound addressed or the claims reframed, the benchmark and its qualitative findings would be a solid basis for a journal paper.

major comments (4)
  1. [Sec. 3.1 and Sec. 4.1 / Table 2] The paper's stated design is that the framework 'maintain[s] constant visual inputs across 5 VLMs and 42 countries' (Sec. 1, RQ1), but Sec. 3.1 collects a separate image per (concept, country) using the search query 'concept for a/an identity kid' and selects images for 'perceived cultural relevance.' Thus nationality is confounded with image content: Canadian prompts are paired with maple/hockey images, Egyptian prompts with pyramids/Nile images, etc. The ANOVA in Sec. 4.1 compares across-nationality lexical variance with within-nationality sampling variance while both text nationality and image content change together. The enormous F-statistics are real, but they cannot be attributed to the model processing the textual nationality cue; they may reflect image-conditioned story generation. This is load-bearing for RQ1 and the paper's main claim. Please re-run with identical or content-co
  2. [Sec. 4.4 / Table 4] The claim that 'story representations are reliably distinguishable ... when grounded in identical visual content' is not supported by the data, since the visual content is not identical across nationalities. The within-nationality vs. cross-nationality CLIP recall gap (28.7% vs. 0.2%) is partly by construction: the query image already contains country markers (landmarks, clothing, flags), and stories generated from that image tend to mention those markers. High within-nationality recall therefore does not prove that the VLM adapted to the textual nationality cue. Please report retrieval results with image content controlled (e.g., the same image paired with different nationalities) or restrict the Sec. 4.4/5.1 conclusions to 'culturally distinct outputs are detectable through visual-semantic similarity when both the image and the text prompt carry the same country markers.'
  3. [Sec. 4.5 / Table 3] The human-judgment study is load-bearing for the conclusion that CLIPScore is architecturally biased, but the reporting is currently insufficient. With only two annotators and 250 stories, no inter-annotator agreement (Cohen's kappa, Krippendorff's alpha, or per-item raw scores) is reported. Claude Sonnet 4 is described as an additional evaluator, but it is unclear whether its ratings are merged into Table 3 or reported separately. Without reliability evidence and a precise aggregation rule, the 'human judgments contradict CLIPScore' finding is not yet established. Please report per-evaluator scores, agreement statistics, and the averaging procedure.
  4. [Sec. 3.3.2 and Sec. 4.3] The lexical-diversity section states 'we only measure model outputs for culturally-appropriate lexical variations' but does not define how 'culturally-appropriate' is determined, and the Word Edit Ratio / BLEU pipeline is only sketched. More importantly, the HCD/WVS correlation analyses in Sec. 4.3 report a small positive median for Qwen (+0.02) and negative medians for Gemma models (-0.08 to -0.10), and the WVS has data for only 58 countries while the study covers 42; the paper does not state how missing WVS countries were handled. Please clarify the metric details and the missing-data handling so the null and inverse results can be interpreted.
minor comments (6)
  1. [Sec. 3.2] Typo: 'These arew Gemma3 4B' should be 'These are Gemma3 4B'; also 'downstram' in Sec. 3.3.2.
  2. [Table 1 caption] 'xorrelated' appears to be a typo for 'correlated'. Model names are inconsistent ('Gemma3' vs. 'Gemma 3'); please unify.
  3. [Fig. 2 caption] The caption contains '\uni0000002a/...' escape-like sequences that appear to be a rendering artifact; these should be cleaned before final submission.
  4. [Sec. 3.3.1 / Ref. [18]] The text refers to the '2015 version' of Hofstede's dimensions, but the cited reference is a 2011 article; please clarify which dataset version was actually used and cite it precisely.
  5. [Table 4] The 'Random Baseline' row would benefit from showing all six Recall values explicitly (R@1/R@5/R@10 for both within and cross nationality); the current alignment is ambiguous. Also, the table header 'R@10' is used but the text only defines R@1; please define.
  6. [Appendix figures] Figures 6–9 are said to cover the 'remaining four models,' but the caption of Figure 9 again says 'Qwen 2.5 VL 7B,' which would duplicate Figure 7. Please check the captions and assign each model unique figure.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation; the main issue is a stimulus confound (nationality-specific images), not a self-referential loop.

full rationale

The paper's core measurements are anchored to external benchmarks: HCD and WVS are fixed survey frameworks, CLIPScore is a pretrained metric, and human ratings are independent judgments. No parameter is fitted to the target conclusion, and no result is obtained by substituting the conclusion into an equation. The closest circularity-adjacent concern is that Sec. 1 claims 'maintaining constant visual inputs across 5 VLMs and 42 countries' while Sec. 3.1 collects a distinct culturally-relevant image per nationality using the query 'concept for a/an identity kid.' This means textual nationality and image content vary together, so the Sec. 4.1 ANOVA and the Sec. 4.4 within-nationality vs. cross-nationality CLIP recall may partly reflect image-conditioned differences rather than cultural adaptation to the text cue. However, this is a causal confound and a validity threat, not a derivation loop: the reported F-statistics, correlations, and recall values are empirical measurements that could have turned out differently. The paper itself partially acknowledges the issue in Sec. 7 ('Our framework also doesn't consider the biases in Google's Search algorithm for sampling culturally relevant images'). Therefore the circularity score is low; the confound belongs in correctness risk rather than in a finding of circularity.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper's central claims rest on several externally sourced measurement frameworks and on the cultural representativeness of scraped images. No novel theoretical entities are introduced; the main 'free' choices are experimental design choices (country list, concept list, sampling temperatures, aggregation rule) rather than fitted parameters.

assumptions (5)
  • domain assumption Hofstede's Cultural Dimensions (2015) provide a valid measure of cultural distance relevant to narrative content.
    Used as ground truth in Sec 3.3.1 for correlation with story similarity; the paper itself questions this in Limitations.
  • domain assumption World Values Survey Wave 7 provides a valid 259-dimensional cultural representation.
    Same use as HCD; also questioned in Limitations.
  • domain assumption Google Images search results, filtered by one author, yield culturally representative images for each (concept, country) pair.
    Section 3.1; the images are the visual cultural cues and the basis of cross-modal recall.
  • domain assumption Open-ended children's story generation is a valid downstream task for eliciting cultural competence.
    Section 3.1; the central task choice.
  • domain assumption The 42 selected countries, chosen by English-speaking population, adequately represent global cultural diversity.
    Section 3.1; selection by English-speaking population may bias toward Anglophone cultures.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Toward Socially Aware Vision-Language Models: Evaluating Cultural Competence Through Multimodal Story Generation." pith.science (2026). https://pith.science/paper/DTFDUJEB

@misc{pith2026250816762,
  author       = {Pith},
  title        = {Pith review of: Toward Socially Aware Vision-Language Models: Evaluating Cultural Competence Through Multimodal Story Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DTFDUJEB}},
  note         = {Machine review of arXiv:2508.16762}
}
read the original abstract

As Vision-Language Models (VLMs) achieve widespread deployment across diverse cultural contexts, ensuring their cultural competence becomes critical for responsible AI systems. While prior work has evaluated cultural awareness in text-only models and VLM object recognition tasks, no research has systematically assessed how VLMs adapt outputs when cultural identity cues are embedded in both textual prompts and visual inputs during generative tasks. We present the first comprehensive evaluation of VLM cultural competence through multimodal story generation, developing a novel multimodal framework that perturbs cultural identity and evaluates 5 contemporary VLMs on a downstream task: story generation. Our analysis reveals significant cultural adaptation capabilities, with rich culturally-specific vocabulary spanning names, familial terms, and geographic markers. However, we uncover concerning limitations: cultural competence varies dramatically across architectures, some models exhibit inverse cultural alignment, and automated metrics show architectural bias contradicting human assessments. Cross-modal evaluation shows that culturally distinct outputs are indeed detectable through visual-semantic similarity (28.7% within-nationality vs. 0.2% cross-nationality recall), yet visual-cultural understanding remains limited. In essence, we establish the promise and challenges of cultural competence in multimodal AI. We publicly release our codebase and data: https://github.com/ArkaMukherjee0/mmCultural

Figures

Figures reproduced from arXiv: 2508.16762 by the authors.

Figure 1
Figure 1. Overview of our dataset creation framework. The frame [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Lexical variance comparison across different vision [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Box plots showing the distribution of Kendall’s tau correlation coefficients between model-generated stories and cultural survey [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: BLEU-based story similarity correlations with cultural dimensions for [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Geographic distribution of correlations between CLIP similarity and cultural distance measures of [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: BLEU-based story similarity correlations with cultural dimensions for [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: BLEU-based story similarity correlations with cultural dimensions for [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: BLEU-based story similarity correlations with cultural dimensions for [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: BLEU-based story similarity correlations with cultural dimensions for [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Geographic distribution of correlations between CLIP similarity and cultural distance measures of [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Geographic distribution of correlations between CLIP similarity and cultural distance measures of [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Geographic distribution of correlations between CLIP similarity and cultural distance measures of [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: Geographic distribution of correlations between [PITH_FULL_IMAGE:figures/full_fig_p016_13.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 27 canonical work pages

  1. [1]

    To- wards measuring and modeling “culture” in LLMs: A sur- vey

    Muhammad Farid Adilazuarda, Sagnik Mukherjee, Prad- hyumna Lavania, Siddhant Shivdutt Singh, Alham Fikri Aji, Jacki O’Neill, Ashutosh Modi, and Monojit Choudhury. To- wards measuring and modeling “culture” in LLMs: A sur- vey. In Proceedings of the 2024 Conference on Empiri- cal Methods in Natural Language Processing, pages 15763– 15784, Miami, Florida, U...

    work page 2024

  2. [2]

    Investigating cultural alignment of large language models

    Badr AlKhamissi, Muhammad ElNokrashy, Mai AlKhamissi, and Mona Diab. Investigating cultural alignment of large language models. arXiv preprint arXiv:2402.13231, 2024. 1, 2

    arXiv 2024

  3. [3]

    Probing pre-trained language models for cross-cultural dif- ferences in values

    Arnav Arora, Lucie-Aim ´ee Kaffee, and Isabelle Augenstein. Probing pre-trained language models for cross-cultural dif- ferences in values. arXiv preprint arXiv:2203.13722, 2022. 1, 2

    arXiv 2022

  4. [4]

    Probing pre-trained language models for cross-cultural dif- ferences in values

    Arnav Arora, Lucie-aim ´ee Kaffee, and Isabelle Augenstein. Probing pre-trained language models for cross-cultural dif- ferences in values. In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP) , pages 114–130, Dubrovnik, Croatia, 2023. Association for Com- putational Linguistics. 2

    work page 2023

  5. [5]

    Qwen2.5-vl technical report, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhao- hai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025. 3

    work page 2025

  6. [6]

    Venkatesh Babu, and Danish Pruthi

    Abhipsa Basu, R. Venkatesh Babu, and Danish Pruthi. In- specting the geographical representativeness of images from text-to-image models. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision (ICCV) , pages 5136–5147, 2023. 1

    work page 2023

  7. [7]

    From local concepts to univer- sals: Evaluating the multicultural understanding of vision- language models

    Mehar Bhatia, Sahithya Ravi, Aditya Chinchure, EunJeong Hwang, and Vered Shwartz. From local concepts to univer- sals: Evaluating the multicultural understanding of vision- language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages 6763–6782, Miami, Florida, USA, 2024. Association for Computational L...

    work page 2024

  8. [8]

    Extrinsic evaluation of cultural competence in large language models

    Shaily Bhatt and Fernando Diaz. Extrinsic evaluation of cultural competence in large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 16055–16074, Miami, Florida, USA, 2024. As- sociation for Computational Linguistics. 2

    work page 2024

Show all 35 references
  1. [9]

    Language (technology) is power: A critical sur- vey of “bias” in NLP

    Su Lin Blodgett, Solon Barocas, Hal Daum ´e III, and Hanna Wallach. Language (technology) is power: A critical sur- vey of “bias” in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 5454–5476, Online, 2020. Association for...

    2020

  2. [10]

    Assessing cross-cultural alignment between chatgpt and human societies: An empirical study

    Yong Cao, Li Zhou, Seolhwa Lee, Laura Cabello, Min Chen, and Daniel Hershcovich. Assessing cross-cultural alignment between chatgpt and human societies: An empirical study. arXiv preprint arXiv:2303.17466, 2023. 2

    2023 arXiv

  3. [11]

    MaXM: Towards multilingual visual question an- swering

    Soravit Changpinyo, Linting Xue, Michal Yarom, Ashish Thapliyal, Idan Szpektor, Julien Amelot, Xi Chen, and Radu Soricut. MaXM: Towards multilingual visual question an- swering. In Findings of the Association for Computational Linguistics: EMNLP 2023 , pages 2667–2682, Singapore,

    2023

  4. [12]

    The SAGE handbook of intercultural competence

    Darla K Deardorff. The SAGE handbook of intercultural competence. Sage Publications, 2009. 1, 2

    2009

  5. [13]

    Towards measuring the representation of subjec- tive global opinions in language models

    Esin Durmus, Karina Nguyen, Thomas I Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. Towards measuring the representation of subjec- tive global opinions in language models. arXiv preprint arXiv:23...

    2023 arXiv

  6. [14]

    ”i wouldn’t say offensive but...”: Disability-centered perspectives on large language models

    Vinitha Gadiraju, Shaun Kane, Sunipa Dev, Alex Taylor, Ding Wang, Remi Denton, and Robin Brewer. ”i wouldn’t say offensive but...”: Disability-centered perspectives on large language models. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparenc...

    2023

  7. [15]

    World values survey: Round seven - country-pooled datafile version 5.0, 2022

    Christian Haerpfer, Ronald Inglehart, Alejandro Moreno, Christian Welzel, Kseniya Kizilova, Jaime Diez-Medrano, Marta Lagos, Pippa Norris, Eduard Ponarin, and Bjorn Pu- ranen. World values survey: Round seven - country-pooled datafile version 5.0, 2022. Version 5.0. 1, 2

    2022

  8. [16]

    Challenges and strategies in cross- cultural NLP

    Daniel Hershcovich, Stella Frank, Heather Lent, Miryam de Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Piqueras, Ilias Chalkidis, Ruixi- ang Cui, Constanza Fierro, Katerina Margatina, Phillip Rust, and Anders Søgaard. Challenges and strategies ...

    2022

  9. [17]

    Clipscore: A reference-free evaluation met- ric for image captioning, 2022

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation met- ric for image captioning, 2022. 4, 7

    2022

  10. [18]

    Dimensionalizing cultures: The hofstede model in context

    Geert Hofstede. Dimensionalizing cultures: The hofstede model in context. Online Readings in Psychology and Cul- ture, 2(1), 2011. 1, 2

    2011

  11. [19]

    Hannah Rose Kirk, Alexander Whitefield, Paul R ¨ottger, An- drew Bean, Katerina Margatina, Juan Ciro, Rafael Mos- quera, Max Bartolo, Adina Williams, He He, et al. The prism alignment project: What participatory, representative and individualised human feedback reveals about t...

    2024 arXiv

  12. [20]

    Visually 9 grounded reasoning across languages and cultures

    Fangyu Liu, Emanuele Bugliarello, Edoardo Maria Ponti, Siva Reddy, Nigel Collier, and Desmond Elliott. Visually 9 grounded reasoning across languages and cultures. In Pro- ceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10467–10485, O...

    2021

  13. [21]

    Wong, Qing- song Wen, Lichao Sun, Haipeng Chen, Xing Xie, and Jin- dong Wang

    Shudong Liu, Yiqiao Jin, Cheng Li, Derek F. Wong, Qing- song Wen, Lichao Sun, Haipeng Chen, Xing Xie, and Jin- dong Wang. Culturevlm: Characterizing and improving cul- tural understanding of vision-language models for over 100 countries, 2025. 2

    2025

  14. [22]

    Smolvlm: Re- defining small and efficient multimodal models, 2025

    Andr ´es Marafioti, Orr Zohar, Miquel Farr ´e, Merve Noyan, Elie Bakouch, Pedro Cuenca, Cyril Zakka, Loubna Ben Allal, Anton Lozhkov, Nouamane Tazi, Vaibhav Srivastav, Joshua Lochner, Hugo Larcher, Mathieu Morlon, Lewis Tun- stall, Leandro von Werra, and Thomas Wolf. Smolvlm: ...

    2025

  15. [23]

    Cultural alignment in large language models: An explanatory analysis based on hofst- ede’s cultural dimensions

    Reem I Masoud, Ziquan Liu, Martin Ferianc, Philip Tre- leaven, and Miguel Rodrigues. Cultural alignment in large language models: An explanatory analysis based on hofst- ede’s cultural dimensions. arXiv preprint arXiv:2309.12342,

    arXiv

  16. [24]

    Assessing demographic bias in named entity recognition, 2020

    Shubhanshu Mishra, Sijun He, and Luca Belli. Assessing demographic bias in named entity recognition, 2020. 1

    2020

  17. [25]

    Benchmarking vision language models for cultural understanding

    Shravan Nayak, Kanishk Jain, Rabiul Awal, Siva Reddy, Sjoerd Van Steenkiste, Lisa Anne Hendricks, Karolina Stanczak, and Aishwarya Agrawal. Benchmarking vision language models for cultural understanding. In Proceed- ings of the 2024 Conference on Empirical Methods in Natu- ral...

    2024

  18. [26]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics , page 311–318, USA, 2002. Association for Computational Linguistics. 3

    2002

  19. [27]

    Knowledge of cultural moral norms in large language models

    Aida Ramezani and Yang Xu. Knowledge of cultural moral norms in large language models. arXiv preprint arXiv:2306.01857, 2023. 2

    2023 arXiv

  20. [28]

    Normad: A benchmark for measuring the cultural adaptability of large language mod- els

    Abhinav Rao, Akhila Yerukola, Vishwa Shah, Katharina Reinecke, and Maarten Sap. Normad: A benchmark for measuring the cultural adaptability of large language mod- els. CoRR, 2024. 2

    2024

  21. [29]

    Cvqa: culturally-diverse multilingual visual question answering benchmark

    David Romero, Chenyang Lyu, Haryo Akbarianto Wibowo, Teresa Lynn, Injy Hamed, Aditya Nanda Kishore, Aishik Mandal, Alina Dragonetti, Artem Abzaliev, Atnafu Lam- bebo Tonja, Bontu Fufa Balcha, Chenxi Whitehouse, Chris- tian Salamea, Dan John Velasco, David Ifeoluwa Adelani, Dav...

    2025

  22. [30]

    Geographical erasure in language generation

    Pola Schw ¨obel, Jacek Golebiowski, Michele Donini, Cedric Archambeau, and Danish Pruthi. Geographical erasure in language generation. In Findings of the Association for Com- putational Linguistics: EMNLP 2023 , pages 12310–12324, Singapore, 2023. Association for Computational...

    2023

  23. [31]

    Cultural bias and cultural alignment of large language mod- els

    Yan Tao, Olga Viberg, Ryan S Baker, and Ren ´e F Kizilcec. Cultural bias and cultural alignment of large language mod- els. PNAS Nexus, 3(9):pgae346, 2024. 2

    2024

  24. [32]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Ta- tiana Matejovicova, Alexandre Ram ´e, Morgane Rivi `ere, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Ca...

    2025

  25. [33]

    Broaden the vision: Geo-diverse visual com- monsense reasoning

    Da Yin, Liunian Harold Li, Ziniu Hu, Nanyun Peng, and Kai- Wei Chang. Broaden the vision: Geo-diverse visual com- monsense reasoning. In Proceedings of the 2021 Confer- ence on Empirical Methods in Natural Language Process- ing, pages 2115–2129, Online and Punta Cana, Dominica...

    2021

  26. [34]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models, 2025

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shen- glong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang, Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang, Haomin Wang, Weiye Xu, Hao Li, Jiahao Wang, Nianchen Deng, Songze L...

    2025

  27. [2023]

    Association for Computational Linguistics. 1, 2

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.