Pith. sign in

REVIEW 3 major objections 6 minor 6 cited by

CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fine-tuning open-source VLMs on CultureVerse, a 228,053-question benchmark across 19,682 concepts and 188 countries, substantially raises cultural accuracy, generalizes across continents and to CVQA, and preserves general VQA performance.

desk verdict CultureVerse is a genuinely large multicultural VLM benchmark with a solid disparity study, but the headline fine-tuning gains are confounded by concept leakage; the paper needs a concept-disjoint evaluation before the generalization claims can stand. read the letter →

arxiv 2501.01282 v1 pith:I6AHPNSR submitted 2025-01-02 cs.AI cs.CLcs.CV

Shudong Liu , Yiqiao Jin , Cheng Li , Derek F. Wong , Qingsong Wen , Lichao Sun , Haipeng Chen , Xing Xie
show 1 more author
Jindong Wang
This is my paper · ORCID
classification cs.AIcs.CLcs.CV
keywords culturalunderstandingvision-languagemodelsmultimodalbenchmarkCultureVersebiasfine-tuningvisualquestionansweringcross-culturalgeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper builds CultureVerse, a multimodal benchmark of 228,053 GPT-4o-generated multiple-choice questions covering 19,682 tangible cultural concepts from 188 countries, and uses it to diagnose and then improve the cultural perception of vision-language models (VLMs). Evaluation of 16 open and proprietary models finds a consistent regional hierarchy: all models answer most accurately on concepts from the Americas, worst on African and Asian concepts, and weakest on history and landmarks. The authors fine-tune LLaVA-1.5, Phi-3-Vision, and LLaMA-3.2-Vision for one epoch on CultureVerse to produce CultureVLM, and report large gains on the benchmark, performance comparable to closed-source models, and generalization to other continents and to the independently collected CVQA dataset (7% average improvement). They also report that this fine-tuning leaves ScienceQA and TextVQA scores essentially unchanged, so cultural knowledge can be added without catastrophic forgetting. The claim that would matter most if true is that a single epoch of synthetic-data fine-tuning produces genuine cross-cultural and cross-dataset cultural understanding rather than memorization of the training format.

What carries the argument

The machinery is CultureVerse plus a full fine-tuning recipe. CultureVerse is built by a three-stage pipeline: GPT-4o extracts country-specific tangible concepts from Wikipedia under 15 predefined categories; images are scraped from Google Images (five per concept, first image held out for the test set); and GPT-4o generates three multiple-choice question types—image recognition, cultural knowledge, and scene reasoning—along with explanatory reasoning traces. Human annotators validate the test set (with 93–99% per-component annotation accuracy reported), while the training set uses automated GPT-4o quality checks. CultureVLM is then produced by one epoch of full fine-tuning of LLaVA-1.5, Phi-3-Vision, or LLaMA-3.2-Vision on the 196,673-sample training split, keeping hyperparameters close to the official recipes. The reasoning traces in the training data are the mechanism that lets the model answer new questions in the same style, and the continent- and category-sliced training runs are the mechanism by which the paper tests whether cultural knowledge transfers.

What would settle it

Train a CultureVLM on the training split, then evaluate it on a held-out set of cultural concepts that were never used during fine-tuning (concept-disjoint, not just image-disjoint) and on question templates GPT-4o did not produce; if the accuracy gain over the base model largely disappears, the generalization claim is refuted. The same check can be run on the current test set by replacing its questions with independently written human questions about the same images.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the cultural weakness of open-source VLMs is largely a training-data problem rather than a fixed architecture limitation. After one epoch of supervised fine-tuning on CultureVerse, all three tested base models—LLaVA-1.5 (7B and 13B), Phi-3-Vision 4B, and LLaMA-3.2-Vision 11B—jump to average CultureVerse accuracies above 90%, from baselines in the mid-50s to high-70s, reaching the level of the best proprietary models. The improvement is not confined to in-distribution examples: models fine-tuned on one continent's data improve on other continents, models fine-tuned on one concept category improve on other categories, and a model fine-tuned on the full dataset gains about 7% on CVQA, a human-annotated cultural VQA benchmark collected independently. The paper also establishes a descriptive fact about cultural bias: all 16 evaluated models show the same ordering of performance by region, strongest for the Americas, then Europe and Oceania, weakest for Asia and Africa, with historical figures and landmarks the hardest category.

Load-bearing premise

The load-bearing premise is that the test set, whose concepts are drawn from the same 19,682-concept pool used for training, measures generalization; if the model simply memorizes concept-answer associations and GPT-4o's question style, the reported CultureVerse gains are inflated, leaving only the 7% CVQA gain as a clean cross-dataset signal.

Editorial extensions

If this is right

  • A single epoch of fine-tuning on CultureVerse lifts open-source VLMs' cultural accuracy to roughly the level of closed-source models, so cultural awareness can be added without retraining a model from scratch.
  • Fine-tuning on one continent or category improves performance on other continents and categories, with the in-distribution diagonal highest, suggesting shared structure in cultural knowledge across regions.
  • The 7% average gain on CVQA, an independently collected and human-annotated cultural VQA benchmark, indicates that the benefit is not limited to GPT-4o's own question style.
  • Scores on ScienceQA and TextVQA stay roughly flat after fine-tuning, showing that adding culture-specific multiple-choice data does not necessarily erase general visual commonsense.
  • The consistent regional ranking across all 16 models implies that any future culturally aware VLM should report performance broken down by region, because a single global accuracy number hides a large Western bias.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A strict test the paper does not run is concept-disjoint evaluation: holding out entire concepts (not just images) from training; on the current split, the test set's concepts are a subset of the training pool, so some of the on-benchmark gain could be memorization of concept–answer pairs rather than cultural understanding.
  • Because culture is proxied by country and language, the pipeline may scale well to nations with rich Wikipedia and Google Images coverage but may underrepresent oral or low-resource cultures, where community-generated data would likely be needed.
  • The data-size ablation suggests most of the benefit appears even at 5% of the training data, implying that the binding constraint is concept coverage rather than sample count and that a much cheaper dataset could be built.
  • If the reasoning-trace training is what drives the improvement, then the same recipe should reduce hallucination in stepwise cultural explanations on other benchmarks—a testable prediction, since the paper shows stepwise prompting hurts base models while CultureVLM was trained to produce such traces.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper introduces CultureVerse, a multimodal benchmark built from 19,682 cultural concepts across 188 countries, with three multiple-choice question types (image recognition, cultural knowledge, scene reasoning) and over 228k samples; a human-validated subset of 31,382 samples serves as the test set. The authors evaluate 16 open-source and proprietary VLMs, documenting markedly lower accuracy on African and Asian content than on Western content. They then fine-tune several open-source models on CultureVerse (referred to as CultureVLM), reporting large accuracy increases on the CultureVerse test set, some cross-continent and cross-category transfer, and a +7% average improvement on the external CVQA benchmark, with minimal degradation on ScienceQA and TextVQA. The paper's central claim is that fine-tuning on CultureVerse yields cross-cultural, cross-continent, and cross-dataset generalization.

Significance. If the fine-tuning results were validly evaluated, this work would be a significant step toward scalable cultural benchmarking for VLMs, given its order-of-magnitude scale advantage, transparent pipeline, and the demonstration that a lightweight fine-tuning can shift model performance on culturally diverse content. The paper's strengths include the human QA process (93-99% per check item), the breadth of the evaluation (16 models), the direct comparison of task types and regions, and the external CVQA sanity check. However, the contribution's central claim rests on a train/test split that may allow concept-level memorization, so the significance is conditional on a re-analysis that separates memorization from transfer.

major comments (3)
  1. [Sections 5.1 and 3.1] The claimed fine-tuning generalization is confounded by the train/test concept overlap. Section 5.1 states that the test set consists of 'more common cultural concepts from the entire dataset' while 'the training set includes all cultural concepts,' and Section 3.1 assigns the first image of each concept to the test and the remaining four to training. Consequently every test concept appears with four same-concept training examples, and the test questions are produced by the same GPT-4o templates (Appendix E) as the training questions. The large in-distribution gains in Tables 3 and 4 (e.g., LLaVA-1.5-7B from 58.75 to 91.70 average) can therefore be attributed to memorizing concept-answer associations rather than to transferable cultural understanding. The abstract's claim of 'cross-cultural, cross-continent, and cross-dataset generalization' is not supported by the CultureVerse test alone; the authors must provide a concept-disjoint split or restrict the generalization claims to the held-out continent/category and CVQA experiments.
  2. [Section 5.4, Tables 5 and 6] The continent- and category-holdout analyses are cleaner than the full fine-tuning evaluation because the training subset excludes the tested continents/categories, but they are still reported for a single base model (LLaVA-1.5-7B) and share the GPT-4o question-generation distribution with the test set. The paper should report these results for at least one additional base model and with multiple fine-tuning seeds, and should verify that the test concepts of the held-out groups do not appear in the training subset. As written, the evidence for cross-continent transfer is suggestive but not sufficient to support the strength of the claim.
  3. [Section 5.4, Figure 6] The external CVQA experiment is the only evidence for generalization to an independently constructed benchmark, and it shows a meaningful +7% average improvement. However, it is a single model (LLaVA-1.5-7B) with a single fine-tuning run, and the authors do not report whether the CVQA concepts overlap with the CultureVerse training concepts. Adding per-country breakdowns with confidence intervals and checking concept overlap would substantially strengthen this result; without that, the cross-dataset generalization claim rests on limited evidence.
minor comments (6)
  1. [Section 3.3] The sentence 'With over 98% of the evaluation set samples correctly annotated by the automated process' is confusing because the preceding text describes human annotation; Table 9 reports human verification accuracy, not automated accuracy, so this wording should be clarified.
  2. [Table 7] The TextVQA rows for LLaVA-1.5 are labeled 'FT' while other rows are not; this appears to be a typo and should be corrected.
  3. [Figure 2(a)] The figure contains stray numbers ('1', '2 3') that are likely layout artifacts from the text labels; these should be removed.
  4. [Sections 5.2 and 5.3] Several occurrences of 'LLaV A' (with a space) should be corrected to 'LLaVA' throughout the text.
  5. [Appendix E] The mapping from prompts to the three question types is implicit; adding a table or explicit references would help readers connect each prompt to Section 3.2.
  6. [Section 6] The limitation text says 'we use languages as proxies for cultural boundaries,' but the dataset construction is country-based; the role of language in the pipeline should be clarified.

Circularity Check

1 steps flagged · score 6.0 of 10

The main fine-tuning evaluation is partly circular: every test concept also appears in the training set, so large in-dataset gains may reflect concept memorization rather than cultural generalization; the external CVQA and held-out continent/category results remain independent.

  1. fitted input called prediction [Section 5.1 (Data Split); supporting counts in Table 1 and Appendix A]
    "We select more common cultural concepts from the entire dataset for the test set, which underwent manual quality checks, while the training set includes all cultural concepts. We ensured that the images in the training and test sets did not overlap to prevent data leakage."

    The test set's 11,085 concepts are a subset of the 19,682-concept pool used for fine-tuning, so every test concept has four same-concept training images and GPT-4o-generated questions from the same prompt templates (Appendix E). Only images are disjoint. A one-epoch fine-tune on roughly 197k such examples can memorize concept–answer associations and option patterns for essentially all test concepts, so the reported post-fine-tuning improvements (e.g., LLaVA-1.5-7B from 58.75 to 91.70 in Tables 3/4) are partly forced by construction rather than evidence of generalization to unseen cultural concepts. The external CVQA result (+7% average) and the held-out-continent/category ablations in Figure 5 do measure transfer and are not circular, which is why the circularity is partial.

full rationale

The paper's central training claim is evaluated on a test set whose concepts are included in the training set by design, so the headline in-dataset gains are not a clean measure of learned cultural generalization. This is a genuine but partial circularity: the CVQA cross-dataset evaluation and the continent/category hold-out experiments provide independent evidence of transfer, and the ScienceQA/TextVQA forgetting checks are external. I did not find load-bearing self-citation or a uniqueness argument; prior work by overlapping authors is used only for background and limitations. The score of 6 reflects that one key 'prediction' — fine-tuned accuracy on the CultureVerse test set — reduces partly to memorization of concepts and question style seen during training.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the reliability of GPT-4o-generated data and on a train/test split that shares concept names. These are the main uncharged premises; no new physical or theoretical entities are introduced.

free parameters (3)
  • Frequency-based threshold = 2
    Only entities appearing at least twice in a country's Wikipedia documents are retained (Section 3.1). This threshold is chosen by hand, not derived, and affects concept coverage and dataset composition.
  • Images per concept = 5
    Five images are scraped per concept; the first is used for the test set and the other four for training (Section 3.1). This split is arbitrary and affects the train/test distribution.
  • Fine-tuning learning rates = 2e-5, 4e-5, 1e-5 per model
    Learning rates for LLaVA, Phi-3, and LLaMA are chosen from official scripts (Section 5.1, Appendix B.1). Standard hyperparameters, not fitted to data.
assumptions (4)
  • domain assumption Cultural understanding can be approximated by multiple-choice accuracy on three question types over tangible, visually presentable concepts.
    The dataset restricts itself to tangible cultural concepts and measures understanding by GPT-4o-generated MCQ accuracy; stated in Sections 3 and 6.
  • domain assumption GPT-4o can reliably extract cultural concepts, generate correct questions, and filter abstract entities.
    The whole pipeline delegates to GPT-4o (Sections 3.1-3.2); human validation on the test set reaches 93-99% but the training set is unvalidated.
  • domain assumption Wikipedia and Google Images provide representative coverage of world cultures.
    The pipeline depends on these sources (Section 3.1); bias toward well-documented cultures is partially acknowledged in the Conclusion.
  • ad hoc to paper A test set whose concept names appear in the fine-tuning set is a valid held-out evaluation.
    Section 5.1 states the training set includes all cultural concepts and the test set is a subset, with only images disjoint; the paper implicitly assumes this does not bias the results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries." pith.science (2026). https://pith.science/paper/I6AHPNSR

@misc{pith2026250101282,
  author       = {Pith},
  title        = {Pith review of: CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I6AHPNSR}},
  note         = {Machine review of arXiv:2501.01282}
}
read the original abstract

Vision-language models (VLMs) have advanced human-AI interaction but struggle with cultural understanding, often misinterpreting symbols, gestures, and artifacts due to biases in predominantly Western-centric training data. In this paper, we construct CultureVerse, a large-scale multimodal benchmark covering 19, 682 cultural concepts, 188 countries/regions, 15 cultural concepts, and 3 question types, with the aim of characterizing and improving VLMs' multicultural understanding capabilities. Then, we propose CultureVLM, a series of VLMs fine-tuned on our dataset to achieve significant performance improvement in cultural understanding. Our evaluation of 16 models reveals significant disparities, with a stronger performance in Western concepts and weaker results in African and Asian contexts. Fine-tuning on our CultureVerse enhances cultural perception, demonstrating cross-cultural, cross-continent, and cross-dataset generalization without sacrificing performance on models' general VLM benchmarks. We further present insights on cultural generalization and forgetting. We hope that this work could lay the foundation for more equitable and culturally aware multimodal AI systems.

Figures

Figures reproduced from arXiv: 2501.01282 by the authors.

Figure 1
Figure 1. Our pipeline to build CultureVerse and CultureVLM. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of CultureVerse. In total, there are over 220k instances and 19k cultural concepts for training and [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Accuracy of different models on three tasks (upper), five regions (middle), and three categories of concepts [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Results and analysis of CultureVLM by fine-tuning on our CultureVerse. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Generalization and Robustness. Left: Performance of CultureVLM (y-axis) evaluated across data from different continents (x-axis). CultureVLM achieves the highest performance for in-distribution settings, while still demonstrating strong generalizability for out-of-doma…
Figure 6
Figure 6. Figure 6: Results on CVQA (Romero et al., 2024). Question: Which cultural synthesis is reflected in the architecture and artistic features of the structure shown in the image? Options: (A) Native American and Colonial Spanish (B) Italian Renaissance and Gothic (C) Byzantine and …
Figure 8
Figure 8. Figure 8: Performance of open-source and proprietary models on general VQA datasets. [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Impact of different decoding temperatures on performance [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Failing to See or Failing to Know? Attributing Errors in Vision-Language Models

    cs.CV 2026-07 conditional novelty 6.5 of 10

    VLM wrong answers in knowledge-intensive visual QA can be attributed to four decision points—recognition, visual evidence, answer success, factual access—with different pre-generation representations best predicting e...

  2. When Cultures Move: Measuring and Improving Multicultural Text-to-Video Generation

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    Parallel, role-specialized prompt agents improve cultural relevance in text-to-video generation, with a new cross-cultural benchmark showing the largest gains for location cues.

  3. Evaluation of Cultural Competence of Vision-Language Models

    cs.CV 2025-05 conditional novelty 6.0 of 10

    The paper proposes five theory-informed frameworks from visual cultural studies for evaluating cultural competence in vision-language models.

  4. SARA: Selective and Adaptive Retrieval-augmented Generation with Context Compression

    cs.CL 2025-07 conditional novelty 5.0 of 10

    SARA combines short natural-language snippets with vector-compressed summaries of the remaining retrieved documents, improving RAG answer quality under 512/1024-token context budgets.

  5. Uncovering Cultural Representation Disparities in Vision-Language Models

    cs.CV 2025-05 conditional novelty 4.0 of 10

    Across six VLMs and 211 countries, country-recognition accuracy varies widely and errors concentrate on a few overpredicted countries (USA, India, Brazil), not on a uniform Western bias.

  6. Domain Specific Benchmarks for Evaluating Multimodal Large Language Models

    cs.LG 2025-06 conditional novelty 3.0 of 10

    A review paper that organizes domain-specific MLLM benchmarks into an eight-discipline taxonomy, with summary tables and performance highlights.

Reference graph

Works this paper leans on

34 extracted references · 4 canonical work pages · cited by 6 Pith papers

  1. [1]

    Persianllama: Towards building first persian large language model

    Mohammad Amin Abbasi, Arash Ghafouri, Mahdi Firouzmandi, Hassan Naderi, and Behrouz Minaei Bidgoli. Persianllama: Towards building first persian large language model. arXiv:2312.15713,

    arXiv

  2. [4]

    Pixtral 12b

    Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Baptiste Bout, Devendra Chaplot, Jessica Chudnovsky, Diogo Costa, Baudouin De Monicault, Saurabh Garg, Theophile Gervet, et al. Pixtral 12b. arXiv:2410.07073,

    arXiv

  3. [6]

    From local concepts to universals: Evaluating the multicultural understanding of vision-language models

    Mehar Bhatia, Sahithya Ravi, Aditya Chinchure, Eunjeong Hwang, and Vered Shwartz. From local concepts to universals: Evaluating the multicultural understanding of vision-language models. arXiv:2407.00263,

    arXiv

  4. [7]

    Enhancing Content Moderation with Culturally-Aware Models

    Alex J Chan, José Luis Redondo García, Fabrizio Silvestri, Colm O’Donnel, and Konstantina Palla. Harmonizing global voices: Culturally-aware models for enhanced content moderation. arXiv:2312.02401,

    work page Pith review arXiv

  5. [8]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. arXiv:2312.14238,

    arXiv

  6. [9]

    Culturalbench: a robust, diverse and challenging benchmark on measuring the (lack of) cultural knowledge of llms

    Yu Ying Chiu, Liwei Jiang, Bill Yuchen Lin, Chan Young Park, Shuyue Stella Li, Sahithya Ravi, Mehar Bhatia, Maria Antoniak, Yulia Tsvetkov, Vered Shwartz, et al. Culturalbench: a robust, diverse and challenging benchmark on measuring the (lack of) cultural knowledge of llms. arXiv:2410.02677,

    arXiv

  7. [10]

    Deconstructing the ethics of large language models from long-standing issues to new-emerging dilemmas.arXiv:2406.05392,

    Chengyuan Deng, Yiqun Duan, Xin Jin, Heng Chang, Yijun Tian, Han Liu, Henry Peng Zou, Yiqiao Jin, Yijia Xiao, Yichen Wang, et al. Deconstructing the ethics of large language models from long-standing issues to new-emerging dilemmas.arXiv:2406.05392,

    arXiv

  8. [11]

    Massively multi-cultural knowledge acquisition & lm benchmarking

    Yi Fung, Ruining Zhao, Jae Doo, Chenkai Sun, and Heng Ji. Massively multi-cultural knowledge acquisition & lm benchmarking. arXiv:2402.09369,

    arXiv

Show all 34 references
  1. [12]

    Normsage: Multi-lingual multi-cultural norm discovery from conversations on-the-fly

    Yi R Fung, Tuhin Chakraborty, Hao Guo, Owen Rambow, Smaranda Muresan, and Heng Ji. Normsage: Multi-lingual multi-cultural norm discovery from conversations on-the-fly. arXiv:2210.08604,

    arXiv

  2. [13]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv:2410.21276,

    arXiv

  3. [14]

    Better to ask in english: Cross-lingual evaluation of large language models for healthcare queries

    Yiqiao Jin, Mohit Chandra, Gaurav Verma, Yibo Hu, Munmun De Choudhury, and Srijan Kumar. Better to ask in english: Cross-lingual evaluation of large language models for healthcare queries. In Web Conference, pages 2627–2638, 2024a. Yiqiao Jin, Minje Choi, Gaurav Verma, Jindong...

    arXiv

  4. [15]

    Llava-onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv:2408.03326, 2024a. 11 Cheng Li, Mengzhou Chen, Jindong Wang, Sunayana Sitaram, and Xing Xie. Culturell...

    arXiv

  5. [16]

    Are multilingual llms culturally-diverse reasoners? an investigation into multicultural proverbs and sayings

    Chen Cecilia Liu, Fajri Koto, Timothy Baldwin, and Iryna Gurevych. Are multilingual llms culturally-diverse reasoners? an investigation into multicultural proverbs and sayings. arXiv:2309.08591, 2023a. Fangyu Liu, Emanuele Bugliarello, Edoardo Maria Ponti, Siva Reddy, Nigel Co...

    arXiv

  6. [19]

    The 2012 stein rokkan lecture: Three decades of popu list radical right parties in western europe: so what? In The Populist Radical Right, pages 545–558

    Cas Mudde. The 2012 stein rokkan lecture: Three decades of popu list radical right parties in western europe: so what? In The Populist Radical Right, pages 545–558. Routledge,

    2012

  7. [21]

    Readme++: Benchmarking multilingual language models for multi-domain readability assessment

    Tarek Naous, Michael J Ryan, Anton Lavrouk, Mohit Chandra, and Wei Xu. Readme++: Benchmarking multilingual language models for multi-domain readability assessment. In EMNLP, 2023a. Tarek Naous, Michael J Ryan, and Wei Xu. Having beer after prayer? measuring cultural bias in la...

    arXiv

  8. [22]

    Extracting cultural commonsense knowledge at scale

    Tuan-Phong Nguyen, Simon Razniewski, Aparna Varde, and Gerhard Weikum. Extracting cultural commonsense knowledge at scale. In Web Conference, pages 1907–1917, 2023a. Xuan-Phi Nguyen, Wenxuan Zhang, Xin Li, Mahani Aljunied, Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Se...

    1907 arXiv

  9. [23]

    Gpt-4o, 2024a

    OpenAI. Gpt-4o, 2024a. OpenAI. Gpt-4v, 2024b. Kunat Pipatanakul, Phatrasek Jirabovonvisut, Potsawee Manakul, Sittipong Sripaisarnmongkol, Ruangsak Patomwong, Pathomporn Chokchainant, and Kasima Tharnpipitchai. Typhoon: Thai large language models. arXiv:2312.13951,

    arXiv

  10. [24]

    Normad: A benchmark for measuring the cultural adaptability of large language models

    12 Abhinav Rao, Akhila Yerukola, Vishwa Shah, Katharina Reinecke, and Maarten Sap. Normad: A benchmark for measuring the cultural adaptability of large language models. arXiv:2404.12464,

    arXiv

  11. [25]

    Cvqa: Culturally-diverse multilingual visual question answering benchmark

    David Romero, Chenyang Lyu, Haryo Akbarianto Wibowo, Teresa Lynn, Injy Hamed, Aditya Nanda Kishore, Aishik Mandal, Alina Dragonetti, Artem Abzaliev, Atnafu Lambebo Tonja, et al. Cvqa: Culturally-diverse multilingual visual question answering benchmark. arXiv:2406.05967,

    arXiv

  12. [27]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: a family of highly capable multimodal models. arXiv:2312.11805,

    arXiv

  13. [28]

    A comprehensive survey of hallucination mitigation techniques in large language models

    SM Tonmoy, SM Zaman, Vinija Jain, Anku Rani, Vipula Rawte, Aman Chadha, and Amitava Das. A comprehensive survey of hallucination mitigation techniques in large language models. arXiv:2401.01313,

    arXiv

  14. [29]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv:2409.12191,

    arXiv

  15. [30]

    Not all countries celebrate thanksgiving: On the cultural dominance in large language models

    Wenxuan Wang, Wenxiang Jiao, Jingyuan Huang, Ruyi Dai, Jen-tse Huang, Zhaopeng Tu, and Michael R Lyu. Not all countries celebrate thanksgiving: On the cultural dominance in large language models. arXiv:2310.12481,

    arXiv

  16. [31]

    Worldcuisines: A massive-scale benchmark for multilingual and multicultural visual question answering on global cuisines

    Genta Indra Winata, Frederikus Hudi, Patrick Amadeus Irawan, David Anugraha, Rifki Afina Putri, Yutong Wang, Adam Nohejl, Ubaidillah Ariq Prathama, Nedjma Ousidhoum, Afifa Amriani, et al. Worldcuisines: A massive-scale benchmark for multilingual and multicultural visual questi...

    arXiv

  17. [32]

    Minicpm-v: A gpt-4v level mllm on your phone

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level mllm on your phone. arXiv:2408.01800,

    arXiv

  18. [34]

    14 B.2 Detailed Main Results

    13 Appendix CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries Contents A Details of the CultureVerse Dataset 14 B Experimental Details and Results 14 B.1 Experiment Setup . . . . . . . . . . . . . . . . . . . . . ....

    2023

  19. [1996]

    Hallucination of multimodal large language models: A survey

    10 Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. Hallucination of multimodal large language models: A survey. arXiv:2404.18930,

    arXiv

  20. [2014]

    Worldvaluesbench: A large-scale benchmark dataset for multi-cultural value awareness of language models

    Wenlong Zhao, Debanjan Mondal, Niket Tandon, Danica Dillion, Kurt Gray, and Yuling Gu. Worldvaluesbench: A large-scale benchmark dataset for multi-cultural value awareness of language models. arXiv:2404.16308,

    arXiv

  21. [2016]

    Blend: A benchmark for llms on everyday knowledge in diverse cultures and languages

    Junho Myung, Nayeon Lee, Yi Zhou, Jiho Jin, Rifki Afina Putri, Dimosthenis Antypas, Hsuvas Borkakoty, Eunsu Kim, Carla Perez-Almendros, Abinew Ali Ayele, et al. Blend: A benchmark for llms on everyday knowledge in diverse cultures and languages. arXiv:2406.09948,

    arXiv

  22. [2019]

    Llama 3.2

    Meta. Llama 3.2. https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices,

    2024

  23. [2021]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In CVPR, pages 26296–26306, 2024a. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. NeurIPS, 36, 2024b. Shudong Liu, Xuebo Liu, Derek F ...

    arXiv

  24. [2022]

    Culturebank: An online community-driven knowledge base towards culturally aware language technologies

    Weiyan Shi, Ryan Li, Yutong Zhang, Caleb Ziems, Raya Horesh, Rogério Abreu de Paula, Diyi Yang, et al. Culturebank: An online community-driven knowledge base towards culturally aware language technologies. arXiv:2404.15238,

    arXiv

  25. [2023]

    Phi-3 technical report: A highly capable language model locally on your phone

    Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv:2404.14219,

    arXiv

  26. [2024]

    culture

    Muhammad Farid Adilazuarda, Sagnik Mukherjee, Pradhyumna Lavania, Siddhant Singh, Ashutosh Dwivedi, Alham Fikri Aji, Jacki O’Neill, Ashutosh Modi, and Monojit Choudhury. Towards measuring and modeling" culture" in llms: A survey.arXiv:2403.15412,

    arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.