arxiv: 2510.24081 · v2 · ★pith:R7CQ6SGUnew · submitted 2025-10-28 · 💻 cs.CL
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
show 372 more authors
read the original abstract
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we present Global PIQA, a participatory commonsense reasoning benchmark for over 100 languages, constructed by hand by over 350 researchers from over 65 countries around the world. The 141 language varieties in Global PIQA cover five continents, 19 language families, and 24 writing systems. In the non-parallel split of Global PIQA, over 50% of examples reference local foods, customs, traditions, or other culturally-specific elements. In the parallel split, we translate more "culturally agnostic" commonsense reasoning questions into 131 language varieties, for direct cross-lingual comparisons. In both splits, all examples have been verified by native speakers of the languages. We find that state-of-the-art LLMs perform well on Global PIQA in aggregate, but they exhibit weaker performance in lower-resource languages (e.g. up to a 68% accuracy gap between languages in the parallel split). Global PIQA highlights that in many languages and cultures, everyday knowledge remains an area for improvement in LLMs, alongside more widely-discussed capabilities such as complex reasoning and expert knowledge. Beyond its uses for LLM evaluation, Global PIQA provides a glimpse into the wide diversity of cultures in which human language is embedded.
This paper has not been read by Pith yet.
Forward citations
Cited by 4 Pith papers
Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.
-
Multilingual Knowledge Transfer under Data Constraints via Lexical Interventions
cs.CL 2026-05 unverdicted novelty 6.0
LINK improves cross-lingual knowledge transfer via lexical substitutions in English pretraining data, yielding notable downstream gains and up to 2x training speedup across eight languages and five model sizes.
-
Maistros: A Greek Large Language Model Adapted Through Knowledge Distillation From Large Reasoning Models
cs.CL 2026-05 unverdicted novelty 6.0
Maistros 8B is a new state-of-the-art open-weights Greek LLM built via knowledge distillation from large reasoning models on the CulturaQA dataset.
-
What properties of reasoning supervision are associated with improved downstream model quality?
cs.AI 2026-05 unverdicted novelty 5.0
Intrinsic data metrics predict reasoning dataset utility for model fine-tuning, with different predictors working best for smaller versus larger models.
-
A Survey of Text and Speech Resources for Hausa and Fongbe: Availability, Quality, and Gaps for NLP Development
cs.CL 2026-04 unverdicted novelty 4.0
A survey catalogs text and speech resources for Hausa and Fongbe, documenting sizes, domains, licensing, and gaps including limited Fongbe text diversity and missing Hausa speech corpora.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.