IRLBench provides 1,700 parallel English-Irish exam questions and shows state-of-the-art LLMs perform markedly worse in Irish than in English.
Towards measuring and modeling “culture
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
IRLBench: A Multi-modal, Culturally Grounded, Parallel Irish-English Benchmark for Open-Ended LLM Reasoning Evaluation
IRLBench provides 1,700 parallel English-Irish exam questions and shows state-of-the-art LLMs perform markedly worse in Irish than in English.