A cross-evaluation setup with native-speaker SME rubrics benchmarks LLMs on underrepresented Arabic dialects and quantifies automated judge bias and cultural-reasoning gaps.
How reliable is multilingual LLM-as-a-judge? In Findings of the Association for Computational Linguistics: EMNLP 2025 , pages 11040–11053, Suzhou, China, 2025
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Benchmarking Frontier LLMs on Arabic Cultural and Sociolinguistic Knowledge: A Cross-Evaluation Framework with Human SME Ground Truth
A cross-evaluation setup with native-speaker SME rubrics benchmarks LLMs on underrepresented Arabic dialects and quantifies automated judge bias and cultural-reasoning gaps.