Commercial AI chatbots reach over 90% multiple-choice accuracy on recent news facts but lose 11-17% in free response and drop to 19-70% on subtle false-premise questions, with retrieval failures causing most errors and clear Anglophone bias.
u ksel, A., K \
6 Pith papers cite this work, alongside 8 external citations. Polarity classification is still indexing.
6
Pith papers citing it
8
external citations · external index
citation-role summary
background 2
citation-polarity summary
fields
cs.CL 6roles
background 2polarities
background 2representative citing papers
A new 7,044-question native Sinhala exam benchmark shows the best LLM at 67.65% accuracy, with large drops on culturally specific subjects.
Adding borrowed options to multiple-choice questions lowers source-key agreement without improving measurement validity, particularly for negative-stem questions.
MÖVE presents a new German-language benchmark evaluating 39 LLMs on performance and governance criteria using ten public-administration datasets.
No tested retriever beats simple character TF-IDF for Turkish document QA, while model choice drives accuracy, with Trendyol-8B strongest at 75%.
A large-scale analysis of multilingual benchmarks finds English overrepresentation, weak alignment of translated benchmarks with human preferences, and stronger alignment for localized benchmarks like CMMLU.
citing papers explorer
-
Evaluating Commercial AI Chatbots as News Intermediaries
Commercial AI chatbots reach over 90% multiple-choice accuracy on recent news facts but lose 11-17% in free response and drop to 19-70% on subtle false-premise questions, with retrieval failures causing most errors and clear Anglophone bias.
-
SinhalaMMLU: A Comprehensive Benchmark for Evaluating Multitask Language Understanding in Sinhala
A new 7,044-question native Sinhala exam benchmark shows the best LLM at 67.65% accuracy, with large drops on culturally specific subjects.
-
Turkish MMLU Pro: Traceable Option Augmentation and Its Validity Limits in Turkish Multiple-Choice Evaluation
Adding borrowed options to multiple-choice questions lowers source-key agreement without improving measurement validity, particularly for negative-stem questions.
-
M\"OVE: A Holistic LLM Benchmark for the German Public Sector
MÖVE presents a new German-language benchmark evaluating 39 LLMs on performance and governance criteria using ten public-administration datasets.
-
Evaluating Open-Weight LLMs for Turkish Domain Documents Under Retrieval and Hardware Constraints
No tested retriever beats simple character TF-IDF for Turkish document QA, while model choice drives accuracy, with Trendyol-8B strongest at 75%.
-
The Bitter Lesson Learned from 2,000+ Multilingual Benchmarks
A large-scale analysis of multilingual benchmarks finds English overrepresentation, weak alignment of translated benchmarks with human preferences, and stronger alignment for localized benchmarks like CMMLU.