Pith. sign in

REVIEW 2 cited by

Translation Errors Significantly Impact Low-Resource Languages in Cross-Lingual Learning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.02080 v1 pith:FBO7UB72 submitted 2024-02-03 cs.CL

Ashish Sunil Agrawal , Barah Fazili , Preethi Jyothi This is my paper
classification cs.CL
keywords languagestargettranslationcross-lingualerrorsenglishexisthuman-translated
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Popular benchmarks (e.g., XNLI) used to evaluate cross-lingual language understanding consist of parallel versions of English evaluation sets in multiple target languages created with the help of professional translators. When creating such parallel data, it is critical to ensure high-quality translations for all target languages for an accurate characterization of cross-lingual transfer. In this work, we find that translation inconsistencies do exist and interestingly they disproportionally impact low-resource languages in XNLI. To identify such inconsistencies, we propose measuring the gap in performance between zero-shot evaluations on the human-translated and machine-translated target text across multiple target languages; relatively large gaps are indicative of translation errors. We also corroborate that translation errors exist for two target languages, namely Hindi and Urdu, by doing a manual reannotation of human-translated test instances in these two languages and finding poor agreement with the original English labels these instances were supposed to inherit.

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. How and Where to Translate? The Impact of Translation Strategies in Cross-lingual LLM Prompting

    cs.CL 2025-07 conditional novelty 5.0 of 10

    For multilingual RAG intent classification, the best translation strategy depends on the model and language; translating instructions into the user's language helps some models, while making the model answer in low-re...

  2. The Multilingual Divide and Its Impact on Global AI Safety

    cs.AI 2025-05 conditional novelty 3.0 of 10

    The language gap in AI models creates safety disparities across languages, and closing it requires funding multilingual datasets, transparency, and research.

Pith tools