REVIEW 7 cited by
Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) have demonstrated multilingual capabilities, yet they are mostly English-centric due to the imbalanced training corpora. While prior works have leveraged this bias to enhance multilingual performance through translation, they have been largely limited to natural language processing (NLP) tasks. In this work, we extend the evaluation to real-world user queries and non-English-centric LLMs, offering a broader examination of multilingual performance. Our key contribution lies in demonstrating that while translation into English can boost the performance of English-centric LLMs on NLP tasks, it is not universally optimal. For culture-related tasks that need deep language understanding, prompting in the native language proves more effective as it better captures the nuances of culture and language. Our experiments expose varied behaviors across LLMs and tasks in the multilingual context, underscoring the need for a more comprehensive approach to multilingual evaluation. Therefore, we call for greater efforts in developing and evaluating LLMs that go beyond English-centric paradigms.
Forward citations
Cited by 7 Pith papers
-
The Emergence of Abstract Thought in Large Language Models Beyond Any Language
Across 20 open LLMs, shared multilingual neurons grow in number and per-neuron importance over release generations, which the authors interpret as evidence of language-agnostic abstract thought and use to guide neuron...
-
MMATH: A Multilingual Benchmark for Mathematical Reasoning
A new multilingual math benchmark shows that reasoning models often respond in the wrong language, and English-reasoning training improves both accuracy and language consistency.
-
MPO: Multilingual Safety Alignment via Reward Gap Optimization
MPO reduces jailbreak success in multilingual LLMs by regressing target-language reward gaps onto the English reward gap, outperforming DPO and related methods while preserving utility.
-
How and Where to Translate? The Impact of Translation Strategies in Cross-lingual LLM Prompting
For multilingual RAG intent classification, the best translation strategy depends on the model and language; translating instructions into the user's language helps some models, while making the model answer in low-re...
-
Making Sense of Korean Sentences: A Comprehensive Evaluation of LLMs through KoSEnd Dataset
A new Korean benchmark, KoSEnd, shows LLMs have limited grasp of Korean sentence endings, and warning them about potentially missing endings improves their choices.
-
Information Loss in LLMs' Multilingual Translation: The Role of Training Data, Language Proximity, and Language Family
Round-trip translation quality in GPT-4 and Llama 2 is jointly shaped by training data volume and language distance from English, with orthographic, phylogenetic, syntactic, and geographic distances as the strongest p...
-
Cross-lingual Aspect-Based Sentiment Analysis: A Survey on Tasks, Approaches, and Challenges
A comprehensive survey of cross-lingual aspect-based sentiment analysis that catalogs tasks, datasets, modeling paradigms, and cross-lingual transfer techniques, and identifies research gaps.
Discussion (0). Sign in to comment.