Pith. sign in

REVIEW 3 major objections 7 minor 55 references

Beyond English: The Impact of Prompt Translation Strategies across Languages and Tasks in Multilingual LLMs

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Partially translated prompts beat full translation in 92% of languages

desk verdict A genuinely systematic benchmark of selective pre-translation, but the 'consistently outperforms' claim is inflated by best-of-24 selection without error bars. read the letter →

arxiv 2502.09331 v1 pith:RVWLW5VS submitted 2025-02-13 cs.CL

Itai Mondshine , Tzuf Paz-Argaman , Reut Tsarfaty This is my paper · ORCID
classification cs.CL
keywords multilingualLLMsprompttranslationselectivepre-translationcross-lingualpromptinglow-resourcelanguagesengineeringnaturallanguageinferencenamedentityrecognition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether translating part of a prompt to English before querying a multilingual LLM beats translating all of it or none of it. Treating a prompt as four components — instruction, context, examples, and output — the authors exhaustively test all 24 component-language combinations across 35 languages and four tasks on three models. They find that the best selective configuration beats full pre-translation in 92% of languages and beats direct source-language inference in 90%, with the largest gains for low-resource languages. The paper also shows that translation quality matters, and that choosing which components to translate can neutralize the negative effect of poor translations. If right, this turns component-wise translation into a reliable default strategy for multilingual LLM applications.

What carries the argument

The formal object is a prompt configuration $c = \langle I_{l_i}, X_{l_x}, E_{l_e}, O_{l_o}\rangle$, where each of the four functional components — instruction, context, examples, output — is either in English ($e$) or the source language ($s$). The paper enumerates all 24 valid configurations per language and task and evaluates each one, using point-biserial correlations, association-rule mining with the Apriori algorithm, and performance-gap analysis to isolate which component's language choice drives gains. This decomposition turns the 'translate or not' decision into a fully examinable design space, and the component-level attribution is what makes the practical guidelines possible.

What would settle it

Repeat the comparison on a model with a publicly documented pre-training distribution (for example, a released checkpoint with exact per-language token counts) and choose languages whose GPT-3-based resource class differs from the model's actual token counts. If selective pre-translation no longer beats full pre-translation in a comparable majority of those languages, or if the low-resource advantage reverses, the central claim and its resource-level guidelines would be falsified for that model.

Watch

Extended reading notes

Core claim

The central claim is that selective pre-translation — translating only chosen parts of a prompt into English — consistently outperforms both translating the whole prompt and prompting directly in the source language. Across natural language inference, question answering, named entity recognition, and summarization, the best per-language configuration beats full pre-translation in 92% of tested languages and beats direct inference in 90%, with relative gains exceeding 200% for some low-resource languages such as Malayalam and Telugu. For extractive tasks the source-language context is repeatedly part of the best configuration, while generative tasks such as summarization often benefit from English output even when the final answer is back-translated for scoring. The advantage is attributed largely to selective translation's ability to sidestep translation errors: under full pre-translation the correlation between translation quality and task score is positive and significant, but under the best selective configuration it nearly disappears.

Load-bearing premise

The paper labels languages as high- or low-resource using GPT-3's pre-training token distribution and assumes this proxy holds for the other evaluated models, whose training distributions are not public; if a language is actually well represented in those models' training data, the resource-level guidelines built on this proxy could fail for that model.

Editorial extensions

If this is right

  • For extractive tasks such as question answering and named entity recognition, keeping the context in the source language is a consistent part of the best configuration, especially for low-resource languages.
  • For generative tasks like summarization, instructing the model to output in English often beats source-language output, even though the final text is back-translated before scoring.
  • Selective translation reduces the penalty of low machine-translation quality: the correlation between translation quality and task performance becomes negligible when only the right components are translated.
  • Few-shot prompts with source-language examples outperform zero-shot prompts, with the largest few-shot gains occurring in high-resource languages.
  • The practical guideline is to choose which prompt components to translate based on task type and language resource level, rather than applying a one-size-fits-all rule.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The component-wise search framework could generalize to other prompt choices (example count, component order, output format tokens), where partial transfer may similarly outperform full transfer.
  • The resource-level conclusions hinge on the GPT-3 token distribution used as a proxy; recomputing language classes from each model's actual pre-training data could shift which languages are called low-resource and alter the recommended configurations.
  • A testable decision rule implicit in the translation-quality results is that for a language with poor machine translation, the best configuration should keep the source context and translate only the instruction or output; the paper does not test this rule directly.
  • The near-zero correlation between translation quality and performance under selective translation suggests a ceiling: further improving the translation engine may matter less than choosing the correct components, a hypothesis the paper leaves open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper formalizes a multilingual prompting setup in which a prompt is decomposed into four components—instruction, context, examples, and output—each of which can either remain in the source language or be pre-translated into English. Across 24 configurations, four tasks, six datasets, 35 languages, and several models (GPT-3.5-Turbo, Gemini-1.0-pro, Mixtral-8x7B, BLOOMZ), the authors compare selective pre-translation with full pre-translation and with direct inference in the source language. They report that the best selective configuration per language outperforms full pre-translation for 92% of languages and direct inference for 90% of languages, with particular gains for low-resource languages. They further analyze component-level correlations, mine Apriori association rules, and study the influence of translation quality on downstream performance, concluding that selective pre-translation mitigates the negative effect of lower translation quality and proposing practical guidelines.

Significance. If confirmed, the paper's exhaustive 24-configuration grid and broad language coverage would be a valuable resource for practitioners choosing prompt-translation strategies, and the public HuggingFace Space and released pipeline are useful artifacts. The paper's component-level analysis (e.g., the point-biserial correlations in Table 4 and the performance-gap plots) is informative and goes beyond a simple best-configuration comparison. The translation-quality analysis in Section 4 is a direct empirical relationship—not a circular derivation—and the finding that selective pre-translation reduces the translation-quality correlation to a low value (0.05) is a concrete, falsifiable claim. However, the headline 'consistently outperforms' claim rests on per-language maxima over 24 configurations without error bars, and the resource-level conclusions depend on a single proxy distribution; both issues are load-bearing for the main claims.

major comments (3)
  1. [Section 3.2.1, Table 3] The 92%/90% improvement figures are computed by taking each language's highest-performing configuration among the 24 evaluated configurations and comparing it to the fixed full-English and full-source baselines, which are themselves members of that configuration set. With only 250 examples per language and no confidence intervals or significance tests, the per-language maximum is a positively biased estimator of the achievable improvement: even if no genuine improvement existed, the maximum over 24 noisy scores would exceed the baselines for many languages by chance. This is not merely a cosmetic concern, because several entries in Table 3 show 0% relative improvement (e.g., vi, ro, el in QA; fa, po, tr in summarization), which contradicts the 'consistently outperforms' wording. I ask the authors to add bootstrap confidence intervals or paired significance tests for the top-configuration-versus-baseline comparison, and to temper the 'consistently' claim if the corrected intervals do not support it.
  2. [Section 3.2.2, Table 5] The Apriori-based association rules in Table 5 are mined from the same 24-configuration results used to define the per-language maxima, so they are descriptive summaries of the benchmark rather than validated predictors of which configuration a practitioner should choose. Without held-out validation (for example, learning rules on a subset of languages and testing on the remaining ones) or multiple-comparison correction, the rules do not establish that a user can identify a superior configuration in advance. I recommend adding a simple split-half or language-level cross-validation, or explicitly reframing Table 5 as exploratory analysis.
  3. [Section 3.1 and Section 3.3] The resource-level classification (High/Medium/Low/Unrepresented) is based on GPT-3's training token distribution, as acknowledged in the Limitations section, but it is used to draw conclusions about GPT-3.5-Turbo, Gemini, Mixtral, and BLOOMZ in Table 5 and in the resource-level claims in Section 3.3. Since each model has a different pre-training mixture, the proxy assumption is load-bearing for the practical guidelines for low-resource versus high-resource languages. Please add a robustness check using an alternative resource grouping (e.g., speaker-based categories or per-model data where available) or explicitly restrict the resource-level conclusions to models known to follow a GPT-3-like distribution.
minor comments (7)
  1. [Abstract and Section 1] The word 'sporadic' is used in the abstract; the earlier version 'sporagic' appears in the provided text and should be corrected.
  2. [Table 2] The language label 'Turkuish' should be 'Turkish'.
  3. [Table 3] The language code 'po' for Portuguese is nonstandard and easily confused with Polish; the paper should use 'pt' consistently (and 'pl' for Polish, where applicable).
  4. [Section 3.2.2, Table 4] The caption states 'Positive |τ| values correlate with the source language', but the table reports signed τ values, not absolute values; please clarify the sign convention in the caption or in the text.
  5. [Section 3.1] The phrase 'we experiments with 24 configurations' should be 'we experiment with 24 configurations', and the later sentence describing the '~11 languages per task' count should reconcile with the 35-language total.
  6. [Table 5 and Section 3.3] The model names are inconsistent: 'BLOOMZ', 'Bloomz', 'Bloom', and 'bloomz' are used; standardize to one form.
  7. [Section 4.2] The downstream translation-quality analysis in Figure 4 is limited to the QA task, a limitation the authors acknowledge in the Limitations section; please state this restriction prominently in Section 4.2 as well, rather than only in the Limitations.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation found: the paper's claims are empirical benchmark summaries, and its self-citations are background; the best-of-24 selection issue is a statistical validity concern, not a circularity.

full rationale

This paper is an empirical evaluation, not a derivation, and no load-bearing step reduces to its own inputs. The central 'selective pre-translation outperforms' claim is supported by Table 3, which reports the highest-scoring configuration among 24 per language and compares it with two baselines that are themselves members of the evaluated set. That comparison is a descriptive maximum-vs-baseline summary; the reported 92%/90% figures are computed directly from the benchmark rather than predicted from a fitted parameter, so any concern is best framed as best-of-N selection bias and lack of held-out validation, not as circularity. The Apriori rules in Table 5 are mined from the same 24-configuration results and are likewise descriptive; they are not used to re-derive the benchmark from a fitted model. Self-citations, including Intrator et al. (2024), Seker et al. (2022), and Shaham et al. (2024), appear as background or motivation and are not load-bearing for the empirical claims. The acknowledged GPT-3 resource proxy and the QA-only translation-quality analysis are limitations on external validity, not circular steps. No uniqueness theorem, ansatz, or definition is imported from the authors' prior work. The paper would benefit from confidence intervals and held-out configuration selection, but those are correctness and robustness issues, not circular reasoning.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper does not construct a mathematical derivation, so the ledger records manual analysis choices and domain assumptions rather than fitted physical constants. The most consequential entries are the GPT-3 distribution proxy, the Apriori thresholds, and the absence of uncertainty estimates.

free parameters (4)
  • Apriori minimum support and confidence thresholds = support > 0.15, confidence > 0.8 (Table 5); Appendix A.3.1 states support > 0.05, confidence > 0.75
    Hand-chosen thresholds for association rule mining; the paper reports inconsistent values in two places, which changes the extracted optimal configurations.
  • Score binning percentiles for Apriori = 30th and 60th percentiles into low/medium/high bins
    Binning continuous scores into three bins is a hand-chosen preprocessing step that affects which rules are discovered.
  • Evaluation sample size = 250 examples per language and task
    Subsampling is a design choice; no confidence intervals or repeated sampling are reported, so top-configuration comparisons are noisy.
  • Resource class boundaries = A: p >= 0.1%, B: 0.01% < p < 0.1%, C: 0 < p <= 0.01%, D: p = 0
    Follows Lai et al. (2023), but applied to GPT-3 token counts as a proxy for all models; these hand-set boundaries drive all low/high resource conclusions.
assumptions (5)
  • domain assumption GPT-3's public pre-training distribution is a valid proxy for the resource levels of GPT-3.5, Gemini, Mixtral, and BLOOMZ.
    Section 3.1 explicitly states this proxy is used because other distributions are unavailable; all resource-level findings and guidelines depend on it.
  • domain assumption The four-component prompt decomposition (instruction, context, examples, output) captures all functionally relevant parts of a prompt.
    Section 2 defines the configuration space over these four components; if other prompt elements matter, the exhaustive claim is incomplete.
  • domain assumption Machine translation via Google Translate API, and back-translation of model outputs, introduces no systematic bias that changes the relative ranking of configurations.
    Section 3.1 uses Google Translate for component translation and the appendix describes output translation before scoring; the paper assumes translation errors do not invalidate cross-configuration comparisons.
  • domain assumption Automatic metrics (F1, ROUGE, accuracy) are valid for morphologically rich and low-resource languages in this comparison.
    The Limitations section acknowledges ROUGE can be problematic for morphologically rich languages, yet the main comparisons rely on these metrics.
  • standard math Apriori algorithm and point-biserial correlations are applied correctly to the pooled configuration results.
    These are standard data mining and statistical tools; the paper provides implementation details in Appendix A.3.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond English: The Impact of Prompt Translation Strategies across Languages and Tasks in Multilingual LLMs." pith.science (2026). https://pith.science/paper/RVWLW5VS

@misc{pith2026250209331,
  author       = {Pith},
  title        = {Pith review of: Beyond English: The Impact of Prompt Translation Strategies across Languages and Tasks in Multilingual LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RVWLW5VS}},
  note         = {Machine review of arXiv:2502.09331}
}
read the original abstract

Despite advances in the multilingual capabilities of Large Language Models (LLMs) across diverse tasks, English remains the dominant language for LLM research and development. So, when working with a different language, this has led to the widespread practice of pre-translation, i.e., translating the task prompt into English before inference. Selective pre-translation, a more surgical approach, focuses on translating specific prompt components. However, its current use is sporagic and lacks a systematic research foundation. Consequently, the optimal pre-translation strategy for various multilingual settings and tasks remains unclear. In this work, we aim to uncover the optimal setup for pre-translation by systematically assessing its use. Specifically, we view the prompt as a modular entity, composed of four functional parts: instruction, context, examples, and output, either of which could be translated or not. We evaluate pre-translation strategies across 35 languages covering both low and high-resource languages, on various tasks including Question Answering (QA), Natural Language Inference (NLI), Named Entity Recognition (NER), and Abstractive Summarization. Our experiments show the impact of factors as similarity to English, translation quality and the size of pre-trained data, on the model performance with pre-translation. We suggest practical guidelines for choosing optimal strategies in various multilingual settings.

Figures

Figures reproduced from arXiv: 2502.09331 by the authors.

Figure 1
Figure 1. Prompting Strategies: Direct Inference, Selective Pre-Translation, and Pre-Translation pre-translation approach, which calls for translat￾ing only specific parts of the prompt (Ahuja et al., 2023; Kim et al., 2023; Kim et al.). For example, Liu et al. (2024) show that translating only the con￾text to English outperforms direct-inference in sum￾marization and NLI. Ahuja et al. (2023) translated few-shot examples to E… view at source ↗
Figure 2
Figure 2. Performance Gap Analysis for the Examples [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Scatter plot showing the relationship between syntactic similarity to English (further right is more similar) [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Correlation between translation quality (BERTScore) and accuracy (F1) for Pre-Translation￾Zero-shot prompting, each dot is a different language. focuses on a key question: Are these factors primar￾ily due to the limitations of LLMs, or are due to the quality of the pre…
Figure 5
Figure 5. Figure 5: Examples of 3 configurations of German. Each configuration is in the following format [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Few-Shot and Zero-Shot Performance Gap (Few-Shot - Zero-Shot) for each task/language. [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Output Performance Gap (English - Source) for each task/language [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Percentage improvement over pre-translation approach, when using the highest configuration for each task For GPT-3.5-Turbo. The bars are color-coded based on the language family script [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Recomended Configuration: Overview of our application. From top to bottom: Task Details—general configuration about the task (Task, Language, Model). Clicking on "Recommended Configuration" provides a suggested selective pre-translation configuration [PITH_FULL_IMAGE:…
Figure 10
Figure 10. Figure 10: Generating selective pre-translation prompt for zero-shot: The user needs to configure the instruction (optional) and the languages for the components under "Language Component Selection": instruction, context, examples, and output. Additionally, under "Prompt Input D…
Figure 11
Figure 11. Figure 11: Generating selective pre-translation prompt for few-shot: Here, the user must also configure the few-shot settings: the dataset to use (from which the few-shot examples are taken) and the number of examples to use (default = 1) [PITH_FULL_IMAGE:figures/full_fig_p019_…
Figure 12
Figure 12. Figure 12: Google Translate API vs Bing Translator Comparsion [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 10 canonical work pages

  1. [1]

    David Ifeoluwa Adelani, Jade Abbott, Graham Neubig, Daniel D’souza, Julia Kreutzer, Constantine Lignos, Chester Palen-Michel, Happy Buzaaba, Shruti Rijhwani, Sebastian Ruder, et al. 2021. Masakhaner: Named entity recognition for african languages. Transactions of the Association for Computational Linguistics, 9:1116--1131

    work page 2021

  2. [2]

    Rakesh Agrawal, Tomasz Imieli \'n ski, and Arun Swami. 1993. Mining association rules between sets of items in large databases. In Proceedings of the 1993 ACM SIGMOD international conference on Management of data, pages 207--216

    work page 1993

  3. [3]

    Kabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng, Krithika Ramesh, Prachi Jain, Akshay Nambi, Tanuja Ganu, Sameer Segal, Maxamed Axmed, et al. 2023. Mega: Multilingual evaluation of generative ai. arXiv preprint arXiv:2303.12528

    arXiv 2023

  4. [4]

    Michael Andersland. 2024. Amharic llama and llava: Multimodal llms for low resource languages. arXiv preprint arXiv:2403.06354

    arXiv 2024

  5. [5]

    Stephen R Anderson. 2010. How many languages are there in the world. Linguistic Society of America, pages 1--12

    work page 2010

  6. [6]

    Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2019. http://arxiv.org/abs/1910.11856 On the cross-lingual transferability of monolingual representations . CoRR, abs/1910.11856

    arXiv 2019

  7. [7]

    Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65--72

    2005

  8. [8]

    Rachel Bawden and Fran c ois Yvon. 2023. Investigating the translation performance of a large multilingual language model: the case of bloom. arXiv preprint arXiv:2303.01911

    arXiv 2023

Show all 55 references
  1. [9]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  2. [10]

    Guanhua Chen, Shuming Ma, Yun Chen, Li Dong, Dongdong Zhang, Jia Pan, Wenping Wang, and Furu Wei. 2021. Zero-shot cross-lingual transfer of neural machine translation with multilingual pretrained encoders. arXiv preprint arXiv:2104.08757

    2021 arXiv

  3. [11]

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1--113

    2023

  4. [12]

    Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm \'a n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116

    2019 arXiv

  5. [13]

    Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. https://doi.org/10.18653/v1/D18-1269 XNLI : Evaluating cross-lingual sentence representations . In Proceedings of the 2018 Conference on Empirical Methods ...

    2018 doi

  6. [14]

    Yiming Cui, Ziqing Yang, and Xin Yao. 2023. Efficient and effective text encoding for chinese llama and alpaca. arXiv preprint arXiv:2304.08177

    2023 arXiv

  7. [15]

    Sumanth Doddapaneni, Rahul Aralikatte, Gowtham Ramesh, Shreya Goyal, Mitesh M Khapra, Anoop Kunchukuttan, and Pratyush Kumar. 2022. Indicxtreme: A multi-task benchmark for evaluating indic languages. arXiv preprint arXiv:2212.05409

    2022 arXiv

  8. [16]

    Francisco Guzm \'a n, Peng-Jen Chen, Myle Ott, Juan Pino, Guillaume Lample, Philipp Koehn, Vishrav Chaudhary, and Marc ' Aurelio Ranzato. 2019. https://doi.org/10.18653/v1/D19-1632 The FLORES evaluation datasets for low-resource machine translation: N epali -- E nglish and S i...

    2019 doi

  9. [17]

    Tahmid Hasan, Abhik Bhattacharjee, Md Saiful Islam, Kazi Samin, Yuan-Fang Li, Yong-Bin Kang, M Sohel Rahman, and Rifat Shahriyar. 2021. Xl-sum: Large-scale multilingual abstractive summarization for 44 languages. arXiv preprint arXiv:2106.13822

    2021 arXiv

  10. [18]

    Haoyang Huang, Tianyi Tang, Dongdong Zhang, Wayne Xin Zhao, Ting Song, Yan Xia, and Furu Wei. 2023. Not all languages are created equal in llms: Improving multilingual capability by cross-lingual-thought prompting. arXiv preprint arXiv:2305.07004

    2023 arXiv

  11. [19]

    Jie Huang and Kevin Chen-Chuan Chang. 2022. Towards reasoning in large language models: A survey. arXiv preprint arXiv:2212.10403

    2022 arXiv

  12. [20]

    Yotam Intrator, Matan Halfon, Roman Goldenberg, Reut Tsarfaty, Matan Eyal, Ehud Rivlin, Yossi Matias, and Natalia Aizenberg. 2024. Breaking the language barrier: Can direct inference outperform pre-translation in multilingual llm applications? arXiv preprint arXiv:2403.04792

    2024 arXiv

  13. [21]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    2023 arXiv

  14. [22]

    Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088

    2024 arXiv

  15. [23]

    Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The state and fate of linguistic diversity and inclusion in the nlp world. arXiv preprint arXiv:2004.09095

    2020 arXiv

  16. [24]

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361

    2020 arXiv

  17. [25]

    Translating qa is enough: A key to unlocking in-context cross-lingual performance

    Sunkyoung Kim, Dayeon Ki, Yireun Kim, and Jinsik Lee. Translating qa is enough: A key to unlocking in-context cross-lingual performance. In ICML 2024 Workshop on In-Context Learning

    2024

  18. [26]

    Sunkyoung Kim, Dayeon Ki, Yireun Kim, and Jinsik Lee. 2023. Boosting cross-lingual transferability in multilingual models via in-context learning. arXiv preprint arXiv:2305.15233

    2023 arXiv

  19. [27]

    Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, Ond r ej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, et al. 2023. Findings of the 2023 conference on machine translation (wmt23): Llms are here but not quite there...

    2023

  20. [28]

    Viet Dac Lai, Nghia Trung Ngo, Amir Pouran Ben Veyseh, Hieu Man, Franck Dernoncourt, Trung Bui, and Thien Huu Nguyen. 2023. Chatgpt beyond english: Towards a comprehensive evaluation of large language models in multilingual learning. arXiv preprint arXiv:2304.05613

    2023 arXiv

  21. [29]

    Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ili \'c , Daniel Hesslow, Roman Castagn \'e , Alexandra Sasha Luccioni, Fran c ois Yvon, Matthias Gall \'e , et al. 2023. Bloom: A 176b-parameter open-access multilingual language model

    2023

  22. [30]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74--81

    2004

  23. [31]

    Patrick Littell, David R Mortensen, Ke Lin, Katherine Kairis, Carlisle Turner, and Lori Levin. 2017. Uriel and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors. In Proceedings of the 15th Conference of the European Chapter of the Associat...

    2017

  24. [32]

    Chaoqun Liu, Wenxuan Zhang, Yiran Zhao, Anh Tuan Luu, and Lidong Bing. 2024. Is translation all you need? a study on solving multilingual tasks with large language models. arXiv preprint arXiv:2403.10258

    2024 arXiv

  25. [33]

    Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong, Hailey Schoelkopf, et al. 2022. Crosslingual generalization through multitask finetuning. arXiv preprint arXiv:2211.01786

    2022 arXiv

  26. [34]

    Sina Bagheri Nezhad and Ameeta Agrawal. 2024. What drives performance in multilingual language models? arXiv preprint arXiv:2404.19159

    2024 arXiv

  27. [35]

    Gabriel Nicholas and Aliya Bhatia. 2023. Lost in translation: Large language models in non-english content analysis. arXiv preprint arXiv:2306.07377

    2023 arXiv

  28. [36]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 3...

    2022

  29. [37]

    Xiaoman Pan, Boliang Zhang, Jonathan May, Joel Nothman, Kevin Knight, and Heng Ji. 2017. Cross-lingual name tagging and linking for 282 languages. In Proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: long papers), pages 1946--1958

    2017

  30. [38]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311--318

    2002

  31. [39]

    Gregory Piatetsky-Shapiro. 1991. Discovery, analysis, and presentation of strong rules. Knowledge Discovery in Data-bases, pages 229--248

    1991

  32. [40]

    Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang. 2023. Is chatgpt a general-purpose natural language processing task solver? arXiv preprint arXiv:2302.06476

    2023 arXiv

  33. [41]

    Libo Qin, Qiguang Chen, Yuhang Zhou, Zhi Chen, Yinghui Li, Lizi Liao, Min Li, Wanxiang Che, and Philip S Yu. 2024. Multilingual large language model: A survey of resources, taxonomy and frontiers. arXiv preprint arXiv:2404.04925

    2024 arXiv

  34. [42]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1--67

    2020

  35. [43]

    Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. Comet: A neural framework for mt evaluation. arXiv preprint arXiv:2009.09025

    2020 arXiv

  36. [44]

    Walid S Saba. 2024. Llms' understanding of natural language revealed. arXiv preprint arXiv:2407.19630

    2024 arXiv

  37. [45]

    Amit Seker, Elron Bandel, Dan Bareket, Idan Brusilovsky, Refael Greenfeld, and Reut Tsarfaty. 2022. Alephbert: Language model pre-training and evaluation from sub-word to sentence level. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics...

    2022

  38. [46]

    Uri Shaham, Jonathan Herzig, Roee Aharoni, Idan Szpektor, Reut Tsarfaty, and Matan Eyal. 2024. Multilingual instruction tuning with just a pinch of multilinguality. arXiv preprint arXiv:2401.01854

    2024 arXiv

  39. [47]

    Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, et al. 2022. Language models are multilingual chain-of-thought reasoners. arXiv preprint arXiv:2210.03057

    2022 arXiv

  40. [48]

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805

    2023 arXiv

  41. [49]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  42. [50]

    Genta Indra Winata, Andrea Madotto, Zhaojiang Lin, Rosanne Liu, Jason Yosinski, and Pascale Fung. 2021. Language models are few-shot multilingual learners. arXiv preprint arXiv:2109.07684

    2021 arXiv

  43. [51]

    Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2020. mt5: A massively multilingual pre-trained text-to-text transformer. arXiv preprint arXiv:2010.11934

    2020 arXiv

  44. [52]

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675

    2019 arXiv

  45. [53]

    Mengjie Zhao and Hinrich Sch \"u tze. 2021. Discrete and soft prompting for multilingual models. arXiv preprint arXiv:2109.03630

    2021 arXiv

  46. [54]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  47. [55]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.