Pith. sign in

REVIEW 3 major objections 4 minor 84 references

Multilingual Prompt Engineering in Large Language Models: A Survey Across NLP Tasks

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This survey sorts 39 multilingual prompting techniques by NLP task and names the best method for each benchmark dataset.

desk verdict A genuinely useful task-organized map of multilingual prompting, undercut by a SoTA column the authors admit cannot be checked. read the letter →

arxiv 2505.11665 v1 pith:KDLQI2FI submitted 2025-05-16 cs.CL cs.AIcs.LG

Shubham Vatsal , Harsh Dubey , Aditi Singh This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords multilingualpromptengineeringlargelanguagemodelscross-lingualtransferchain-of-thoughtpromptingstate-of-the-artmethodslow-resourcelanguagesNLPtasktaxonomyin-contextlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This survey claims that multilingual prompt engineering is now systematic enough to be organized by NLP task, and that a task-based taxonomy is the right way to compare methods. It reviews 36 papers, 39 prompting techniques, and 30 multilingual tasks spanning roughly 250 languages, and assigns each dataset a potential best (SoTA) prompting method. The point of the exercise is practical: a researcher or practitioner who wants to prompt an LLM in multiple languages can look up which technique has worked best for their task and language, rather than re-deriving the field from scratch. The survey also documents a clear imbalance: most methods and studies concentrate on high-resource languages, while the wide low-resource coverage is driven almost entirely by machine translation.

What carries the argument

The carrying apparatus is a task-first taxonomy plus a standardized naming scheme for prompting techniques. The survey groups 30 NLP tasks, from reasoning and question answering to translation, sequence labeling, and dialogue evaluation, and for each dataset it tabulates the prompting strategies tested, the LLMs used, the language count, and the designated SoTA method. The standardization of names, which collapses Basic, Standard, Vanilla, and Direct prompting into En-Basic or Native-Basic and merges variants into {Technique} + Variations, is what lets results from different papers sit side by side in one comparison.

What would settle it

Run the designated SoTA method and its strongest competitor on the same dataset, with the same metric and decoding settings; if the competitor wins on MGSM, XNLI, or FLORES, that dataset's SoTA designation is wrong. A milder check: for any SoTA entry backed by a single study, reproduction by an independent group would be the first test.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that multilingual prompt engineering can be productively standardized and catalogued: methods with different names across the literature are consolidated under common labels such as En-Basic, Native-Basic, X-Basic, XLT, CLP, CLSP, X-InSTA, SAP, DIPMT, DecoMT, MAPS, and MEEP, and each is mapped to the NLP task and datasets where it has been tested. The survey then identifies a potential state-of-the-art prompting method for each dataset, for instance CLSP for MGSM, XLT for XNLI and PAWS-X, X-InSTA for MARC and CLS, DecoMT for FLORES, and MAPS for WMT-22. It further argues that cross-lingual reasoning methods that route through English or align a native language with English generally beat plain English or native baselines on reasoning-heavy tasks, and that this whole field is recent and fast-moving.

Load-bearing premise

The whole per-dataset SoTA ranking rests on the assumption that results from different studies can be compared at all, even though evaluation metrics differ and dataset versions vary across the surveyed papers.

Editorial extensions

If this is right

  • A practitioner can select a starting-point prompt for a multilingual dataset from the SoTA column, such as CLSP for MGSM or X-InSTA for MARC, instead of testing every published variant.
  • Cross-lingual chain-of-thought and translation-anchored methods are the strongest family for reasoning and inference tasks, while dictionary-based and memory-based prompting dominates translation for low-resource languages.
  • The taxonomy gives future surveys and benchmarks a common vocabulary, since it consolidates multiple names for the same core prompting idea.
  • Low-resource languages are covered more broadly than commonly assumed, but almost entirely inside machine translation; other tasks remain high-resource dominated.
  • Because most included studies appeared within the last two years, the SoTA designations are expected to shift quickly as new methods are published.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Read the SoTA column as provisional: the survey itself notes that evaluation metrics are omitted and that designations rely on informed judgment, so the real ranking is an empirical question until methods meet on identical benchmarks.
  • A testable extension would sort the same prompts by language family rather than by task, asking whether English-routing methods hold their edge in typologically distant families such as Dravidian or Niger-Congo.
  • The high-resource versus low-resource gap suggests an immediate research target: port the twenty prompting techniques already used on high-resource languages to low-resource settings beyond translation, where only a subset has been tried.
  • The taxonomy could be extended into a living leaderboard, replacing informed judgment with standardized evaluation across the same model family.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This manuscript surveys multilingual prompt engineering for large language models, reviewing 36 research papers, 39 prompting techniques, 30 NLP tasks, and roughly 250 languages. It organizes the literature by NLP task, presenting per-task tables that list prompting strategies, LLMs, language counts, references, and a designated 'potential SoTA' method for each dataset. It then derives descriptive insights about the distribution of tasks and prompting techniques across language families and high- vs. low-resource languages. The central claims are that multilingual prompt engineering can be systematically categorized by NLP task and that per-dataset SoTA prompting methods can be identified from the surveyed literature.

Significance. If the taxonomy and SoTA designations were adequately supported, this survey would be a useful reference for practitioners selecting prompting methods in multilingual settings, and the language-family/resource-level analyses would help identify coverage gaps. A notable strength is that the descriptive statistics in Section 4 are transparent tallies of the authors' own curated tables rather than fitted or predicted quantities, and the paper states its selection criteria for included papers. However, the usefulness of the survey as a method-selection resource depends on the reliability of the SoTA column and on the consistency of the task taxonomy, both of which currently need substantial revision.

major comments (3)
  1. [Section 3 (intro) and Tables 2-31] The SoTA designations are not verifiable as presented. The paper states that 'Evaluation metrics are omitted, as they differ across studies' and that 'the use of varying versions of the same dataset further complicates direct performance comparisons,' with SoTA chosen by 'informed judgment.' Because each SoTA entry aggregates results across different LLMs, few-shot counts, decoding settings, and unstated metrics from different source papers, a reader cannot independently check entries such as CLSP for MGSM (Table 2), XLT for XNLI (Table 22), or X-InSTA for MARC (Table 8). To support the central claim, the authors should either report the metric and experimental protocol behind each SoTA designation, or explicitly restrict the claim to 'best among methods compared under a common protocol' and remove designations that aggregate incomparable results.
  2. [Section 3 taxonomy; Tables 22, 25, 28] The claim that 'we ensure each dataset is associated with a single NLP task' is contradicted by the survey's own tables. XNLI appears in both Natural Language Inference (Table 22) and Task Understanding Consistency (Table 25), while PAWS-X appears in both Task Understanding Consistency (Table 25) and Paraphrasing (Table 28). This undermines the stated principle of the taxonomy and complicates the descriptive analyses that count tasks per dataset. The authors should either allow explicit multi-label assignments with justification, or remove one of the duplicate assignments and clarify the boundaries between the affected task definitions.
  3. [Section 1.2] The literature search process is not reproducible as described. The paper lists 11 Google Scholar queries, says manual filtering produced 189 articles, and then applies two selection criteria to reach 36 papers, but it does not report the search date, the exact query strings used, the number of papers retrieved per query, or the screening decisions that removed papers. Since the language-family and resource-level statistics in Section 4 depend entirely on this selected corpus, the authors should provide a more complete and reproducible selection protocol, including a flowchart or exclusion log.
minor comments (4)
  1. [Section 3.15, Table 16] The text says 'we found two datasets which are XGLUE-POS, UDPOS and XTREME,' but lists three datasets; the table also contains three rows. Please correct the count or the wording.
  2. [Abstract and Section 4] The abstract reports 39 prompting techniques, while Section 4.1 and Figure 7 state '38 distinct prompting techniques.' Please reconcile the counts.
  3. [Table 13 and Section 3.12] There are typographical errors in language-pair names: 'Tamil-Finish,' 'Finish-French,' 'Finish-German,' 'Finish-Arabic,' and similar entries should read 'Finnish'; 'Gujrati-Hindi' should be 'Gujarati-Hindi.'
  4. [Throughout] Several LaTeX artifacts appear in the text, including 'Y oruba,' 'V osoughi,' and 'ty pologically.' A copyediting pass is needed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the survey's SoTA labels are explicit literature syntheses, not predictions derived from fitted inputs.

full rationale

This survey does not derive quantitative predictions from fitted parameters. Its 'potential SoTA' entries are explicitly hedged: the authors state 'Evaluation metrics are omitted, as they differ across studies' and 'Given these limitations, we rely on informed judgment to identify the most effective prompting method per dataset based on existing literature and multilingual results,' so the SoTA column is a literature synthesis, not a benchmark reproduction. The descriptive statistics (e.g., counts of prompting techniques per language family or resource level) are immediate tallies of the authors' own curated tables, not external benchmarks, and no equation reduces a claimed finding to its own input. The only author self-citations appear in a related-work list of generic prompt-engineering surveys (Vatsal & Dubey 2024; Singh et al. 2024); they are not used to justify the taxonomy, SoTA choices, or conclusions, so they are not load-bearing. Internal inconsistencies such as PAWS-X appearing under both Paraphrasing and Task Understanding Consistency, and the omission of metrics, are verification and consistency concerns rather than circular reasoning.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The survey introduces no fitted parameters and no new entities. Its claims rest on selection criteria (hard prompts, at least two languages), the accuracy of the cited papers, and the authors' resource and family classifications, none of which are externally validated in the paper.

assumptions (5)
  • domain assumption Hard prompt focus: only techniques using discrete natural-language prompts are included; soft prompt tuning is excluded by criterion 1 in Section 1.2.
    Selection criterion restricting the survey's scope; if soft prompting were included, the taxonomy and SoTA designations would change.
  • domain assumption Multilingual scope: each included study must apply the method in at least two languages (criterion 2, Section 1.2).
    This definition of 'multilingual' excludes single-language and English-only studies, shaping all coverage counts and figures.
  • domain assumption Reported results in the 36 surveyed papers are taken as accurate.
    The survey does not re-run experiments, so its SoTA designations inherit the validity of the source papers.
  • domain assumption High-resource versus low-resource classification and language-family assignments are reliable and consistent across the survey.
    Figures 2-9 aggregate languages by family and resource level, but the underlying assignment procedure is not described in the text, making these aggregations unauditable.
  • domain assumption The authors' 'informed judgment' is sufficient to identify a best prompting method per dataset despite heterogeneous metrics and dataset versions.
    Stated in Section 3 intro: evaluation metrics are omitted and SoTA is chosen by informed judgment from existing literature.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multilingual Prompt Engineering in Large Language Models: A Survey Across NLP Tasks." pith.science (2026). https://pith.science/paper/KDLQI2FI

@misc{pith2026250511665,
  author       = {Pith},
  title        = {Pith review of: Multilingual Prompt Engineering in Large Language Models: A Survey Across NLP Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KDLQI2FI}},
  note         = {Machine review of arXiv:2505.11665}
}
read the original abstract

Large language models (LLMs) have demonstrated impressive performance across a wide range of Natural Language Processing (NLP) tasks. However, ensuring their effectiveness across multiple languages presents unique challenges. Multilingual prompt engineering has emerged as a key approach to enhance LLMs' capabilities in diverse linguistic settings without requiring extensive parameter re-training or fine-tuning. With growing interest in multilingual prompt engineering over the past two to three years, researchers have explored various strategies to improve LLMs' performance across languages and NLP tasks. By crafting structured natural language prompts, researchers have successfully extracted knowledge from LLMs across different languages, making these techniques an accessible pathway for a broader audience, including those without deep expertise in machine learning, to harness the capabilities of LLMs. In this paper, we survey and categorize different multilingual prompting techniques based on the NLP tasks they address across a diverse set of datasets that collectively span around 250 languages. We further highlight the LLMs employed, present a taxonomy of approaches and discuss potential state-of-the-art (SoTA) methods for specific multilingual datasets. Additionally, we derive a range of insights across language families and resource levels (high-resource vs. low-resource), including analyses such as the distribution of NLP tasks by language resource type and the frequency of prompting methods across different language families. Our survey reviews 36 research papers covering 39 prompting techniques applied to 30 multilingual NLP tasks, with the majority of these studies published in the last two years.

Figures

Figures reproduced from arXiv: 2505.11665 by the authors.

Figure 1
Figure 1. Taxonomy Diagram of Prompt Engineering Methods and Languages Applied Across Different [PITH_FULL_IMAGE:figures/full_fig_p016_1.png] view at source ↗
Figure 2
Figure 2. NLP Task Distribution by Language Family [PITH_FULL_IMAGE:figures/full_fig_p041_2.png] view at source ↗
Figure 3
Figure 3. Number of Prompting Techniques by Language Family [PITH_FULL_IMAGE:figures/full_fig_p042_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Number of Papers by Language Family demonstrate moderate research engagement, each with around 15-19 papers, suggesting a growing but still limited recognition of the importance of linguistic diversity in NLP research. In contrast, families such as Tupian, Kartvelian, …
Figure 5
Figure 5. Figure 5: Language Coverage by Language Family regions. The observed disparities highlight the necessity for more targeted research and resource devel￾opment for low-resource and underrepresented language families, ensuring that future advances in prompt engineering can better s…
Figure 6
Figure 6. Figure 6: NLP Task Distribution by Resource Level low-resource settings, thereby advancing language technology for a broader and more inclusive set of users worldwide. Understanding the distribution of prompting techniques across language resource levels is essential for as￾sess…
Figure 7
Figure 7. Figure 7: Number of Prompting Techniques by Resource Level [PITH_FULL_IMAGE:figures/full_fig_p046_7.png]
Figure 8
Figure 8. Figure 8: Number of Papers by Resource Level The breadth of experimentation in high-resource settings contributes to rapid advancements in performance, robustness and nuanced capabilities for different NLP tasks. In contrast, low-resource languages see far less methodological va…
Figure 9
Figure 9. Figure 9: Language Coverage by Resource Levels overview of the number of languages from high-resource and low-resource categories that have been ex￾plored in the survey, spanning a total of 249 languages. Notably, while the majority of research papers and prompting techniques fo…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

84 extracted references · 22 canonical work pages

  1. [4]

    Mega: Multilingual evaluation of generative ai

    Kabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng, Krithika Ramesh, Prachi Jain, Akshay Nambi, Tanuja Ganu, Sameer Segal, Maxamed Axmed, et al. Mega: Multilingual evaluation of generative ai. arXiv preprint arXiv:2303.12528,

    arXiv

  2. [6]

    On the cross-lingual transferability of monolingual representations

    Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. On the cross-lingual transferability of monolingual representations. arXiv preprint arXiv:1910.11856,

    arXiv 1910

  3. [8]

    The belebele benchmark: a parallel reading comprehension dataset in 122 language variants.arXiv preprint arXiv:2308.16884,

    Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Na- man Goyal, Abhinandan Krishnan, Luke Zettlemoyer, and Madian Khabsa. The belebele benchmark: a parallel reading comprehension dataset in 122 language variants.arXiv preprint arXiv:2308.16884,

    arXiv

  4. [9]

    Semeval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter

    Valerio Basile, Cristina Bosco, Elisabetta Fersini, Debora Nozza, Viviana Patti, Francisco Manuel Rangel Pardo, Paolo Rosso, and Manuela Sanguinetti. Semeval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter. In Proceedings of the 13th international workshop on semantic evaluation, pp. 54–63,

    2019

  5. [10]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901,

    1901

  6. [11]

    A Sentiment Analysis Dataset for Code-Mixed Malayalam-English

    Bharathi Raja Chakravarthi, Navya Jose, Shardul Suryawanshi, Elizabeth Sherly, and John P McCrae. A sentiment analysis dataset for code-mixed malayalam-english. arXiv preprint arXiv:2006.00210, 2020a. Bharathi Raja Chakravarthi, Vigneshwaran Muralidaran, Ruba Priyadharshini, and John P McCrae. Corpus creation for sentiment analysis in code-mixed tamil-eng...

    work page Pith review arXiv 2006

  7. [12]

    A survey on evaluation of large language models

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Kaijie Zhu, Hao Chen, Linyi Yang, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. A survey on evaluation of large language models. arXiv preprint arXiv:2307.03109,

    arXiv

  8. [13]

    Unleashing the potential of prompt engineering in large language models: a comprehensive review

    Banghao Chen, Zhaofeng Zhang, Nicolas Langren´e, and Shengxin Zhu. Unleashing the potential of prompt engineering in large language models: a comprehensive review. arXiv preprint arXiv:2310.14735,

    arXiv

Show all 84 references
  1. [14]

    Xnli: Evaluating cross-lingual sentence representations

    Alexis Conneau, Guillaume Lample, Ruty Rinott, Adina Williams, Samuel R Bowman, Holger Schwenk, and Veselin Stoyanov. Xnli: Evaluating cross-lingual sentence representations. arXiv preprint arXiv:1809.05053,

    arXiv

  2. [16]

    Indicxtreme: A multi-task benchmark for evaluating indic languages

    Sumanth Doddapaneni, Rahul Aralikatte, Gowtham Ramesh, Shreya Goyal, Mitesh M Khapra, Anoop Kunchukuttan, and Pratyush Kumar. Indicxtreme: A multi-task benchmark for evaluating indic languages. arXiv preprint arXiv:2212.05409,

    arXiv

  3. [18]

    Privacy preserving prompt engineering: A survey

    Kennedy Edemacu and Xintao Wu. Privacy preserving prompt engineering: A survey. arXiv preprint arXiv:2404.06001,

    arXiv

  4. [19]

    Do multilingual language models think better in english? arXiv preprint arXiv:2308.01223,

    Julen Etxaniz, Gorka Azkune, Aitor Soroa, Oier Lopez de Lacalle, and Mikel Artetxe. Do multilingual language models think better in english? arXiv preprint arXiv:2308.01223,

    arXiv

  5. [20]

    Meep: Is this engaging? prompting large language models for dialogue evaluation in multilingual settings

    Amila Ferron, Amber Shore, Ekata Mitra, and Ameeta Agrawal. Meep: Is this engaging? prompting large language models for dialogue evaluation in multilingual settings. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 2078–2100,

    2023

  6. [21]

    Gptscore: Evaluate as you desire.arXiv preprint arXiv:2302.04166,

    Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. Gptscore: Evaluate as you desire.arXiv preprint arXiv:2302.04166,

    arXiv

  7. [22]

    Overview of hope at iberlef 2024: Approaching hope speech detection in social media from two perspectives, for equality, diversity and inclusion and as expectations

    49 Daniel Garc ´ıa-Baena, Fazlourrahman Balouchzahi, Sabur Butt, Miguel ´Angel Garc ´ıa-Cumbreras, At- nafu Lambebo Tonja, Jos ´e Antonio Garc´ıa-D´ıaz, Selen Bozkurt, Bharathi Raja Chakravarthi, Hector G Ceballos, Rafael Valencia-Garc´ıa, et al. Overview of hope at iberlef 20...

    2024

  8. [23]

    Dictionary-based phrase-level prompting of large language models for machine translation

    Marjan Ghazvininejad, Hila Gonen, and Luke Zettlemoyer. Dictionary-based phrase-level prompting of large language models for machine translation. arXiv preprint arXiv:2302.07856,

    arXiv

  9. [24]

    Teaching large language models to translate on low-resource languages with textbook prompting

    Ping Guo, Yubing Ren, Yue Hu, Yunpeng Li, Jiarui Zhang, Xingsheng Zhang, and He-Yan Huang. Teaching large language models to translate on low-resource languages with textbook prompting. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Lan...

    2024

  10. [25]

    Xl-sum: Large-scale multilingual abstractive summarization for 44 languages

    Tahmid Hasan, Abhik Bhattacharjee, Md Saiful Islam, Kazi Samin, Yuan-Fang Li, Yong-Bin Kang, M So- hel Rahman, and Rifat Shahriyar. Xl-sum: Large-scale multilingual abstractive summarization for 44 languages. arXiv preprint arXiv:2106.13822,

    arXiv

  11. [26]

    Prompting chatgpt for translation: A comparative analysis of translation brief and persona prompts

    Sui He. Prompting chatgpt for translation: A comparative analysis of translation brief and persona prompts. arXiv preprint arXiv:2403.00127,

    arXiv

  12. [27]

    Ocnli: Original chinese natural language inference

    Hai Hu, Kyle Richardson, Liang Xu, Lu Li, Sandra K ¨ubler, and Lawrence S Moss. Ocnli: Original chinese natural language inference. arXiv preprint arXiv:2010.05444, 2020a. Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. Xtreme: A ma...

    2010 arXiv

  13. [28]

    A survey on large language models with multilingualism: Recent advances and new frontiers

    Kaiyu Huang, Fengran Mo, Xinyu Zhang, Hongliang Li, You Li, Yuanchi Zhang, Weijian Yi, Yulong Mao, Jinchen Liu, Yuzhuang Xu, et al. A survey on large language models with multilingualism: Recent advances and new frontiers. arXiv preprint arXiv:2405.10936,

    arXiv

  14. [29]

    Breaking the language barrier: Can direct inference outperform pre-translation in multilingual llm applications? arXiv preprint arXiv:2403.04792,

    Yotam Intrator, Matan Halfon, Roman Goldenberg, Reut Tsarfaty, Matan Eyal, Ehud Rivlin, Yossi Matias, and Natalia Aizenberg. Breaking the language barrier: Can direct inference outperform pre-translation in multilingual llm applications? arXiv preprint arXiv:2403.04792,

    arXiv

  15. [30]

    Towards effective disambiguation for machine translation with large language models

    Vivek Iyer, Pinzhen Chen, and Alexandra Birch. Towards effective disambiguation for machine translation with large language models. arXiv preprint arXiv:2309.11668,

    arXiv

  16. [31]

    Indicnlpsuite: Monolingual corpora, evaluation benchmarks and pre-trained multi- lingual language models for indian languages

    50 Divyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, Gokul NC, Avik Bhattacharyya, Mitesh M Khapra, and Pratyush Kumar. Indicnlpsuite: Monolingual corpora, evaluation benchmarks and pre-trained multi- lingual language models for indian languages. InFindings of the associati...

    2020

  17. [32]

    The multilingual amazon reviews corpus

    Phillip Keung, Yichao Lu, Gy ¨orgy Szarvas, and Noah A Smith. The multilingual amazon reviews corpus. arXiv preprint arXiv:2010.02573,

    2010 arXiv

  18. [33]

    Gluecos: An evaluation benchmark for code-switched nlp

    Simran Khanuja, Sandipan Dandapat, Anirudh Srinivasan, Sunayana Sitaram, and Monojit Choudhury. Gluecos: An evaluation benchmark for code-switched nlp. arXiv preprint arXiv:2004.12376,

    2004 arXiv

  19. [34]

    Cross-lingual qa: A key to unlocking in-context cross-lingual performance

    Sunkyoung Kim, Dayeon Ki, Yireun Kim, and Jinsik Lee. Cross-lingual qa: A key to unlocking in-context cross-lingual performance. arXiv preprint arXiv:2305.15233,

    arXiv

  20. [35]

    Findings of the 2022 conference on machine translation (wmt22)

    Tom Kocmi, Rachel Bawden, Ondˇrej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Thamme Gowda, Yvette Graham, Roman Grundkiewicz, Barry Haddow, et al. Findings of the 2022 conference on machine translation (wmt22). In Proceedings of the Seventh Conference on Machin...

    2022

  21. [36]

    Chatgpt beyond english: Towards a comprehensive evaluation of large language models in multilingual learning

    Viet Dac Lai, Nghia Trung Ngo, Amir Pouran Ben Veyseh, Hieu Man, Franck Dernoncourt, Trung Bui, and Thien Huu Nguyen. Chatgpt beyond english: Towards a comprehensive evaluation of large language models in multilingual learning. arXiv preprint arXiv:2304.05613,

    arXiv

  22. [37]

    The power of scale for parameter-efficient prompt tuning

    Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691,

    arXiv

  23. [39]

    Duie: A large-scale chinese dataset for information extraction

    Shuangjie Li, Wei He, Yabing Shi, Wenbin Jiang, Haijin Liang, Ye Jiang, Yang Zhang, Yajuan Lyu, and Yong Zhu. Duie: A large-scale chinese dataset for information extraction. In Natural Language Processing and Chinese Computing: 8th CCF International Conference, NLPCC 2019, Dun...

    2019

  24. [40]

    Duee: a large-scale dataset for chinese event extraction in real-world scenarios

    Xinyu Li, Fayuan Li, Lu Pan, Yuguang Chen, Weihua Peng, Quan Wang, Yajuan Lyu, and Yong Zhu. Duee: a large-scale dataset for chinese event extraction in real-world scenarios. InNatural Language Processing and Chinese Computing: 9th CCF International Conference, NLPCC 2020, Zhe...

    2020

  25. [41]

    Xglue: A new benchmark dataset for cross-lingual pre-training, understanding and generation

    Yaobo Liang, Nan Duan, Yeyun Gong, Ning Wu, Fenfei Guo, Weizhen Qi, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, et al. Xglue: A new benchmark dataset for cross-lingual pre-training, understanding and generation. arXiv preprint arXiv:2004.01401,

    2004 arXiv

  26. [42]

    Common sense beyond english: Eval- uating and improving multilingual language models for commonsense reasoning

    Bill Yuchen Lin, Seyeon Lee, Xiaoyang Qiao, and Xiang Ren. Common sense beyond english: Eval- uating and improving multilingual language models for commonsense reasoning. arXiv preprint arXiv:2106.06937,

    arXiv

  27. [43]

    Few-shot learning with multilingual generative language models

    51 Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, et al. Few-shot learning with multilingual generative language models. In Proceedings of the 2022 conference on empirical methods in n...

    2022

  28. [44]

    Is translation all you need? a study on solving multilingual tasks with large language models

    Chaoqun Liu, Wenxuan Zhang, Yiran Zhao, Anh Tuan Luu, and Lidong Bing. Is translation all you need? a study on solving multilingual tasks with large language models. arXiv preprint arXiv:2403.10258,

    arXiv

  29. [45]

    Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing

    Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM computing surveys, 55(9):1–35, 2023a. Yang Liu, Dan Iter, Yichong Xu, Shuohang ...

    arXiv

  30. [46]

    Chain-of- dictionary prompting elicits translation in large language models.arXiv preprint arXiv:2305.06575,

    Hongyuan Lu, Haoran Yang, Haoyang Huang, Dongdong Zhang, Wai Lam, and Furu Wei. Chain-of- dictionary prompting elicits translation in large language models.arXiv preprint arXiv:2305.06575,

    arXiv

  31. [47]

    Semeval-2022 task 11: Multilingual complex named entity recognition (multiconer)

    Shervin Malmasi, Anjie Fang, Besnik Fetahu, Sudipta Kar, and Oleg Rokhlenko. Semeval-2022 task 11: Multilingual complex named entity recognition (multiconer). In Proceedings of the 16th international workshop on semantic evaluation (SemEval-2022), pp. 1412–1437,

    2022

  32. [48]

    Gupshup: Summarizing open-domain code-switched conversations

    Laiba Mehnaz, Debanjan Mahata, Rakesh Gosangi, Uma Sushmitha Gunturi, Riya Jain, Gauri Gupta, Amardeep Kumar, Isabelle G Lee, Anish Acharya, and Rajiv Shah. Gupshup: Summarizing open-domain code-switched conversations. In Proceedings of the 2021 Conference on Empirical Methods...

    2021

  33. [49]

    Unsupervised evaluation of interactive dialog with dialogpt

    Shikib Mehri and Maxine Eskenazi. Unsupervised evaluation of interactive dialog with dialogpt. arXiv preprint arXiv:2006.12719,

    2006 arXiv

  34. [50]

    Simple llm prompting is state-of-the-art for robust and multilingual dialogue evaluation

    John Mendonc ¸a, Patr´ıcia Pereira, Helena Moniz, Joao Paulo Carvalho, Alon Lavie, and Isabel Trancoso. Simple llm prompting is state-of-the-art for robust and multilingual dialogue evaluation. arXiv preprint arXiv:2308.16797,

    arXiv

  35. [51]

    Adaptive machine translation with large language models

    Yasmin Moslem, Rejwanul Haque, John D Kelleher, and Andy Way. Adaptive machine translation with large language models. arXiv preprint arXiv:2301.13294,

    arXiv

  36. [52]

    Crosslingual generalization through multitask finetuning

    Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong, Hailey Schoelkopf, et al. Crosslingual generalization through multitask finetuning. arXiv preprint arXiv:2211.01786,

    arXiv

  37. [53]

    Decomposed prompting: Unveiling multilingual linguistic structure knowledge in english- centric large language models

    Ercong Nie, Shuzhou Yuan, Bolei Ma, Helmut Schmid, Michael F ¨arber, Frauke Kreuter, and Hinrich Sch¨utze. Decomposed prompting: Unveiling multilingual linguistic structure knowledge in english- centric large language models. arXiv preprint arXiv:2402.18397,

  38. [54]

    Swiss-judgment-prediction: A multilingual legal judg- ment prediction benchmark

    Joel Niklaus, Ilias Chalkidis, and Matthias St ¨urmer. Swiss-judgment-prediction: A multilingual legal judg- ment prediction benchmark. arXiv preprint arXiv:2110.00806,

    arXiv

  39. [55]

    Cross-lingual name tagging and linking for 282 languages

    52 Xiaoman Pan, Boliang Zhang, Jonathan May, Joel Nothman, Kevin Knight, and Heng Ji. Cross-lingual name tagging and linking for 282 languages. In Proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: long papers), pp. 1946–1958,

    1946

  40. [56]

    Ajay Patel, Bryan Li, Mohammad Sadegh Rasooli, Noah Constant, Colin Raffel, and Chris Callison-Burch

    arXiv preprint arXiv:2105.09680. Ajay Patel, Bryan Li, Mohammad Sadegh Rasooli, Noah Constant, Colin Raffel, and Chris Callison-Burch. Bidirectional language models are also few-shot learners. arXiv preprint arXiv:2209.14500,

    arXiv

  41. [57]

    Interactive-chain-prompting: Ambigu- ity resolution for crosslingual conditional generation with interaction

    Jonathan Pilault, Xavier Garcia, Arthur Braˇzinskas, and Orhan Firat. Interactive-chain-prompting: Ambigu- ity resolution for crosslingual conditional generation with interaction. arXiv preprint arXiv:2301.10309,

    arXiv

  42. [58]

    Xcopa: A multilingual dataset for causal commonsense reasoning

    Edoardo Maria Ponti, Goran Glavaˇs, Olga Majewska, Qianchu Liu, Ivan Vuli´c, and Anna Korhonen. Xcopa: A multilingual dataset for causal commonsense reasoning. arXiv preprint arXiv:2005.00333,

    2005 arXiv

  43. [59]

    Machine translation with large language models: Prompt engineering for persian, english, and russian directions

    Nooshin Pourkamali and Shler Ebrahim Sharifi. Machine translation with large language models: Prompt engineering for persian, english, and russian directions. arXiv preprint arXiv:2401.08429,

    arXiv

  44. [61]

    Multilingual large language model: A survey of resources, taxonomy and frontiers

    Libo Qin, Qiguang Chen, Yuhang Zhou, Zhi Chen, Yinghui Li, Lizi Liao, Min Li, Wanxiang Che, and Philip S Yu. Multilingual large language model: A survey of resources, taxonomy and frontiers. arXiv preprint arXiv:2404.04925,

    arXiv

  45. [62]

    Xl-wic: A multilingual benchmark for evaluating semantic contextualization

    Alessandro Raganato, Tommaso Pasini, Jose Camacho-Collados, and Mohammad Taher Pilehvar. Xl-wic: A multilingual benchmark for evaluating semantic contextualization. arXiv preprint arXiv:2010.06478,

    2010 arXiv

  46. [63]

    Leveraging gpt-4 for automatic translation post-editing

    Vikas Raunak, Amr Sharaf, Yiren Wang, Hany Hassan Awadallah, and Arul Menezes. Leveraging gpt-4 for automatic translation post-editing. arXiv preprint arXiv:2305.14878,

    arXiv

  47. [64]

    arXiv preprint arXiv:2306.12794,

    arXiv

  48. [65]

    A systematic survey of prompt engineering in large language models: Techniques and applications

    53 Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. A systematic survey of prompt engineering in large language models: Techniques and applications. arXiv preprint arXiv:2402.07927,

    arXiv

  49. [66]

    The prompt report: A systematic survey of prompting techniques

    Sander Schulhoff, Michael Ilie, Nishant Balepur, Konstantine Kahadze, Amanda Liu, Chenglei Si, Yinheng Li, Aayush Gupta, HyoJung Han, Sevien Schulhoff, et al. The prompt report: A systematic survey of prompting techniques. arXiv preprint arXiv:2406.06608,

    arXiv

  50. [67]

    What makes a good conversation? how controllable attributes affect human judgments

    Abigail See, Stephen Roller, Douwe Kiela, and Jason Weston. What makes a good conversation? how controllable attributes affect human judgments. arXiv preprint arXiv:1902.08654,

    1902 arXiv

  51. [68]

    Mul- tilingual entity and relation extraction dataset and model

    Alessandro Seganti, Klaudia Firlkag, Helena Skowronska, Michał Satława, and Piotr Andruszkiewicz. Mul- tilingual entity and relation extraction dataset and model. In Proceedings of the 16th conference of the european chapter of the association for computational linguistics: Ma...

    1946

  52. [69]

    Language models are multilingual chain-of-thought reasoners

    Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush V osoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, et al. Language models are multilingual chain-of-thought reasoners. arXiv preprint arXiv:2210.03057,

    arXiv

  53. [70]

    Exploring prompt engineering: A systematic review with swot analysis

    Aditi Singh, Abul Ehtesham, Gaurav Kumar Gupta, Nikhil Kumar Chatta, Saket Kumar, and Tala Ta- laei Khoei. Exploring prompt engineering: A systematic review with swot analysis. arXiv preprint arXiv:2410.12843,

    arXiv

  54. [71]

    Evaluating gender bias in machine translation

    Gabriel Stanovsky, Noah A Smith, and Luke Zettlemoyer. Evaluating gender bias in machine translation. arXiv preprint arXiv:1906.00591,

    1906 arXiv

  55. [72]

    Multilingual llms are better cross-lingual in-context learners with alignment

    Eshaan Tanwar, Subhabrata Dutta, Manish Borthakur, and Tanmoy Chakraborty. Multilingual llms are better cross-lingual in-context learners with alignment. arXiv preprint arXiv:2305.05940,

    arXiv

  56. [73]

    Legal prompt engineering for multilingual legal judgement prediction

    Dietrich Trautmann, Alina Petrova, and Frank Schilder. Legal prompt engineering for multilingual legal judgement prediction. arXiv preprint arXiv:2212.02199,

    arXiv

  57. [74]

    A survey of prompt engineering methods in large language models for different nlp tasks

    Shubham Vatsal and Harsh Dubey. A survey of prompt engineering methods in large language models for different nlp tasks. arXiv preprint arXiv:2407.12994,

    arXiv

  58. [76]

    A large-scale chinese short-text conversation dataset

    54 Yida Wang, Pei Ke, Yinhe Zheng, Kaili Huang, Yong Jiang, Xiaoyan Zhu, and Minlie Huang. A large-scale chinese short-text conversation dataset. In Natural Language Processing and Chinese Computing: 9th CCF International Conference, NLPCC 2020, Zhengzhou, China, October 14–18...

    2020

  59. [77]

    Crossweigh: Training named entity tagger from imperfect annotations

    Zihan Wang, Jingbo Shang, Liyuan Liu, Lihao Lu, Jiacheng Liu, and Jiawei Han. Crossweigh: Training named entity tagger from imperfect annotations. arXiv preprint arXiv:1909.01441,

    1909 arXiv

  60. [78]

    Chatie: Zero-shot information extraction via chatting with chatgpt

    Xiang Wei, Xingyu Cui, Ning Cheng, Xiaobin Wang, Xin Zhang, Shen Huang, Pengjun Xie, Jinan Xu, Yufeng Chen, Meishan Zhang, et al. Chatie: Zero-shot information extraction via chatting with chatgpt. arXiv preprint arXiv:2302.10205,

    arXiv

  61. [79]

    Ernie-gen: An en- hanced multi-flow pre-training and fine-tuning framework for natural language generation.arXiv preprint arXiv:2001.11314,

    Dongling Xiao, Han Zhang, Yukun Li, Yu Sun, Hao Tian, Hua Wu, and Haifeng Wang. Ernie-gen: An en- hanced multi-flow pre-training and fine-tuning framework for natural language generation.arXiv preprint arXiv:2001.11314,

    2001 arXiv

  62. [80]

    mt5: A massively multilingual pre-trained text-to-text transformer

    Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. mt5: A massively multilingual pre-trained text-to-text transformer. arXiv preprint arXiv:2010.11934,

    2010 arXiv

  63. [81]

    Human-in-the-loop machine translation with large language model

    Xinyi Yang, Runzhe Zhan, Derek F Wong, Junchao Wu, and Lidia S Chao. Human-in-the-loop machine translation with large language model. arXiv preprint arXiv:2310.08908,

    arXiv

  64. [82]

    Paws-x: A cross-lingual adversarial dataset for paraphrase identification

    Yinfei Yang, Yuan Zhang, Chris Tar, and Jason Baldridge. Paws-x: A cross-lingual adversarial dataset for paraphrase identification. arXiv preprint arXiv:1908.11828,

    1908 arXiv

  65. [83]

    Prompting multilingual large language models to generate code-mixed texts: The case of south east asian languages

    Zheng-Xin Yong, Ruochen Zhang, Jessica Zosa Forde, Skyler Wang, Arjun Subramonian, Holy Lovenia, Samuel Cahyawijaya, Genta Indra Winata, Lintang Sutawika, Jan Christian Blaise Cruz, et al. Prompting multilingual large language models to generate code-mixed texts: The case of s...

    arXiv

  66. [84]

    Cross-lingual cross-temporal summarization: Dataset, models, evaluation

    Ran Zhang, Jihed Ouni, and Steffen Eger. Cross-lingual cross-temporal summarization: Dataset, models, evaluation. arXiv preprint arXiv:2306.12916, 2023a. Ruochen Zhang, Samuel Cahyawijaya, Jan Christian Blaise Cruz, Genta Indra Winata, and Alham Fikri Aji. Multilingual large l...

    2004 arXiv

  67. [2006]

    Mlqa: Evaluating cross- lingual extractive question answering

    Patrick Lewis, Barlas O ˘guz, Ruty Rinott, Sebastian Riedel, and Holger Schwenk. Mlqa: Evaluating cross- lingual extractive question answering. arXiv preprint arXiv:1910.07475,

    1910 arXiv

  68. [2010]

    Decomposed prompt- ing for machine translation between related languages using large language models

    Ratish Puduppully, Anoop Kunchukuttan, Raj Dabre, Ai Ti Aw, and Nancy F Chen. Decomposed prompt- ing for machine translation between related languages using large language models. arXiv preprint arXiv:2305.13085, 2023a. Ratish Puduppully, Anoop Kunchukuttan, Raj Dabre, Aiti Aw...

    2023 arXiv

  69. [2016]

    Ace 2005 multilingual training corpus

    Christopher Walker, Stephanie Strassel, Julie Medero, and Kazuaki Maeda. Ace 2005 multilingual training corpus. (No Title),

    2005

  70. [2018]

    No language left behind: Scaling human-centered machine translation

    Marta R Costa-Juss`a, James Cross, Onur C ¸ elebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672,

    arXiv

  71. [2019]

    Buffet: Benchmarking large language models for few-shot cross-lingual transfer

    Akari Asai, Sneha Kudugunta, Xinyan Velocity Yu, Terra Blevins, Hila Gonen, Machel Reid, Yulia Tsvetkov, Sebastian Ruder, and Hannaneh Hajishirzi. Buffet: Benchmarking large language models for few-shot cross-lingual transfer. arXiv preprint arXiv:2305.14857,

    arXiv

  72. [2020]

    Unsupervised domain clusters in pretrained language models

    Roee Aharoni and Yoav Goldberg. Unsupervised domain clusters in pretrained language models. arXiv preprint arXiv:2004.02105,

    2004 arXiv

  73. [2021]

    Indicxnli: Evaluating multilingual inference for indian languages

    Divyanshu Aggarwal, Vivek Gupta, and Anoop Kunchukuttan. Indicxnli: Evaluating multilingual inference for indian languages. arXiv preprint arXiv:2204.08776,

    arXiv

  74. [2022]

    Lince: A centralized benchmark for linguistic code- switching evaluation

    Gustavo Aguilar, Sudipta Kar, and Thamar Solorio. Lince: A centralized benchmark for linguistic code- switching evaluation. arXiv preprint arXiv:2005.04322,

    2005 arXiv

  75. [2023]

    Tico-19: the translation initiative for covid-19

    Antonios Anastasopoulos, Alessandro Cattelan, Zi-Yi Dou, Marcello Federico, Christian Federman, Dmitriy Genzel, Francisco Guzm ´an, Junjie Hu, Macduff Hughes, Philipp Koehn, et al. Tico-19: the translation initiative for covid-19. arXiv preprint arXiv:2007.01788,

    2007 arXiv

  76. [2024]

    Americasnli: Evaluating zero-shot natural language understanding of pretrained multilingual models in truly low-resource languages.arXiv preprint arXiv:2104.08726,

    Abteen Ebrahimi, Manuel Mager, Arturo Oncevay, Vishrav Chaudhary, Luis Chiruzzo, Angela Fan, John Ortega, Ricardo Ramos, Annette Rios, Ivan Meza-Ruiz, et al. Americasnli: Evaluating zero-shot natural language understanding of pretrained multilingual models in truly low-resourc...

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.