Pith. sign in

REVIEW 3 major objections 4 minor 53 references

Exploring In-context Example Generation for Machine Translation

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A language model can generate its own in-context translation examples without any human-annotated pairs, and those self-made demonstrations improve low-resource translation quality.

desk verdict A practical, mostly sound recipe for self-generating MT demonstrations in low-resource directions; the central claim holds up, but the paper has an internal text-table contradiction and hides the MMR weight. read the letter →

arxiv 2506.00507 v1 pith:PW6JI2Q6 submitted 2025-05-31 cs.CL

Dohyun Lee , Seungil Chad Lee , Chanwoo Yang , Yujin Baek , Jaegul Choo This is my paper
classification cs.CL
keywords in-contextlearningmachinetranslationlow-resourcelanguagesdemonstrationgenerationmaximalmarginalrelevancefew-shotpromptingzero-shot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Machine translation into low-resource languages usually needs parallel sentence pairs for in-context examples, but such paired data barely exists. This paper proposes Demonstration Augmentation for Translation (DAT), a method in which the translation model itself writes source sentences that resemble the user's query while remaining different from one another, filters the most relevant and diverse few, and translates those sentences with the same model to build its own demonstration pairs. The claim is that these self-generated, query-tailored examples improve translation quality over zero-shot translation in most of the five tested English-to-low-resource settings, and even outperform few-shot prompting with fixed human-curated pairs. If true, low-resource translation would no longer require a human-annotated demonstration pool, and the same trick could bootstrap reusable data as translations accumulate at test time.

What carries the argument

The carrying mechanism is a two-stage demonstration factory. A source generator paraphrases the user query under explicit relevance and diversity instructions, then a Maximal Marginal Relevance (MMR) filter scores each candidate by averaged n-gram overlap with the query ($\alpha(q,x_i)$) minus a redundancy penalty that averages overlap with already selected sentences. This produces k source sentences; the same LLM then acts as target generator, producing zero-shot translations of those sources. The resulting pairs are query-specific on the source side, which the paper argues matters more than the intrinsic quality of fixed pairs.

What would settle it

Compare DAT against a version where the k target sentences are replaced by deliberately wrong translations of the same generated source sentences: if the two perform alike, then target-side quality, not source-side relevance, is carrying the result.

Watch

Extended reading notes

Core claim

The paper's central discovery is that an LLM can bootstrap its own in-context demonstrations for translation without any external parallel data. Given one source-language query, the LLM is prompted to produce m paraphrastic sentences that are both relevant to the query and mutually diverse; an MMR-style filter selects the k most relevant, least redundant ones; and the same LLM translates each selected sentence into the target language. Those k self-constructed source-target pairs are then placed in a few-shot prompt before the query. Across English-to-Nepali, Khmer, Pashto, Zulu, and Swahili, DAT improves COMET scores relative to zero-shot translation in most cases, with the clearest gains for Nepali (about 2.8 points on Llama-3.1-8B). The paper also reports a counterintuitive result: high-quality fixed human pairs can backfire, such as a 21.6-point drop for English-to-Khmer on Llama-3.1-70B, which it attributes to missing relevance to the query.

Load-bearing premise

The method assumes that the LLM's own translations of the sentences it generates are reliable enough to serve as correct target examples, and the paper checks this only with an automatic quality estimator rather than human judgment.

Editorial extensions

If this is right

  • DAT gives better COMET scores than zero-shot translation in most tested low-resource language pairs, so an LLM can extract translation gains from its own generation without human-annotated examples.
  • In the fixed-pair setting, DAT outperforms few-shot prompting with human-curated pairs in most languages because DAT's generated sources are more relevant to the query.
  • Fixed human pairs that are irrelevant to the query can actively hurt translation, as in the Khmer backfire, so relevance should be checked before any static demonstration pool is used.
  • Accumulating generated pairs into a pool and retrieving them with R-BM25 improves with more seed inputs, though it does not yet match per-query generation.
  • Using more unfiltered generated demonstrations (10 rather than 4) can lower quality, so the filtering step is doing real work rather than merely saving tokens.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, DAT's source-side paraphrasing suggests a cheap way to create domain-adapted demonstrations: because the generated paraphrases inherit terms from the user query, the method may transfer to technical or dialectal text without retraining.
  • The backfire result implies that a small static human-curated demonstration set can be worse than no demonstrations when the query distribution drifts; a testable extension is to measure DAT against retrieval from a human pool whose topics overlap with the test set to find the crossover point.
  • One could test the paper's implicit claim that target-side quality is less important than source-side relevance by generating targets with a stronger, weaker, or deliberately corrupted translation model and checking whether DAT's gains persist.
  • The accumulation experiment points toward a fully unsupervised data flywheel: generated pairs could eventually fine-tune a dedicated translation model, a direction the paper names only in discussion.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Demonstration Augmentation for Translation (DAT), a method for generating in-context examples for LLM-based machine translation without relying on human-annotated parallel data. Given a user query, DAT prompts an LLM to generate several diverse and relevant source-language sentences, filters them via maximal marginal relevance (MMR), translates the selected sentences with the same LLM, and uses the resulting source-target pairs as few-shot demonstrations for translating the query. Experiments on English-to-Nepali, Khmer, Pashto, Zulu, and Swahili using Llama-3.1-8B and Llama-3.1-70B are reported, with COMET and reference-free COMET scores. The authors report that DAT improves over zero-shot translation 'in most' low-resource languages, that fixed human-curated pairs can backfire (notably for Khmer with Llama-3.1-70B), and that an accumulation variant can build a reusable demonstration pool at reduced computational cost. The implementation is publicly available.

Significance. The paper tackles a real and timely problem: enabling in-context learning for machine translation in low-resource languages where human-annotated demonstration pairs are unavailable. The idea of having the LLM generate its own demonstrations is genuinely novel in the context of MT and could be practically useful as a bootstrapping technique. The experimental setup is reasonable for a first exploration, and the authors include some positive elements: publicly released code, significance testing with compare-mt, ablations on the number of generated versus selected demonstrations, and a surface-level qualitative example. However, the central claim is expressed too strongly relative to the data, a key diversity mechanism is contradicted by the paper's own metrics, and a core hyperparameter is unreported, weakening both the internal consistency and reproducibility of the contribution.

major comments (3)
  1. [§5.1, Table 1] The sentence 'In Table 1, DAT without fixed pairs outperforms the few-shot approach in translation quality for all languages except Zulu' is factually incorrect for Llama-3.1-8B. For that model, DAT without fixed pairs is worse than the fixed-pair few-shot baseline on Nepali (74.9 vs 75.9), Khmer (64.4 vs 65.0), Pashto (54.6 vs 57.9), and Zulu (22.3 vs 24.7). The claim holds only for Llama-3.1-70B, and even then Zulu is an exception. This overgeneralization underlies the subsequent research question about high-quality fixed pairs versus query-relevant moderate-quality pairs; it needs to be corrected and the model-size interaction discussed.
  2. [§5.2, Table 2 and Eq. (3)] The reported Uniformity scores conflict with the method's stated diversity objective. DAT's mean pairwise source-side n-gram overlap is 24.1, far higher than the Fixed set's 2.8, indicating that DAT's selected sources are highly similar to one another. The paper acknowledges a relevance/uniformity interdependence but does not reconcile it with the claim that MMR filtering promotes diversity. Moreover, the diversity weight λ in Eq. (3) is never specified anywhere in the paper, so the filtering procedure is not reproducible and the effective trade-off cannot be assessed. Please report λ for each experiment and provide an analysis (e.g., a small sweep) of how λ affects the relevance-diversity balance and downstream COMET.
  3. [§3.3 and Table 2] The method's success rests on the assumption that the LLM's zero-shot translations of the generated source sentences are of sufficient accuracy to serve as demonstrations. The only support offered is the reference-free COMET 'Quality' column in Table 2, which is a model-based estimator rather than a human judgment. Because the downstream gains are attributed to these self-generated pairs, the paper would be substantially strengthened by a human evaluation of a sample of generated pairs, or by an experiment that varies the target generator (e.g., a fine-tuned NMT model) to quantify how sensitive DAT's gains are to pair fidelity. The Section 6 discussion acknowledges this possibility but does not test it.
minor comments (4)
  1. [Contributions, §1] The bullet point 'Experimentes show that DAT boosts the translation quality...' contains a typo; it should read 'Experiments'.
  2. [Figure 3] The y-axis of Figure 3 is not labeled; the text mentions 'COMET scores', but the axis itself should carry that label for clarity.
  3. [§5.2] The definition of Uniformity is vague ('evaluates the same averaged score among the source sides of different in-context examples'); a precise formula, e.g., the average pairwise α(x_i, x_j) over selected sources, would be easier to interpret.
  4. [Abstract and §7] The abstract and conclusion claim that DAT 'achieves superior translation quality' without noting the Zulu exception where DAT is significantly worse than zero-shot (Table 1). A qualifier such as 'in most of the evaluated low-resource languages' would align the claims with the data.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: DAT's central comparison is empirical against a zero-shot baseline, no parameter is fitted to the test metric, and no load-bearing self-citation appears.

full rationale

The paper's derivation chain is an empirical pipeline, not a formal derivation whose conclusion is built into its premises. The LLM generates source sentences (Section 3.1), filters them using n-gram relevance and MMR diversity scores defined in Equations (1)-(3), generates target sides with the same LLM (Section 3.3), and then translates the query with those demonstrations (Section 3.4). The central claim that DAT 'achieves superior translation quality compared to the baselines' is a comparative empirical outcome evaluated against a zero-shot baseline. The result is not forced by construction: in Table 1, DAT is worse than zero-shot for English-to-Zulu with Llama-3.1-8B (22.3 vs. 23.3), and gains vary by language and model. The MMR weight lambda is a hyperparameter, not a parameter fitted to the test COMET scores, and the relevance/diversity criteria are imported from external prior work rather than from a self-citation chain. The fact that the same LLM generates the demonstrations, translates the source sides, and translates the query is self-referential in a broad sense, but it does not logically guarantee a positive result; the outcome could in principle be negative, and in some settings it is. The reliance on reference-free COMET to assess target-side quality in Table 2 is an empirical validity concern, not a circularity: the metric is an external model-based estimate, and the paper explicitly acknowledges in Section 6 that a fine-tuned NMT model 'could produce higher-fidelity pairs,' leaving that as future work. No uniqueness theorem, no author-imported ansatz, and no fitted-input-called-prediction pattern is present. The manuscript is therefore self-contained against external benchmarks for the purpose of the circularity analysis, and no specific reduction of the central claim to its inputs can be exhibited.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

DAT introduces no new entities. It depends on the pretrained LLM as the sole source of source-side paraphrases, target-side translations, and final query translation, so the central uncharged assumption is that the model's own output is good enough to bootstrap its own few-shot performance.

free parameters (3)
  • lambda = not reported
    MMR diversity weight in Eq. (3), chosen by hand but never specified in Section 4.3.
  • m = 10
    Number of initial source sentences generated before filtering (Section 4.3).
  • k = 4
    Number of demonstrations kept after filtering (Section 4.3).
assumptions (3)
  • domain assumption The LLM's zero-shot translations of generated source sentences are reliable enough to serve as target-side demonstrations
    Introduced in Section 3.3; only checked with reference-free COMET in Table 2, not with human evaluation of pair quality.
  • domain assumption N-gram overlap between query and source-side examples is a valid relevance signal for ICL in MT
    Equations (1) and (2) define relevance via n-gram recall; this proxy comes from prior work but is assumed to transfer to the low-resource setting.
  • domain assumption The LLM has sufficient pretraining competence in the five target languages for both generation and translation
    Section 8 limits the method to English-to-low-resource and to Llama-3.1 because most open LLMs are English-centric.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploring In-context Example Generation for Machine Translation." pith.science (2026). https://pith.science/paper/PW6JI2Q6

@misc{pith2026250600507,
  author       = {Pith},
  title        = {Pith review of: Exploring In-context Example Generation for Machine Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PW6JI2Q6}},
  note         = {Machine review of arXiv:2506.00507}
}
read the original abstract

Large language models (LLMs) have demonstrated strong performance across various tasks, leveraging their exceptional in-context learning ability with only a few examples. Accordingly, the selection of optimal in-context examples has been actively studied in the field of machine translation. However, these studies presuppose the presence of a demonstration pool with human-annotated pairs, making them less applicable to low-resource languages where such an assumption is challenging to meet. To overcome this limitation, this paper explores the research direction of in-context example generation for machine translation. Specifically, we propose Demonstration Augmentation for Translation (DAT), a simple yet effective approach that generates example pairs without relying on any external resources. This method builds upon two prior criteria, relevance and diversity, which have been highlighted in previous work as key factors for in-context example selection. Through experiments and analysis on low-resource languages where human-annotated pairs are scarce, we show that DAT achieves superior translation quality compared to the baselines. Furthermore, we investigate the potential of progressively accumulating generated pairs during test time to build and reuse a demonstration pool. Our implementation is publicly available at https://github.com/aiclaudev/DAT.

Figures

Figures reproduced from arXiv: 2506.00507 by the authors.

Figure 1
Figure 1. Motivation. (a) Previous works on LLM￾based in-context learning for translation have primarily focused on selecting in-context examples from paired dataset in high-resource languages. (b) However, in the absence of a paired dataset for low-resource languages, how can in-context learning be applied? duces the training cost on large datasets and allows rapid adaptation to new domains or problems with￾out extensive fin… view at source ↗
Figure 2
Figure 2. An overview of our proposed method. (1) Upon receiving a translation request for the user’s query, the LLM generates m source-side sentences that satisfy both relevance and diversity constraints. (2) A relevant sentence that minimizes redundancy with previously chosen source sentences is iteratively selected and appended to the candidate pool. This process is iterated k times. (3) The LLM then translates each select… view at source ↗
Figure 3
Figure 3. This figure illustrates English-to-Khmer trans [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: This experiment evaluates the accumulation setting when translating English into Zulu and Swahili using [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Prompt format used to generate source sentences with two criteria. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Prompt format used to translate a source sentence. [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

53 extracted references · 22 canonical work pages

  1. [1]

    Sweta Agrawal, Chunting Zhou, Mike Lewis, Luke Zettlemoyer, and Marjan Ghazvininejad. 2023. https://doi.org/10.18653/v1/2023.findings-acl.564 In-context examples selection for machine translation . In Findings of the Association for Computational Linguistics: ACL 2023, pages 8857--8873, Toronto, Canada. Association for Computational Linguistics

    doi:10.18653/v1/2023.findings-acl.564 2023

  2. [2]

    Maxime Bouthors, Josep Crego, and Fran c ois Yvon. 2024. https://doi.org/10.18653/v1/2024.findings-naacl.190 Retrieving examples from memory for retrieval augmented neural machine translation: A systematic comparison . In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3022--3039, Mexico City, Mexico. Association for Computati...

    work page doi:10.18653/v1/2024.findings-naacl.190 2024

  3. [3]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gr...

    2020

  4. [4]

    Jaime Carbonell and Jade Goldstein. 1998. https://doi.org/10.1145/290941.291025 The use of mmr, diversity-based reranking for reordering documents and producing summaries . In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '98, page 335–336, New York, NY, USA. Association for C...

    arXiv 1998

  5. [5]

    Stephanie C. Y. Chan, Adam Santoro, Andrew K. Lampinen, Jane X. Wang, Aaditya K. Singh, Pierre H. Richemond, James L. McClelland, and Felix Hill. 2022. http://papers.nips.cc/paper\_files/paper/2022/hash/77c6ccacfd9962e2307fc64680fc5ace-Abstract-Conference.html Data distributional properties drive emergent in-context learning in transformers . In Advances ...

    work page 2022

  6. [6]

    Wei - Lin Chen, Cheng - Kuang Wu, Yun - Nung Chen, and Hsin - Hsi Chen. 2023. https://doi.org/10.18653/V1/2023.EMNLP-MAIN.968 Self-icl: Zero-shot in-context learning with self-generated demonstrations . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023 , pages 15651--15662...

    work page doi:10.18653/v1/2023.emnlp-main.968 2023

  7. [7]

    Silei Cheng, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan Boyd-Graber, and Lijuan Wang. 2023. https://www.microsoft.com/en-us/research/publication/prompting-gpt-3-to-be-reliable/ Prompting gpt-3 to be reliable . In International Conference on Learning Representations (ICLR 23)

    work page 2023

  8. [8]

    Xin Cheng, Shen Gao, Lemao Liu, Dongyan Zhao, and Rui Yan. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.235 Neural machine translation with contrastive translation memories . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3591--3601, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics

    work page doi:10.18653/v1/2022.emnlp-main.235 2022

Show all 53 references
  1. [9]

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sashank Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vi...

    2023

  2. [10]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zha...

    2024 arXiv

  3. [11]

    Abdellah El Mekki and Muhammad Abdul-Mageed. 2025. https://aclanthology.org/2025.findings-naacl.238/ Effective self-mining of in-context examples for unsupervised machine translation with LLM s . In Findings of the Association for Computational Linguistics: NAACL 2025, pages 4...

    2025

  4. [12]

    Naman Goyal, Cynthia Gao, Vishrav Chaudhary, Peng-Jen Chen, Guillaume Wenzek, Da Ju, Sanjana Krishnan, Marc ' Aurelio Ranzato, Francisco Guzm \'a n, and Angela Fan. 2022. https://doi.org/10.1162/tacl_a_00474 The F lores-101 evaluation benchmark for low-resource and multilingua...

    2022 doi

  5. [13]

    Yaru Hao, Yutao Sun, Li Dong, Zhixiong Han, Yuxian Gu, and Furu Wei. 2022. https://doi.org/10.48550/ARXIV.2212.06713 Structured prompting: Scaling in-context learning to 1, 000 examples . CoRR, abs/2212.06713

    2022 doi arXiv

  6. [14]

    Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. 2023. https://arxiv.org/abs/2302.09210 How good are gpt models at machine translation? a comprehensive evaluation . Preprint, arX...

    2023 arXiv

  7. [15]

    Baijun Ji, Xiangyu Duan, Zhenyu Qiu, Tong Zhang, Junhui Li, Hao Yang, and Min Zhang. 2024. https://aclanthology.org/2024.lrec-main.1337 Submodular-based in-context example selection for llms-based machine translation . In Proceedings of the 2024 Joint International Conference ...

    2024

  8. [16]

    Wenxiang Jiao, Wenxuan Wang, Jen - tse Huang, Xing Wang, and Zhaopeng Tu. 2023. https://doi.org/10.48550/ARXIV.2301.08745 Is chatgpt A good translator? A preliminary study . CoRR, abs/2301.08745

    2023 doi arXiv

  9. [17]

    Hyuhng Joon Kim, Hyunsoo Cho, Junyeob Kim, Taeuk Kim, Kang Min Yoo, and Sang - goo Lee. 2022. https://doi.org/10.48550/ARXIV.2206.08082 Self-generated in-context learning: Leveraging auto-regressive language models as a demonstration generator . CoRR, abs/2206.08082

    2022 doi arXiv

  10. [18]

    Sai Koneru, Miriam Exel, Matthias Huck, and Jan Niehues. 2024. https://doi.org/10.18653/V1/2024.NAACL-LONG.148 Contextual refinement of translations: Large language models for sentence and document-level post-editing . In Proceedings of the 2024 Conference of the North America...

    2024 doi

  11. [19]

    Aswanth Kumar, Ratish Puduppully, Raj Dabre, and Anoop Kunchukuttan. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.519 CTQS corer: Combining multiple features for in-context example selection for machine translation . In Findings of the Association for Computational Li...

    2023 doi

  12. [20]

    Rui Li, Guoyin Wang, and Jiwei Li. 2024. Are human-generated demonstrations necessary for in-context learning? The Twelfth International Conference on Learning Representations

    2024

  13. [21]

    Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O ' Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, M...

    2022 doi

  14. [22]

    Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2022. https://doi.org/10.18653/v1/2022.deelio-1.10 What makes good in-context examples for GPT -3? In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extr...

    2022 doi

  15. [23]

    Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2022. https://doi.org/10.18653/V1/2022.ACL-LONG.556 Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity . In Proceedings of the 60th Annual Meeting of th...

    2022 doi

  16. [24]

    Xinxi Lyu, Sewon Min, Iz Beltagy, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. https://doi.org/10.18653/V1/2023.ACL-LONG.129 Z-ICL: zero-shot in-context learning with pseudo-demonstrations . In Proceedings of the 61st Annual Meeting of the Association for Computational Lin...

    2023 doi

  17. [25]

    Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.759 Rethinking the role of demonstrations: What makes in-context learning work? In Proceedings of the 2022 Conference on ...

    2022 doi

  18. [26]

    Marius Mosbach, Tiago Pimentel, Shauli Ravfogel, Dietrich Klakow, and Yanai Elazar. 2023. https://doi.org/10.18653/V1/2023.FINDINGS-ACL.779 Few-shot fine-tuning vs. in-context learning: A fair comparison and evaluation . In Findings of the Association for Computational Linguis...

    2023 doi

  19. [27]

    Yasmin Moslem, Rejwanul Haque, John Kelleher, and Andy Way. 2022. https://aclanthology.org/2022.amta-research.2 Domain-specific text generation for machine translation . In Proceedings of the 15th biennial conference of the Association for Machine Translation in the Americas (...

    2022

  20. [28]

    Kelleher, Rejwanul Haque, and Andy Way

    Yasmin Moslem, Gianfranco Romani, Mahdi Molaei, John D. Kelleher, Rejwanul Haque, and Andy Way. 2023. https://doi.org/10.18653/V1/2023.WMT-1.82 Domain terminology integration into machine translation: Leveraging large language models . In Proceedings of the Eighth Conference o...

    2023 doi

  21. [29]

    Graham Neubig, Zi - Yi Dou, Junjie Hu, Paul Michel, Danish Pruthi, Xinyi Wang, and John Wieting. 2019. http://arxiv.org/abs/1903.07926 compare-mt: A tool for holistic comparison of language generation systems . CoRR, abs/1903.07926

    2019 arXiv

  22. [30]

    NLLB Team , Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia-Gonzalez, Pran...

    2022

  23. [31]

    OpenAI. 2023. https://doi.org/10.48550/ARXIV.2303.08774 GPT-4 technical report . CoRR, abs/2303.08774

    2023 doi arXiv

  24. [32]

    Vikas Raunak, Arul Menezes, Matt Post, and Hany Hassan. 2023 a . https://doi.org/10.18653/V1/2023.ACL-SHORT.90 Do gpts produce less literal translations? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), ACL 20...

    2023 doi

  25. [33]

    Vikas Raunak, Amr Sharaf, Yiren Wang, Hany Awadalla, and Arul Menezes. 2023 b . https://doi.org/10.18653/v1/2023.findings-emnlp.804 Leveraging GPT -4 for automatic translation post-editing . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 12009-...

    2023 doi

  26. [34]

    Ricardo Rei, Jos \'e G. C. de Souza, Duarte Alves, Chrysoula Zerva, Ana C Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and Andr \'e F. T. Martins. 2022 a . https://aclanthology.org/2022.wmt-1.52 COMET -22: Unbabel- IST 2022 submission for the metrics shared task . In ...

    2022

  27. [35]

    Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, Jos \'e G

    Ricardo Rei, Marcos Treviso, Nuno M. Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, Jos \'e G. C. de Souza, Taisiya Glushkova, Duarte Alves, Luisa Coheur, Alon Lavie, and Andr \'e F. T. Martins. 2022 b . https://aclanthology.org/2022.wmt-1.60 C omet K iwi: IST -u...

    2022

  28. [36]

    Ohad Rubin, Jonathan Herzig, and Jonathan Berant. 2022. https://doi.org/10.18653/v1/2022.naacl-main.191 Learning to retrieve prompts for in-context learning . In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: ...

    2022 doi

  29. [37]

    Seongjin Shin, Sang - Woo Lee, Hwijeen Ahn, Sungdong Kim, HyoungSeok Kim, Boseop Kim, Kyunghyun Cho, Gichang Lee, Woo - Myoung Park, Jung - Woo Ha, and Nako Sung. 2022. https://doi.org/10.18653/V1/2022.NAACL-MAIN.380 On the effect of pretraining corpora on in-context learning ...

    2022 doi

  30. [38]

    Suzanna Sia and Kevin Duh. 2023. https://aclanthology.org/2023.mtsummit-research.15 In-context learning as maintaining coherency: A study of on-the-fly machine translation using large language models . In Proceedings of Machine Translation Summit XIX, Vol. 1: Research Track, p...

    2023

  31. [39]

    Yi Su, Yunpeng Tai, Yixin Ji, Juntao Li, Bowen Yan, and Min Zhang. 2024. Demonstration augmentation for zero-shot in-context learning. arXiv preprint arXiv:2406.01224

    2024 arXiv

  32. [40]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie - Anne Lachaux, Timoth \' e e Lacroix, Baptiste Rozi \` e re, Naman Goyal, Eric Hambro, Faisal Azhar, Aur \' e lien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023 a . https://doi.org/10....

    2023 doi arXiv

  33. [41]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton - Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu,...

    2023 doi arXiv

  34. [42]

    David Vilar, Markus Freitag, Colin Cherry, Jiaming Luo, Viresh Ratnakar, and George Foster. 2023. https://doi.org/10.18653/v1/2023.acl-long.859 Prompting P a LM for translation: Assessing strategies and performance . In Proceedings of the 61st Annual Meeting of the Association...

    2023 doi

  35. [43]

    Jiayi Wang, David Ifeoluwa Adelani, Sweta Agrawal, Marek Masiak, Ricardo Rei, Eleftheria Briakou, Marine Carpuat, Xuanli He, Sofia Bourhim, Andiswa Bukula, Muhidin Mohamed, Temitayo Olatoye, Tosin Adewumi, Hamam Mokayed, Christine Mwase, Wangui Kimotho, Foutse Yuehgoh, Anuoluw...

    2024 arXiv

  36. [44]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. https://arxiv.org/abs/2201.11903 Chain-of-thought prompting elicits reasoning in large language models . Preprint, arXiv:2201.11903

    2023 arXiv

  37. [45]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020 doi

  38. [46]

    Haoran Xu, Young Jin Kim, Amr Sharaf, and Hany Hassan Awadalla. 2024. https://openreview.net/forum?id=farT6XXntP A paradigm shift in machine translation: Boosting translation performance of large language models . In The Twelfth International Conference on Learning Representations

    2024

  39. [47]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629

    2022 arXiv

  40. [48]

    Armel Randy Zebaze, Beno \^i t Sagot, and Rachel Bawden. 2025. https://aclanthology.org/2025.findings-naacl.68/ In-context example selection via similarity search improves low-resource machine translation . In Findings of the Association for Computational Linguistics: NAACL 20...

    2025

  41. [49]

    Biao Zhang, Barry Haddow, and Alexandra Birch. 2023. https://proceedings.mlr.press/v202/zhang23m.html Prompting large language model for machine translation: A case study . In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA , vol...

    2023

  42. [50]

    Yiming Zhang, Shi Feng, and Chenhao Tan. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.622 Active example selection for in-context learning . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9134--9148, Abu Dhabi, United Arab...

    2022 doi

  43. [51]

    Tony Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. https://api.semanticscholar.org/CorpusID:231979430 Calibrate before use: Improving few-shot performance of language models . In International Conference on Machine Learning

    2021

  44. [52]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  45. [53]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.