Pith. sign in

REVIEW 3 major objections 5 minor 36 references

It's Not a Walk in the Park! Challenges of Idiom Translation in Speech-to-text Systems

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Speech-to-text translation systems show a pronounced, statistically significant performance drop on idiomatic sentences, often falling back to literal translations even in their deepest layers, while text-based machine translation and…

desk verdict Useful first benchmark of idiom translation across MT/SLT/LLM/cascades; central claim holds, but the news-vs-idiom comparison is confounded and one headline claim isn't supported by the paper's own table. read the letter →

arxiv 2506.02995 v1 pith:5FU6ZR7V submitted 2025-06-03 cs.CL

Iuliia Zaitova , Badr M. Abdullah , Wei Xue , Dietrich Klakow , Bernd Möbius , Tania Avgustinova This is my paper
classification cs.CL
keywords idiomtranslationspeech-to-textfigurativelanguagemachineevaluationCOMETmetricDecoderLensanalysiscascadedspeechlargemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to test whether speech-to-text translation (SLT) systems, which take audio directly to translated text, fail on idiomatic expressions more than text-based systems do. The authors compare end-to-end SLT models (Whisper and SeamlessM4T) with text machine translation, large language models, and cascaded ASR-plus-MT pipelines on German-to-English and Russian-to-English idiom and news test sets. The central finding is that SLT systems suffer a much larger quality drop when moving from news to idiomatic data, frequently producing literal word-for-word translations, while text-based systems and large language models preserve figurative meaning more often. The paper interprets this as evidence that end-to-end SLT architectures have a specific weakness with figurative language that cascaded systems partly avoid.

What carries the argument

The evaluation design pairs two complementary instruments. COMET, a neural semantic-similarity metric, scores whole-sentence translation quality on matched news and idiom sets, making the news-versus-idiom gap the headline quantity. DecoderLens, which swaps the final encoder representation for intermediate layer activations and decodes them into text, lets the authors annotate how translation quality evolves layer by layer and pinpoints where literalization appears. Both are applied to the same encoder-decoder models, so the layer-wise failure modes can be tied directly to the COMET drops.

What would settle it

A reader could take non-idiomatic sentences from the same idiom corpus, or add matched non-idiomatic controls from the same source, and re-run the news-versus-idiom comparison; if the COMET gap largely disappears, the paper's attribution of the drop to idiomaticity would be refuted.

Watch

Extended reading notes

Core claim

The paper claims to establish that end-to-end speech-to-text translation systems have a systematic, measurable weakness on idiomatic input that is not explained by automatic speech recognition errors alone. Across both language pairs, direct SLT systems (Whisper Large v3 and SeamlessM4T speech-to-text) score substantially lower on the COMET metric for idiom sentences than for news sentences (for example, Whisper drops from 0.844 to 0.640 on German-to-English), while text-only MT systems, large language models, and cascaded ASR-to-MT/LLM pipelines retain more of the figurative meaning. Layer-wise DecoderLens analysis shows SLT encoders produce meaningful output only in high layers and tend to settle on literal translations even there, whereas text encoders refine toward correct idiomatic output more gradually. The conclusion is that idiom handling requires idiom-specific training strategies and better internal representations for figurative meaning in SLT architectures.

Load-bearing premise

The comparison assumes the news and idiom test sets differ mainly in idiomaticity, even though they come from different corpora with different genres and topics; if that assumption fails, the measured performance gap cannot be attributed to idioms alone.

Editorial extensions

If this is right

  • If the central claim holds, end-to-end SLT systems need idiom-specific training data or objectives rather than relying on general scaling.
  • For practical speech translation of content likely to contain idioms, cascaded ASR plus text MT or LLM pipelines are the safer choice.
  • Literal translation is the dominant failure mode across both SLT and MT, so progress will require models to recognize figurative status before decoding.
  • The idiom gap is consistently larger for German than for Russian, suggesting language-specific idiom difficulty that future benchmarks should separate.
  • COMET scores and human annotation align in ranking systems, so automatic semantic metrics can be used to track idiom-specific improvements.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test the paper does not run: probe whether the audio encoder's compressed representations lose idiom-specific cues before the decoder starts, which would predict that better acoustic-semantic alignment layers would close the gap.
  • Because the idiom and news sets come from different corpora, part of the measured gap could reflect genre or topic difficulty rather than idiomaticity per se; a matched control set would isolate the idiom effect.
  • The literalization pattern in higher layers suggests a targeted intervention: adding an idiom-recognition auxiliary task at those layers might reduce SLT literal translations more than adding more data.
  • Applying the same news-versus-idiom comparison to more language pairs and to spontaneous (non-TTS) speech would show whether the SLT idiom weakness is universal or tied to the synthetic audio used here.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This manuscript presents a systematic comparison of idiom translation in speech-to-text (SLT) versus text-to-text MT, LLM, and cascaded systems for German→English and Russian→English. It builds a 250-item news set from News Commentary and a 250-item idiom set from Idioms-InContext-MT, synthesizes speech for the text sources, and evaluates the systems with COMET and human annotations, plus DecoderLens layer-wise diagnostics. The central claim is that end-to-end SLT systems (Whisper, SeamlessM4T audio) drop sharply on idiomatic data and often produce literal translations even in later encoder layers, while text-based systems handle idioms better.

Significance. The paper addresses an underexplored problem and provides a useful empirical benchmark: it compares two language pairs, multiple system classes, automatic and human evaluation, and it releases code, evaluation datasets, and annotated subsets. The direct audio-versus-text comparisons within the idiom set are informative and support the existence of an SLT-specific challenge with figurative language. The DecoderLens analysis is an original diagnostic application. The main caveat is that the cross-corpus news-versus-idiom comparison is not controlled, so the strongest causal claim about idiomaticity per se needs reframing or additional evidence.

major comments (3)
  1. [§3.3, Table 2] The news and idiom evaluation sets are constructed independently from different corpora (News Commentary vs. a manually filtered subset of Idioms-InContext-MT) and differ in genre, register, sentence length, and topic; no matching or controlling for these factors is reported. The claim that SLT systems show a 'pronounced performance drop' on idiomatic data therefore confounds idiomaticity with general corpus difficulty. The within-idiom comparisons in §4.2 (audio vs. text on the same idiom items) provide more direct evidence for an SLT-specific weakness, but the cross-corpus drop in Table 2 cannot by itself establish that idioms are the cause. I ask the authors to either add a matched non-idiomatic control condition (e.g., literal uses of the same idiom-containing sentences or genre-matched non-idiomatic sentences) or to restrict the causal claim and state the corpus mismatch explicitly in the Limitations section.
  2. [§4.1, Table 2, Conclusion point 1] The statement that the news-to-idiom gap 'is more pronounced for SLT systems' is not supported by the reported COMET differences. In German, Whisper drops from 0.8437 to 0.6402 (Δ = 0.2035) and NLLB drops from 0.8841 to 0.6749 (Δ = 0.2092); in Russian, Whisper drops from 0.8318 to 0.6916 (Δ = 0.1402) and NLLB drops from 0.8664 to 0.7214 (Δ = 0.1450). No statistical test of an interaction between system type and dataset is provided. The conclusion should be based on the direct audio-vs-text comparisons on the idiom set, where the difference is clear, or on a proper interaction test, not on the unmatched cross-corpus gaps.
  3. [§3.4.2, §5, Figure 3] The human annotation is based on only 50 items per system-domain combination, and no inter-annotator agreement is reported. The DecoderLens analysis in §5 and Figure 3 is therefore built on a small sample and a single adjudicated annotation pass. Reporting agreement, for example Cohen's kappa, and stating the number of unique idiom items versus repeated outputs per model would strengthen the layer-wise claims. This does not invalidate the qualitative pattern, but it limits the strength of the conclusions drawn from the layer-wise distributions.
minor comments (5)
  1. [Abstract] There is a missing space: 'Inthispaper' should be 'In this paper'.
  2. [Limitations] The first sentence of the Limitations section contains a typo: 'Morever' should be 'Moreover'.
  3. [Appendix B, Table 5(b)] In the Russian Idioms table, the standard deviation for Whisper is shown as 'L0.106', which appears to be a typo for '0.106'.
  4. [Table 2] The row labeled 'Seamless (Text MT and LLM)' mixes a system name with a category label; it would be clearer to call it 'SeamlessM4T (text-to-text)' and to list DeepSeek and LLaMA separately, as is already done.
  5. [Figure 2 and Figure 3] The captions do not state the number of annotated examples per bar; adding this information (for example, 'n = 50 per condition') would help readers assess the reliability of the displayed proportions.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: this is an empirical evaluation whose conclusions are measured against external references (COMET, human annotations, DecoderLens), with no fitted parameters or self-cited theoretical results forcing the outcome.

full rationale

The paper is an empirical benchmark study, not a derivation. Its central claims are established by running externally released systems (Whisper, SeamlessM4T, NLLB, LLaMA, DeepSeek) on two evaluation sets and scoring translations with COMET against human references, plus human annotation of outputs and layer-wise DecoderLens inspection. No parameter is fitted to the idiom test set and then renamed as a prediction; no theoretical claim is imported from the authors' own prior work; and no equation defines the target conclusion into existence. The skeptical concern about the news and idiom sets being drawn from different corpora with unmatched genre and difficulty is a validity/comparability threat to the causal interpretation of the COMET gap, not a circularity: the gap is still measured, not constructed, and the paper does not assume the gap in its methodology. The annotation category 'Literal Translation' is defined only for idiom items, but this is a coding-scheme choice for the human evaluation, not a fitted input that predetermines the reported system rankings. Accordingly, the appropriate circularity verdict is 0, with the corpus-matching issue better placed under correctness risk than circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters are fitted in this evaluation; the paper compares pretrained models on fixed test sets. The central claims rest on several domain assumptions about metric validity, synthetic speech, idiom set selection, and DecoderLens interpretability, all of which are standard for the field but unverified within the paper.

assumptions (4)
  • domain assumption COMET (wmt22-comet-da) reliably reflects semantic equivalence for idiomatic translations.
    COMET is the primary automatic metric; the paper cites Song and Xu (2024) for its correlation with human judgments on multiword expressions, but if COMET is biased against non-literal output, the gap would be inflated. Section 3.4.1.
  • domain assumption Synthetic speech with a single female voice is a faithful proxy for natural spoken idioms.
    All audio is produced with Microsoft Edge TTS using a consistent voice. The paper argues prior work supports this, but acknowledges in Limitations that synthetic speech may differ from spontaneous speech. Section 3.3.2.
  • domain assumption The manually filtered idiom set is representative of idioms that challenge translation.
    The authors selected 250 idioms per language requiring non-literal translation, excluding those where literal translation preserves meaning. This makes the set harder and may not reflect the typical difficulty of idioms. Section 3.3.2.
  • domain assumption DecoderLens intermediate outputs faithfully reveal how the model processes meaning.
    The layer-wise analysis assumes that decoded text from intermediate encoder layers reflects the model's internal state; this is the premise of the DecoderLens method cited from Langedijk et al. (2024). Section 5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of It's Not a Walk in the Park! Challenges of Idiom Translation in Speech-to-text Systems." pith.science (2026). https://pith.science/paper/5FU6ZR7V

@misc{pith2026250602995,
  author       = {Pith},
  title        = {Pith review of: It's Not a Walk in the Park! Challenges of Idiom Translation in Speech-to-text Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5FU6ZR7V}},
  note         = {Machine review of arXiv:2506.02995}
}
read the original abstract

Idioms are defined as a group of words with a figurative meaning not deducible from their individual components. Although modern machine translation systems have made remarkable progress, translating idioms remains a major challenge, especially for speech-to-text systems, where research on this topic is notably sparse. In this paper, we systematically evaluate idiom translation as compared to conventional news translation in both text-to-text machine translation (MT) and speech-to-text translation (SLT) systems across two language pairs (German to English, Russian to English). We compare state-of-the-art end-to-end SLT systems (SeamlessM4T SLT-to-text, Whisper Large v3) with MT systems (SeamlessM4T SLT-to-text, No Language Left Behind), Large Language Models (DeepSeek, LLaMA) and cascaded alternatives. Our results reveal that SLT systems experience a pronounced performance drop on idiomatic data, often reverting to literal translations even in higher layers, whereas MT systems and Large Language Models demonstrate better handling of idioms. These findings underscore the need for idiom-specific strategies and improved internal representations in SLT architectures.

Figures

Figures reproduced from arXiv: 2506.02995 by the authors.

Figure 1
Figure 1. An illustrated example of translating a spo [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Distribution of translation output categories across models for German [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Distribution of translation error categories across encoder layers for German [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 17 canonical work pages

  1. [1]

    McCrae, Kenton Murray, Maria Nadejde, Satoshi Nakamura, Matteo Negri, Ha Nguyen, Jan Niehues, Xing Niu, Atul Kr

    Milind Agarwal, Sweta Agrawal, Antonios Anastasopoulos, Luisa Bentivogli, Ond r ej Bojar, Claudia Borg, Marine Carpuat, Roldano Cattoni, Mauro Cettolo, Mingda Chen, William Chen, Khalid Choukri, Alexandra Chronopoulou, Anna Currey, Thierry Declerck, Qianqian Dong, Kevin Duh, Yannick Est \`e ve, Marcello Federico, Souhir Gahbiche, Barry Haddow, Benjamin Hs...

    2023

  2. [2]

    Andrei Avram, Verginica Barbu Mititelu, and Dumitru-Clementin Cercel. 2023. https://doi.org/10.18653/v1/2023.mwe-1.4 R omanian multiword expression detection using multilingual adversarial training and lateral inhibition . In Proceedings of the 19th Workshop on Multiword Expressions (MWE 2023), pages 7--13, Dubrovnik, Croatia. Association for Computationa...

    work page doi:10.18653/v1/2023.mwe-1.4 2023

  3. [3]

    Bismarck Bamfo Odoom, Nathaniel Robinson, Elijah Rippeth, Luis Tavarez-Arce, Kenton Murray, Matthew Wiesner, Paul McNamee, Philipp Koehn, and Kevin Duh. 2024. https://aclanthology.org/2024.amta-research.15/ Can synthetic speech improve end-to-end conversational speech translation? In Proceedings of the 16th Conference of the Association for Machine Transl...

    work page 2024

  4. [4]

    Loïc Barrault, Yu-An Chung, Mariano Coria Meglioli, David Dale, Ning Dong, Mark Duppenthaler, Paul-Ambroise Duquenne, Brian Ellis, Hady Elsahar, Justin Haaheim, John Hoffman, Min-Jae Hwang, Hirofumi Inaguma, Christopher Klaiber, Ilia Kulikov, Pengwei Li, Daniel Licht, Jean Maillard, Ruslan Mavlyutov, Alice Rakotoarison, Kaushik Ram Sadagopan, Abinesh Rama...

    arXiv 2023

  5. [5]

    Loïc Barrault, Yu-An Chung, Mariano Coria Meglioli, David Dale, Ning Dong, Paul-Ambroise Duquenne, Hady Elsahar, Hongyu Gong, Kevin Heffernan, John Hoffman, Christopher Klaiber, Pengwei Li, Daniel Licht, Jean Maillard, Alice Rakotoarison, Kaushik Ram Sadagopan, Guillaume Wenzek, Ethan Ye, Bapi Akula, Peng-Jen Chen, Naji El Hachem, Brian Ellis, Gabriel Mej...

    2025

  6. [6]

    Christos Baziotis, Prashant Mathur, and Eva Hasler. 2023. https://doi.org/10.18653/v1/2023.eacl-main.267 Automatic evaluation and analysis of idioms in neural machine translation . In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3682--3700, Dubrovnik, Croatia. Association for Computatio...

    doi:10.18653/v1/2023.eacl-main.267 2023

  7. [7]

    Joanne Boisson, Jose Camacho-Collados, and Luis Espinosa-Anke. 2022. https://doi.org/10.18653/v1/2022.semeval-1.20 C ardiff NLP -metaphor at S em E val-2022 task 2: Targeted fine-tuning of transformer-based language models for idiomaticity detection . In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 169--177, ...

    work page doi:10.18653/v1/2022.semeval-1.20 2022

  8. [8]

    Van-Tuan Bui and Agata Savary. 2024. https://aclanthology.org/2024.lrec-main.374 Cross-type F rench multiword expression identification with pre-trained masked language models . In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 4198--4204, Torino, Italia. ELR...

    work page 2024

Show all 36 references
  1. [9]

    Cedric Chan and Jianjing Kuang. 2024. https://api.semanticscholar.org/CorpusID:270893297 Exploring the accuracy of prosodic encodings in state-of-the-art text-to-speech models . Speech Prosody 2024

    2024

  2. [10]

    Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019. https://doi.org/10.18653/v1/W19-4828 What does BERT look at? an analysis of BERT `s attention . In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP...

    2019 doi

  3. [11]

    Verna Dankers, Christopher Lucas, and Ivan Titov. 2022. https://doi.org/10.18653/v1/2022.acl-long.252 Can transformer be too compositional? analysing idiom processing in neural machine translation . In Proceedings of the 60th Annual Meeting of the Association for Computational...

    2022 doi

  4. [12]

    Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei L...

    2025 arXiv

  5. [13]

    Marcio Fuckner, Sophie Horsman, Pascal Wiggers, and Iskaj Janssen. 2023. https://doi.org/10.1109/SpeD59241.2023.10314895 Uncovering bias in asr systems: Evaluating wav2vec2 and whisper for dutch speakers . In 2023 International Conference on Speech Technology and Human-Compute...

    2023

  6. [14]

    Ilya Gusev. 2025. Saiga llama3 8b model. https://huggingface.co/IlyaGusev/saiga_llama3_8b. Accessed: 2025-01-27

    2025

  7. [15]

    Javier Iranzo-Sánchez, Javier Jorge, Pau Baquero-Arnal, Joan Albert Silvestre-Cerdà, Adrià Giménez, Jorge Civera, Albert Sanchis, and Alfons Juan. 2021. https://doi.org/10.1016/j.neunet.2021.05.013 Streaming cascade-based speech translation leveraged by a direct segmentation m...

    2021 doi

  8. [16]

    Zhoulin Ji, Chenhao Lin, Hang Wang, and Chao Shen. 2024. https://doi.org/10.24963/ijcai.2024/46 Speech-forensics: Towards comprehensive synthetic speech dataset establishment and analysis . In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelli...

    2024 doi

  9. [17]

    Weiss, Yuan Cao, Chung-Cheng Chiu, Naveen Ari, Stella Laurenzo, and Yonghui Wu

    Ye Jia, Melvin Johnson, Wolfgang Macherey, Ron J. Weiss, Yuan Cao, Chung-Cheng Chiu, Naveen Ari, Stella Laurenzo, and Yonghui Wu. 2019. https://arxiv.org/abs/1811.02050 Leveraging weakly supervised data to improve end-to-end speech-to-text translation . Preprint, arXiv:1811.02050

    2019 arXiv

  10. [18]

    Xiaotong Jiang. 2024. Speech synthesis and quality evaluation

    2024

  11. [19]

    Anna Langedijk, Hosein Mohebbi, Gabriele Sarti, Willem Zuidema, and Jaap Jumelet. 2024. https://arxiv.org/abs/2310.03686 Decoderlens: Layerwise interpretation of encoder-decoder transformers . Preprint, arXiv:2310.03686

    2024 arXiv

  12. [20]

    Emmy Liu, Aditi Chaudhary, and Graham Neubig. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.933 Crossing the threshold: Idiomatic machine translation through retrieval augmentation and loss weighting . In Proceedings of the 2023 Conference on Empirical Methods in Natural L...

    2023 doi

  13. [21]

    Anna Min, Chenxu Hu, Yi Ren, and Hang Zhao. 2025. https://arxiv.org/abs/2502.00377 When end-to-end is overkill: Rethinking cascaded speech-to-text translation . Preprint, arXiv:2502.00377

    2025 arXiv

  14. [22]

    Yasmin Moslem. 2024. https://arxiv.org/abs/2406.17363 Leveraging synthetic audio data for end-to-end low-resource speech translation . Preprint, arXiv:2406.17363

    2024 arXiv

  15. [23]

    Jan Niehues, Ngoc-Quan Pham, Thanh-Le Ha, Matthias Sperber, and Alex Waibel. 2018. https://arxiv.org/abs/1808.00491 Low-latency neural speech translation . Preprint, arXiv:1808.00491

    2018 arXiv

  16. [24]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 B leu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311--3...

    2002

  17. [25]

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. https://arxiv.org/abs/2212.04356 Robust speech recognition via large-scale weak supervision . Preprint, arXiv:2212.04356

    2022 arXiv

  18. [26]

    Giulia Rambelli, Emmanuele Chersoni, Marco S. G. Senaldi, Philippe Blache, and Alessandro Lenci. 2023. Are frequent phrases directly retrieved like idioms? an investigation with self-paced reading and language models. In Proceedings of the 19th Workshop on Multiword Expression...

    2023

  19. [27]

    Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.213 COMET : A neural framework for MT evaluation . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685--2702, ...

    2020 doi

  20. [28]

    VAGO Solutions. 2025. Llama-3 sauerkrautlm 8b instruct model. https://huggingface.co/VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct. Accessed: 2025-01-27

    2025

  21. [29]

    Huacheng Song and Hongzhi Xu. 2024. https://aclanthology.org/2024.lrec-main.198/ Benchmarking the performance of machine translation evaluation metrics with C hinese multiword expressions . In Proceedings of the 2024 Joint International Conference on Computational Linguistics,...

    2024

  22. [30]

    David Stap, Eva Hasler, Bill Byrne, Christof Monz, and Ke Tran. 2024. https://arxiv.org/abs/2405.20089 The fine-tuning paradox: Boosting translation quality without sacrificing llm abilities . Preprint, arXiv:2405.20089

    2024 arXiv

  23. [31]

    NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prang...

    2022 arXiv

  24. [32]

    Ye Tian, Isobel James, and Hye Son. 2023. https://doi.org/10.18653/v1/2023.starsem-1.16 How are idioms processed inside transformer language models? In Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023), pages 174--179, Toronto, Canada....

    2023 doi

  25. [33]

    Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. 2019. https://doi.org/10.18653/v1/P19-1580 Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned . In Proceedings of the 57th Annual Meeting of the Associatio...

    2019 doi

  26. [34]

    Wester, O

    M. Wester, O. Watts, and G. E. Henter. 2016. https://doi.org/10.21437/SpeechProsody.2016-157 Evaluating comprehension of natural and synthetic conversational speech . In Proceedings of Speech Prosody 2016, pages 766--770

    2016 doi

  27. [35]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  28. [36]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.