Pith. sign in

REVIEW 3 major objections 6 minor 48 references

KIT's Offline Speech Translation and Instruction Following Submission for IWSLT 2025

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that contrastive pretraining—not data or scale—is what makes a frozen-encoder speech LLM suddenly good at four instruction-following tasks, and that LLM fusion plus post-editing completes the offline translation system.

desk verdict A decent shared-task system paper whose ASR/ST results hold up, but whose SQA/SSUM superiority claims rest on self-generated references from the same model family the system uses. read the letter →

arxiv 2505.13036 v1 pith:QGFUKX6X submitted 2025-05-19 cs.CL cs.AI

Sai Koneru , Maike Züfle , Thai-Binh Nguyen , Seymanur Akti , Jan Niehues , Alexander Waibel This is my paper · ORCID
classification cs.CLcs.AI
keywords speechtranslationinstructionfollowingLLMcontrastivepretrainingASRfusionautomaticpost-editingspokenquestionansweringsummarization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports a pair of systems for two IWSLT 2025 tracks, asking whether a single speech-conditioned LLM can follow varied spoken-language instructions and whether LLMs assembled around a cascade improve offline translation. For the instruction-following track, it tries to establish that contrastive pretraining of the projector that bridges a frozen speech encoder and a frozen LLM is the decisive ingredient, beating direct fine-tuning and ASR pretraining on every task. For the offline translation track, it tries to establish that fusing the hypotheses of several ASR systems with an LLM at document level, and then post-editing the translation, pushes quality beyond any single system. The reader should care because both results point to a cheap way to get more from frozen foundation models: train only a small connector and let an LLM reconcile multi-system outputs in context.

What carries the argument

For the IF track, the load-bearing object is a Q-Former projector—four transformer layers with four learnable query tokens—trained to map frozen SeamlessM4T speech features into frozen LLaMA-3.1 text space; the paper's claim is that contrastive pretraining of this bridge, using cosine similarity or Wasserstein distance, creates a representation that downstream fine-tuning can exploit across all four tasks. For the offline track, the load-bearing mechanism is document-level LLM fusion of ASR hypotheses, where a fine-tuned Llama-3-8B with LoRA sees several system transcripts and predicts the reference transcription, followed by automatic post-editing of the machine translation output, both run with surrounding context rather than sentence-by-sentence.

What would settle it

Take the ACL 60/60 audio, obtain human-written answers, summaries, and en-it translations for the same 250-sample subsets, and rerun the Table 8 comparison; if the contrastive-pretrained IF models no longer beat the baselines on those human references, the central claim fails. For the offline track, a comparable check is to replace the MetricX evaluation with a human annotation task on the final translations and see whether the 1.84 versus 2.27 gap persists.

Watch

Extended reading notes

Core claim

The central claim of the instruction-following half is that contrastive pretraining of the trainable connector between a frozen speech encoder (SeamlessM4T-Large) and a frozen LLM (LLaMA-3.1-8B-Instruct) is the decisive improvement over direct fine-tuning and ASR pretraining. On the ACL 60/60 test set, the contrastive-cosine model reaches 18.82 WER for English ASR versus 25.1 with no pretraining and 21.42 with ASR pretraining, and it improves COMET for en→de, en→it, and en→zh translation as well as BERTScore for SQA and SSUM. The submitted model, the Wasserstein-contrastive variant with answer tagging and document-level post-editing, gives the best ASR, SQA, and SSUM results among the system's own models and is competitive in ST. The offline half claims that fusing four ASR hypotheses with a fine-tuned LLM at document level, then translating with a quality-filtered fine-tuned Tower 7B and post-editing with Tower 13B, lowers MetricX on ACL 60/60 from 2.27 for the best single pipeline to 1.84.

Load-bearing premise

The IF track's reported advantages are measured on machine-generated references produced by the same model families used inside the system, so if those references diverge from human judgment, the reported ranking of models may be inflated.

Editorial extensions

If this is right

  • If the IF claim holds, a single frozen speech encoder plus frozen LLM can cover ASR, ST, SQA, and SSUM with only a small trainable bridge, which makes multi-task speech assistants much cheaper to build.
  • Contrastive pretraining should become the default initialization for this kind of SpeechLLM, not direct fine-tuning: it cuts WER by about six points and lifts every downstream metric reported in the paper.
  • Document-level LLM fusion can improve translation even when the fused transcript is not the lowest-WER hypothesis, because translation quality depends on punctuation, casing, and context that WER ignores.
  • Post-editing with surrounding context partially reverses the damage done by segmenting long audio, so long-form spoken translation systems should treat segmentation and post-editing as one joint design problem.
  • In offline translation, the full pipeline reduces MetricX from 2.27 to 1.84 on ACL 60/60, suggesting that refinement layers add more than swapping ASR models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the reported IF gains are measured against test sets whose SQA/SSUM references were generated by LLaMA-3.1-8B-Instruct and whose en-it ST reference was generated by SeamlessM4T-Large; if human-created references disagree with those generations, the margin between the submitted system and the baselines could shrink.
  • Editorial inference: the key result likely transfers to other frozen encoder/LLM pairs with Q-Former style bridges, since the paper's ablation shows the benefit comes from the pretraining objective, not from model scale.
  • Editorial inference: a direct test of the offline pipeline's robustness would be to run it on meetings or lectures with human-annotated transcripts and translations, where domain shift and segmentation noise would stress the fusion and APE components.
  • Editorial inference: the paper's observation that lower WER does not predict translation quality suggests shared-task scoring should weight downstream metrics, not ASR accuracy, when ranking speech translation systems.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper describes KIT's submissions to the IWSLT 2025 Offline Speech Translation track (English-to-German) and the Constrained Long Instruction-Following (IF) track. For the Offline track, the system combines VAD-based segmentation, multiple ASR systems whose outputs are fused by an LLM at document level, a quality-filtered fine-tuned Tower 7B translation model, and a Tower 13B automatic post-editing stage. For the IF track, the authors build an end-to-end SpeechLLM by connecting a SeamlessM4T-Large encoder to LLaMA-3.1-8B-Instruct through a Q-Former, and they compare direct fine-tuning, ASR pretraining, and contrastive pretraining with cosine or Wasserstein losses. They further study VAD segmentation lengths, a chain-of-thought 'tag' variant for SQA, and document-level post-editing. The central claims are that contrastive pretraining substantially improves IF performance over no pretraining and ASR pretraining (Table 5), that LLM fusion and APE improve Offline ST quality (Table 4), and that the final submitted IF model 'offers the best performance for ASR, SQA, and SSUM, and is competitive with the other IF models in ST' (Section 3.7).

Significance. If the results hold, the paper provides useful evidence for two practical design choices: contrastive pretraining of a frozen-encoder SpeechLLM helps in a constrained setting, and a cascaded pipeline with multi-ASR fusion plus automatic post-editing improves noisy speech translation. The ablations in Tables 5-7 are clearly structured and the augmented dataset is released on HuggingFace, which is valuable for reproducibility. However, the SQA, SSUM, and en-it ST evaluations rely on references generated by the same model families (LLaMA-3.1-8B-Instruct and SeamlessM4T-Large) that are used inside the submitted system, so the headline superiority claims on those tasks are not yet established. The core ASR and en-de/en-zh ST results on ACL 60/60 are not affected by this concern, but the paper's final-model selection and some cross-model comparisons need additional support.

major comments (3)
  1. [Section 3.1, Section 3.7, Table 8] The SQA, SSUM, and en-it ST evaluation sets are generated by the same model families used in the submitted system. The 250-sample SQA/SSUM test sets are built from NUTSHELL data using LLaMA-3.1-8B-Instruct-generated questions/answers and LLaMA-3.1 translations (Section 3.1), and the en-it ACL 60/60 references are produced by SeamlessM4T-Large. Since the IF model's LLM is LLaMA-3.1-8B-Instruct, its speech encoder is SeamlessM4T-Large, and the SQA/SSUM augmentation training data are generated with the same LLaMA-3.1 prompts and distribution, the IF models are evaluated on a distribution they were explicitly trained to mimic, while baselines such as Phi-4 and Qwen2-Audio were not. The reported BERTScore/COMET advantages in Table 8 for SQA, SSUM, and en-it ST may therefore reflect distribution familiarity rather than task competence. This directly affects the Section 3.7 claim that the final model 'offers the best performance for ASR, SQA, and SSUM.' The paper should either add a human-annotated evaluation sample for these tasks, compare baselines on references not produced by the authors' own models, or explicitly restrict the superiority claims to exclude these generated-reference settings.
  2. [Section 3.2, Section 3.7, Tables 5 and 7] All experimental results are reported from single runs without error bars, confidence intervals, or significance tests. Several of the decisions that determine the final submission rest on small differences: in Table 5, contrastive cosine versus Wasserstein gives 18.82 vs. 19.07 WER and 77.31 vs. 77.33 COMET for en-de ST; in Table 7, ST en-de COMET improves from 71.61 with no post-editing to 71.91 with a context size of 3, a gain of 0.30 that is likely within run-to-run noise. Given that the paper selects one of several closely matched configurations as the final model, single-run reporting without variance estimates makes it difficult to know whether the selected configuration is genuinely better than the alternatives. The authors should add multiple seeds or significance tests for the key comparisons, or soften the claims that depend on these differences.
  3. [Section 3.7 vs. Table 8] The statement that the submitted model 'IF contr. wasser. tag + post-edit ... offers the best performance for ASR, SQA, and SSUM' is not fully supported by Table 8. For SQA en-en, IF contr. cos. tag achieves BERTScore 82.86, while the submitted model reaches 82.80; for SQA en-zh, IF contr. cos. tag achieves 80.53 vs. 80.32 for the submitted model. If the authors intend 'best among the VAD-segmented variants' or 'best according to a different aggregation criterion,' that criterion should be stated explicitly. As written, the claim contradicts the numbers in the paper's own results table and should be corrected.
minor comments (6)
  1. [Figure 1] The figure caption contains the text 'We use bird score and blue score,' which appears to be a typo for 'BERTScore and BLEU score'; please correct it.
  2. [Section 2.3.2] There are several typos, including 'submision' and 'plently'; please proofread the manuscript.
  3. [Section 2.1] 'optimalchunk' should read 'optimal chunk size'; also update the sentence for clarity.
  4. [Section 3.1] Please clarify how the 250-sample test sets are sampled from NUTSHELL test data and confirm that they are disjoint from the augmented training data, so that readers can assess potential train/eval leakage.
  5. [Section 3.6] Please specify the exact BERTScore configuration (e.g., model and F1 variant) used for SQA and SSUM, and the COMET checkpoint used for ST, to improve reproducibility.
  6. [Table 8] The table is very wide and the '+ post-edit' rows are easy to misalign with their parent rows; consider repeating the model name on each row or using clearer separators.

Circularity Check

1 steps flagged · score 5.0 of 10

Final SQA/SSUM superiority rests on LLaMA-generated references, making that claim partially circular; core contrastive-pretraining/ASR-ST results remain independent.

  1. other [Section 3.1 (Data Augmentation, SQA/SSUM) and Section 3.7 (Results, Final Model)]
    "We then use LLaMA-3.1-8B-Instruct to generate two answerable and one unanswerable QA pair per segment for all language pairs. ... Additionally, we generate a 250-sample test set from a subset of the NUTSHELL test data. SSUM: ... we translate the full NUTSHELL dataset (en→{de, it, zh}) using LLaMA-3.1-8B-Instruct. ... As with SQA, we also generate a 250-sample multilingual test set. Final Model We select IF contr. wasser. tag + post-edit for our final submission. It offers the best performance for ASR, SQA, and SSUM, and is competitive with the other IF models in ST."

    The SQA/SSUM evaluation sets are generated by LLaMA-3.1-8B-Instruct with the same prompts used to create the augmented SQA/SSUM training data, and the submitted IF model's LLM is initialized from LLaMA-3.1-8B-Instruct and fine-tuned on that augmented data. The final-model claim 'best performance for ASR, SQA, and SSUM' therefore rests, for SQA/SSUM, on scoring against the exact output distribution the model was built to imitate; baselines (Phi-4, Qwen2-Audio, Whisper+LLaMA without this fine-tuning) were not trained on that distribution, so their lower BERTScore may reflect distribution mismatch rather than worse instruction following. This makes the SQA/SSUM part of the final-model claim partially circular.

full rationale

The paper's primary technical claim—contrastive pretraining improves over no pretraining and ASR pretraining—is evaluated on the external ACL 60/60 benchmark with WER/COMET against fixed references (Table 5), so it is self-contained and not circular. The offline ST pipeline is likewise assessed on ITV and ACL 60/60 with standard metrics and external baselines. The only load-bearing circularity is in the IF track's SQA/SSUM evaluation: the 250-sample test sets and multilingual references are created by LLaMA-3.1-8B-Instruct (the same model family as the system's LLM backbone and the augmented training-data generator), and the en-it ST reference is generated by SeamlessM4T-Large (the system's speech encoder). Consequently, the final-model sentence claiming best SQA/SSUM performance is partly a self-match to the reference generator rather than an external benchmark. This is not a self-citation-chain or equation-level collapse, and no parameter is fitted to the test set, so the score is moderate, not extreme.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new theoretical entities. Its claims rest on empirical choices: hyperparameters tuned on development sets (chunk sizes, context sizes, filtered data volumes) and domain assumptions about synthetic data and model-generated references being representative proxies for real evaluation conditions.

free parameters (7)
  • Offline ASR chunk size = 25 seconds
    Grid search over 5-25s on IWSLT 2024 dev sets (Peloton, EPTV, ITV) and ACL 60/60; 25s gave lowest WER (Table 1).
  • IF ASR VAD segment max length = 20 seconds
    Ablation on ACL 60/60 (Table 6) selected 20s for best WER.
  • IF ST VAD segment max length = 25 seconds
    Ablation on ACL 60/60 (Table 6) selected 25s for best COMET.
  • ASR post-editing context size = 5 sentences
    Ablation on ACL 60/60 (Table 7) selected 5 sentences for WER.
  • ST post-editing context size = 15 sentences
    Ablation on ACL 60/60 (Table 7) selected 15 sentences for COMET.
  • MT quality-filtered corpus size = top 500k pairs by XCOMET
    Data selection choice in Section 2.3.2 without ablation; affects fine-tuning data.
  • APE training triplets count = 100k
    Sampling choice in Section 2.3.3 without ablation.
assumptions (4)
  • domain assumption TTS-synthesized speech (VITS) is a valid proxy for real noisy audio in training the ASR fusion and post-editing models.
    Section 2.2.1 uses VITS-generated audio from Europarl, OpenSubtitles, NewsCommentary, NUTSHELL transcripts to create training data for LLM fusion; Section 3.4 uses SeamlessM4T transcriptions of TTS abstracts to train post-editing.
  • domain assumption XCOMET quality scores reliably rank translation pair quality for data filtering.
    Section 2.3.2 selects top 500k pairs based on XCOMET scores; no manual or downstream validation of the filter is reported.
  • domain assumption COMET, MetricX, ChrF2, and BERTScore computed on automatically generated references reflect the shared-task evaluation.
    Section 3.6 and Section 3.1 rely on these metrics over self-generated references for en-it ST and SQA/SSUM.
  • domain assumption The 250-sample NUTSHELL-derived test sets represent the official IWSLT 2025 speech QA and summarization test distribution.
    Section 3.1 constructs these test sets from NUTSHELL test data using the same LLM family as the system, with no evidence of distributional match to the official test data.

how reviews work

0 comments
Cite this review

Pith. "Pith review of KIT's Offline Speech Translation and Instruction Following Submission for IWSLT 2025." pith.science (2026). https://pith.science/paper/QGFUKX6X

@misc{pith2026250513036,
  author       = {Pith},
  title        = {Pith review of: KIT's Offline Speech Translation and Instruction Following Submission for IWSLT 2025},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QGFUKX6X}},
  note         = {Machine review of arXiv:2505.13036}
}
read the original abstract

The scope of the International Workshop on Spoken Language Translation (IWSLT) has recently broadened beyond traditional Speech Translation (ST) to encompass a wider array of tasks, including Speech Question Answering and Summarization. This shift is partly driven by the growing capabilities of modern systems, particularly with the success of Large Language Models (LLMs). In this paper, we present the Karlsruhe Institute of Technology's submissions for the Offline ST and Instruction Following (IF) tracks, where we leverage LLMs to enhance performance across all tasks. For the Offline ST track, we propose a pipeline that employs multiple automatic speech recognition systems, whose outputs are fused using an LLM with document-level context. This is followed by a two-step translation process, incorporating additional refinement step to improve translation quality. For the IF track, we develop an end-to-end model that integrates a speech encoder with an LLM to perform a wide range of instruction-following tasks. We complement it with a final document-level refinement stage to further enhance output quality by using contextual information.

Figures

Figures reproduced from arXiv: 2505.13036 by the authors.

Figure 1
Figure 1. For the Instruction-Following track, we train an end-to-end SpeechLLM, while the Offline track relies [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 14 canonical work pages

  1. [1]

    Hewett, Mojan Javaheripi, Piero Kauffmann, James R

    Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, and 8 others. 2024. https://arxiv.org/abs/2412.08905 Phi-4 technic...

    arXiv 2024

  2. [2]

    Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, and 1 others. 2025. Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras. arXiv preprint arXiv:2503.01743

    arXiv 2025

  3. [3]

    Ibrahim Said Ahmad, Antonios Anastasopoulos, Ond r ej Bojar, Claudia Borg, Marine Carpuat, Roldano Cattoni, Mauro Cettolo, William Chen, Qianqian Dong, Marcello Federico, Barry Haddow, D \'a vid Javorsk \'y , Mateusz Krubi \'n ski, Tsz Kin Lam, Xutai Ma, Prashant Mathur, Evgeny Matusov, Chandresh Maurya, John McCrae, and 25 others. 2024. https://doi.org/1...

    doi:10.18653/v1/2024.iwslt-1.1 2024

  4. [4]

    Duarte Miguel Alves, Jos \'e Pombal, Nuno M Guerreiro, Pedro Henrique Martins, Jo \ a o Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, and 1 others. 2024. Tower: An open multilingual large language model for translation-related tasks. In First Conference on Language Modeling

    work page 2024

  5. [5]

    Max Bain, Jaesung Huh, Tengda Han, and Andrew Zisserman. 2023. Whisperx: Time-accurate speech transcription of long-form audio. INTERSPEECH 2023

    work page 2023

  6. [6]

    Hervé Bredin and Antoine Laurent. 2021. https://doi.org/10.21437/Interspeech.2021-560 End-to-end speaker segmentation for overlap-aware resegmentation . In Interspeech 2021, pages 3111--3115

    doi:10.21437/interspeech.2021-560 2021

  7. [8]

    Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, and 1 others. 2024 b . Qwen2-audio technical report. arXiv preprint arXiv:2407.10759

    arXiv 2024

  8. [9]

    Seamless Communication, Loïc Barrault, Yu-An Chung, Mariano Cora Meglioli, David Dale, Ning Dong, Paul-Ambroise Duquenne, Hady Elsahar, Hongyu Gong, Kevin Heffernan, John Hoffman, Christopher Klaiber, Pengwei Li, Daniel Licht, Jean Maillard, Alice Rakotoarison, Kaushik Ram Sadagopan, Guillaume Wenzek, Ethan Ye, and 49 others. 2023. https://arxiv.org/abs/2...

    arXiv 2023

Show all 48 references
  1. [10]

    Menglong Cui, Pengzhi Gao, Wei Liu, Jian Luan, and Bin Wang. 2025. Multilingual machine translation with open large language models at practical scale: An empirical study. arXiv preprint arXiv:2502.02481

    2025 arXiv

  2. [11]

    Mara Finkelstein, David Vilar, and Markus Freitag. 2024. https://doi.org/10.18653/v1/2024.wmt-1.126 Introducing the N ews P a LM MBR and QE dataset: LLM -generated high-quality parallel data outperforms traditional web-crawled data . In Proceedings of the Ninth Conference on M...

    2024 doi

  3. [12]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Art...

    2024 arXiv

  4. [13]

    Nuno M Guerreiro, Ricardo Rei, Daan van Stigt, Luisa Coheur, Pierre Colombo, and Andr \'e FT Martins. 2024. xcomet: Transparent machine translation evaluation through fine-grained error detection. Transactions of the Association for Computational Linguistics, 12:979--995

    2024

  5. [14]

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations

    2022

  6. [15]

    Iranzo-Sánchez , J

    J. Iranzo-Sánchez , J. A. Silvestre-Cerdà , J. Jorge , N. Roselló , A. Giménez , A. Sanchis , J. Civera , and A. Juan . 2020. Europarl-st: A multilingual corpus for speech translation of parliamentary debates. In ICASSP 2020 - 2020 IEEE International Conference on Acoustics, S...

    2020

  7. [16]

    Juraj Juraska, Daniel Deutsch, Mara Finkelstein, and Markus Freitag. 2024. https://doi.org/10.18653/v1/2024.wmt-1.35 M etric X -24: The G oogle submission to the WMT 2024 metrics shared task . In Proceedings of the Ninth Conference on Machine Translation, pages 492--504, Miami...

    2024 doi

  8. [17]

    Jaehyeon Kim, Jungil Kong, and Juhee Son. 2021. Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech. In International Conference on Machine Learning, pages 5530--5540. PMLR

    2021

  9. [18]

    Philipp Koehn. 2005. https://aclanthology.org/2005.mtsummit-papers.11/ E uroparl: A parallel corpus for statistical machine translation . In Proceedings of Machine Translation Summit X: Papers, pages 79--86, Phuket, Thailand

    2005

  10. [19]

    Sai Koneru, Thai Binh Nguyen, Ngoc-Quan Pham, Danni Liu, Zhaolin Li, Alexander Waibel, and Jan Niehues. 2024 a . https://doi.org/10.18653/v1/2024.iwslt-1.24 Blending LLM s into cascaded speech translation: KIT `s offline speech translation system for IWSLT 2024 . In Proceeding...

    2024 doi

  11. [20]

    Sai Koneru, Miriam Exel, Matthias Huck, and Jan Niehues. 2024 b . https://doi.org/10.18653/v1/2024.naacl-long.148 Contextual refinement of translations: Large language models for sentence and document-level post-editing . In Proceedings of the 2024 Conference of the North Amer...

    2024 doi

  12. [21]

    Sai Koneru, Matthias Huck, Miriam Exel, and Jan Niehues. 2025. Quality-aware decoding: Unifying quality estimation and decoding. arXiv preprint arXiv:2502.08561

    2025 arXiv

  13. [22]

    Phuong-Hang Le, Hongyu Gong, Changhan Wang, Juan Pino, Benjamin Lecouteux, and Didier Schwab. 2023. https://arxiv.org/abs/2301.11716 Pre-training for speech translation: Ctc meets optimal transport . Preprint, arXiv:2301.11716

    2023 arXiv

  14. [23]

    Chia-Hsuan Lee, Szu-Lin Wu, Chi-Liang Liu, and Hung-yi Lee. 2018. Spoken squad: A study of mitigating the impact of speech recognition errors on listening comprehension. Proc. Interspeech 2018, pages 3459--3463

    2018

  15. [24]

    Paul Lerner, Juliette Bergo \"e nd, Camille Guinaudeau, Herv \'e Bredin, Benjamin Maurice, Sharleyne Lefevre, Martin Bouteiller, Aman Berhe, L \'e o Galmant, Ruiqing Yin, and Claude Barras. 2022. https://aclanthology.org/2022.lrec-1.367/ Bazinga! a dataset for multi-party dial...

    2022

  16. [25]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven C. H. Hoi. 2023. https://api.semanticscholar.org/CorpusID:256390509 Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models . In International Conference on Machine Learning

    2023

  17. [26]

    Pierre Lison and J \"o rg Tiedemann. 2016. https://aclanthology.org/L16-1147/ O pen S ubtitles2016: Extracting large parallel corpora from movie and TV subtitles . In Proceedings of the Tenth International Conference on Language Resources and Evaluation ( LREC `16) , pages 923...

    2016

  18. [27]

    Danni Liu, Thai Binh Nguyen, Sai Koneru, Enes Yavuz Ugan, Ngoc-Quan Pham, Tuan Nam Nguyen, Tu Anh Dinh, Carlos Mullov, Alexander Waibel, and Jan Niehues. 2023. https://doi.org/10.18653/v1/2023.iwslt-1.6 KIT `s multilingual speech translation system for IWSLT 2023 . In Proceedi...

    2023 doi

  19. [28]

    Gabriel Peyré and Marco Cuturi. 2019. https://doi.org/10.1561/2200000073 Computational optimal transport: With applications to data science . Foundations and Trends® in Machine Learning, 11:355--206

    2019 doi

  20. [29]

    Maja Popovi \'c . 2015. https://doi.org/10.18653/v1/W15-3049 chr F : character n-gram F -score for automatic MT evaluation . In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392--395, Lisbon, Portugal. Association for Computational Linguistics

    2015 doi

  21. [30]

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International conference on machine learning, pages 28492--28518. PMLR

    2023

  22. [31]

    Miguel Ramos, Patrick Fernandes, Ant \'o nio Farinhas, and Andre Martins. 2024. https://aclanthology.org/2024.eamt-1.22/ Aligning neural machine translation models: Human feedback in training and inference . In Proceedings of the 25th Annual Conference of the European Associat...

    2024

  23. [32]

    Ricardo Rei, Jos \'e G. C. de Souza, Duarte Alves, Chrysoula Zerva, Ana C Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and Andr \'e F. T. Martins. 2022 a . https://aclanthology.org/2022.wmt-1.52/ COMET -22: Unbabel- IST 2022 submission for the metrics shared task . In...

    2022

  24. [33]

    Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, Jos \'e G

    Ricardo Rei, Marcos Treviso, Nuno M. Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, Jos \'e G. C. de Souza, Taisiya Glushkova, Duarte Alves, Luisa Coheur, Alon Lavie, and Andr \'e F. T. Martins. 2022 b . https://aclanthology.org/2022.wmt-1.60/ C omet K iwi: IST -...

    2022

  25. [34]

    Elizabeth Salesky, Kareem Darwish, Mohamed Al-Badrashiny, Mona Diab, and Jan Niehues. 2023. https://doi.org/10.18653/v1/2023.iwslt-1.2 Evaluating multilingual speech translation under realistic conditions with resegmentation and terminology . In Proceedings of the 20th Interna...

    2023 doi

  26. [35]

    Jongseo Sohn, Nam Soo Kim, and Wonyong Sung. 1999. https://doi.org/10.1109/97.736233 A statistical model-based voice activity detection . IEEE Signal Processing Letters, 6(1):1--3

    1999 doi

  27. [36]

    Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun MA, and Chao Zhang. 2024. https://openreview.net/forum?id=14rn7HpKVk SALMONN : Towards generic hearing abilities for large language models . In The Twelfth International Conference on Learning ...

    2024

  28. [37]

    Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, MA Zejun, and Chao Zhang. 2023. Salmonn: Towards generic hearing abilities for large language models. In The Twelfth International Conference on Learning Representations

    2023

  29. [38]

    Silero Team. 2021. Silero models: pre-trained enterprise-grade stt / tts models and benchmarks. https://github.com/snakers4/silero-models

    2021

  30. [39]

    Gállego, José A

    Ioannis Tsiamas, Gerard I. Gállego, José A. R. Fonollosa, and Marta R. Costa-jussà. 2022. https://doi.org/10.21437/Interspeech.2022-59 SHAS: Approaching optimal Segmentation for End-to-End Speech Translation . In Proc. Interspeech 2022, pages 106--110

    2022 doi

  31. [40]

    Changhan Wang, Anne Wu, and Juan Pino. 2020. https://arxiv.org/abs/2007.10310 Covost 2: A massively multilingual speech-to-text translation corpus . Preprint, arXiv:2007.10310

    2020 arXiv

  32. [41]

    John Wiseman. 2019. Wiseman/py-webrtcvad. GitHub repository, Nov

    2019

  33. [42]

    Haoran Xu, Kenton Murray, Philipp Koehn, Hieu Hoang, Akiko Eriguchi, and Huda Khayrallah. 2024 a . X-alma: Plug & play modules and adaptive rejection for quality translation at scale. arXiv preprint arXiv:2410.03115

    2024 arXiv

  34. [43]

    Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. 2024 b . Contrastive preference optimization: Pushing the boundaries of llm performance in machine translation. In International Conference on Machine Learning,...

    2024

  35. [44]

    Brian Yan, Patrick Fernandes, Jinchuan Tian, Siqi Ouyang, William Chen, Karen Livescu, Lei Li, Graham Neubig, and Shinji Watanabe. 2024. https://doi.org/10.18653/v1/2024.iwslt-1.22 CMU `s IWSLT 2024 offline speech translation system: A cascaded approach for long-form robustnes...

    2024 doi

  36. [45]

    Weinberger, and Yoav Artzi

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. https://openreview.net/forum?id=SkeHuCVFDr Bertscore: Evaluating text generation with BERT . In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April...

    2020

  37. [47]

    Maike Züfle and Jan Niehues. 2024. https://arxiv.org/abs/2412.15712 Contrastive learning for task-independent speechllm-pretraining . Preprint, arXiv:2412.15712

    2024 arXiv

  38. [48]

    Maike Züfle, Sara Papi, Beatrice Savoldi, Marco Gaido, Luisa Bentivogli, and Jan Niehues. 2025. https://arxiv.org/abs/2502.16942 Nutshell: A dataset for abstract generation from scientific talks . Preprint, arXiv:2502.16942

    2025 arXiv

  39. [49]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  40. [50]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.