StreamUni: Achieving Streaming Speech Translation with a Unified Large Speech-Language Model

Mengge Liu; Shoutao Guo; Wei Chen; Xiang Li; Yang Feng

REVIEW 4 major objections 6 minor 1 cited by

StreamUni: Achieving Streaming Speech Translation with a Unified Large Speech-Language Model

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read StreamUni makes one large speech-language model handle segmentation, timing, and translation together via speech chain-of-thought.

desk verdict A clean idea for unified streaming speech translation, but the SOTA claim is backed by models that never saw the proposed streaming CoT training. read the letter →

arxiv 2507.07803 v2 pith:PIBY654W submitted 2025-07-10 cs.CL cs.SDeess.AS

Shoutao Guo , Xiang Li , Mengge Liu , Wei Chen , Yang Feng This is my paper

classification cs.CLcs.SDeess.AS

keywords streamingspeechtranslationsimultaneouslargespeech-languagemodelchain-of-thoughtgenerationpolicytruncationCoTtrainingsegmentation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the hard parts of streaming speech translation—deciding when to discard old audio, when to start emitting a translation, and what to emit—can all be read off a single intermediate transcription produced by one large speech-language model. It proposes StreamUni, which instructs the model to first transcribe the current audio segment and then translate it, treating two identical consecutive transcriptions as a pause and sentence-ending punctuation as a boundary. Those signals drive a truncation policy and a word-count generation policy that keeps the translation k words behind the source. A streaming CoT training scheme samples partial audio prefixes so the model learns to transcribe and translate before hearing a full sentence. Experiments on MuST-C and CoVoST2 report the best quality-latency trade-offs among the compared methods on StreamST across several language directions.

What carries the argument

Speech chain-of-thought (speech CoT): the LSLM first transcribes the current audio segment, $x^{(n)}$, and then translates it, so one forward pass yields both the signal for policy decisions and the translation itself. It carries the argument because both policies read off $x^{(n)}$: truncation compares consecutive transcriptions, and generation uses the word count $C$ in $O = C - k - (i-1-b_m)$, the number of words the model may emit while staying $k$ words behind the source. The streaming CoT training loss, $\mathcal{L} = -\sum_{s_{\le i} \sim U(S)} \log p(y \mid x^{(i)}, s_{\le i}) p(x^{(i)} \mid s_{\le i})$, extends this two-stage generation to partial audio prefixes.

What would settle it

Run the truncation policy on audio with background noise, overlapping speakers, or self-corrections and align the truncated source segments against the final translations; if discarded audio contains words that never appear in any retained translation, truncation integrity fails and the central claim is wrong. A simpler version: force a retroactive transcription revision mid-stream and check whether the model still truncates at the same places and preserves translation quality.

Watch

Extended reading notes

Core claim

StreamUni establishes that a unified large speech-language model can perform streaming speech translation without a separate voice-activity detection (VAD), segmentation module, or task-specific policy model. Given a truncated speech segment, the model is instructed to generate real-time transcription and then translation in one chain-of-thought pass. The transcription does double duty: comparing it with the previous two transcriptions triggers truncation when the text is unchanged (a pause) or when a sentence-final punctuation is followed by new text (a boundary), and the word count C of the current transcription sets the number of allowed output words via $O = C - k - (i-1-b_m)$. When the paper is right, the same LSLM weights perform segmentation, timing, and translation, and the only per-task component is the prompt structure plus a small amount of streaming CoT training data.

Load-bearing premise

The whole scheme leans on the model's streaming transcription being stable and truthful: if adding audio revises earlier words, if two identical transcriptions do not mean the speaker paused, or if sentence punctuation does not align with real boundaries, the truncation rules will drop untranslated speech or repeat already-translated content.

Editorial extensions

If this is right

Streaming speech translation can be done by a single LSLM without cascading voice activity detection, segmentation, and policy-specific modules.
The same transcription-based generation policy transfers to simultaneous speech translation by disabling truncation, and the paper reports state-of-the-art results there as well.
Because no policy heads are trained, the method carries over to a different LSLM backbone with only language-model fine-tuning.
A modest amount of streaming CoT data (roughly 50 hours) mixed with non-streaming data is enough to improve low-latency translation, whereas streaming-only data does not help.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, the transcription-stability assumption suggests that truncation quality in noisy or overlapping speech could be quantified by comparing model-chosen truncation points against forced-alignment segment boundaries.
Beyond the paper, the fixed word-count lag $O$ could be made adaptive per language pair or per transcription confidence, since it implicitly assumes source-target monotonicity.
Beyond the paper, one could test whether the same prompt-only recipe turns any capable speech-language model into a streaming translator by applying it to speech-to-speech translation or live subtitling.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The headline is right: StreamUni is a neat way to get a single LSLM to handle segmentation, timing, and translation, and the streaming CoT training loss does seem to help on the one direction where it is tested. But the SOTA StreamST claim rests on models that never saw that training, so the paper oversells its central result.

What is actually new: using the intermediate transcription from speech CoT as the signal for both the read/write policy and the truncation policy, plus a training loss (Eq. 8) that forces the model to predict the complete translation from partial audio. That is a clean combination, and the ablations on CoVoST2 En-Zh are convincing that the streaming CoT loss helps at low latency, especially when mixed with non-streaming data. The truncation-policy-only comparison against human-annotated truncation (Table 1) is a good sanity check. Releasing code, data, and models is also a plus.

The big soft spot is the gap between the method and the headline numbers. Figure 3 claims SOTA on MuST-C En-De and En-Es StreamST, but the models used for those curves were trained only on non-streaming CoT data. Streaming CoT training was applied only to En-Zh, and En-Zh is not evaluated on StreamST at all. So the paper never demonstrates that the full method beats the baselines on the task it claims SOTA on. That needs fixing, either by running streaming CoT on En-De/En-Es or by softening the claim.

The stress-test note about transcription instability is also fair, though not fatal. The policies use the word count and sentence boundaries of a real-time transcription that has WER around 20 on partial audio. The paper does not measure how often ASR revisions would change a policy decision or make an emitted translation inconsistent. Since the translation is still generated from the audio, the damage may be limited, but it is an unaddressed robustness concern.

Minor issues: no error bars or significance tests on any of the curves, and the StreamST evaluation relies on mWERSegmenter alignment, which can silently corrupt the metric. The paper should at least discuss the alignment step.

Overall: a solid engineering paper with a real idea and useful artifacts, but the main SOTA claim is not supported by the experiments as presented. A serious referee should take it, but it needs major revision before publication.

Referee Report

4 major / 6 minor

Summary. StreamUni proposes a single large speech-language model for streaming speech translation. It uses a speech chain-of-thought instruction so the model first produces a real-time transcription and then a translation. A truncation policy compares successive transcriptions to find pause or sentence boundaries; a generation policy computes the number of words to emit from the transcription word count with a delay hyperparameter k; when truncation occurs the model outputs the complete translation for the discarded segment. A streaming CoT training objective (Eq. 8) samples partial speech prefixes and trains full-translation prediction from partial input. The method is evaluated on SimulST (CoVoST2 En-Zh, Fr-En; MuST-C En-De, En-Es) and StreamST (MuST-C En-De, En-Es) with Phi-4-Multimodal, with Qwen2.5-Omni generalization experiments. The paper claims state-of-the-art results and releases code, data, and models.

Significance. If the results hold, StreamUni is a useful step toward replacing cascaded VAD segmentation plus SimulST pipelines with a single LSLM, and the policy rules are simple enough to transfer across LSLMs without policy-specific training. Concrete strengths include the public release of code, datasets, and checkpoints; the ablations showing that the proposed streaming CoT training improves SimulST BLEU at low latency (Figures 4-5); and the demonstration that the recipe transfers to a second backbone (Table 3). The main caveats are that the central StreamST state-of-the-art claim is demonstrated only under a restricted training recipe and that the policies' dependence on noisy streaming transcription is not quantified.

major comments (4)

[Experiments, System Settings; Figure 3] The StreamST state-of-the-art claim in Figure 3 is not supported by models trained with the proposed streaming CoT recipe. The System Settings state that only the En-Zh direction uses 50 hours of streaming CoT data plus 50 hours of non-streaming CoT data, while the other three directions (including MuST-C En-De and En-Es, the two StreamST benchmarks) use 100 hours of non-streaming CoT data only. Thus the StreamST curves in Figure 3 are produced without streaming CoT training, and the paper's central claim that streaming CoT training enables unified policy and generation in StreamST is untested for exactly the languages on which the headline result is reported. Please either train with streaming CoT data for the StreamST directions and report the curves, or explicitly qualify the conclusion as applying to the non-streaming-trained configuration.
[Generation Policy, Eq. (6); Table 2] The generation policy assumes that the real-time transcription x(n) is stable and monotonic, but Table 2 reports WER 20.74 for the deployed training recipe on the low-latency evaluation set. Since Eq. (6) computes the allowed output word count O from the word count C of x(n), and since translations emitted under Eq. (7) are never revised, an ASR error or a later revision of an earlier word can directly produce a wrong output length and permanently inconsistent translations. The paper does not measure how often x(n) changes across consecutive chunks, nor does it analyze the interaction between transcription instability and the two truncation rules (e.g., two identical consecutive transcriptions can reflect a frozen ASR state rather than a speaker pause). Please add a chunk-level transcription stability analysis and a sensitivity ablation that perturbs x(n) or uses a reference transcription to quantify the impact on StreamST quality.
[Experiments, Table 1] The truncation-policy evaluation in Table 1 is run with the generation policy disabled, so it does not exercise the coupled decision loop of Eqs. (5)-(7): in actual StreamST the truncation decision changes the context for subsequent generation, and the generation policy in turn affects what translation context is available at the truncation point. The Table 1 comparison between human and model truncation on document-level inputs is therefore not sufficient to support the claim that the proposed truncation policy improves end-to-end StreamST; it only shows translation quality under a non-streaming, whole-segment decoding setup. Please add a StreamST ablation that varies the truncation policy while holding the generation policy fixed and reports StreamLAAL and Stream SacreBLEU on the same document-level benchmark.
[Experiments, Main Results; Figures 2-5] All reported numbers are single runs without error bars, confidence intervals, or significance tests, and several headline differences in Figure 3 are on the order of 1-2 BLEU points at comparable latency. Because the state-of-the-art claim depends on these small margins, please report the number of independent runs and variance, or bootstrap confidence intervals, for the main SimulST and StreamST comparisons.

minor comments (6)

[Truncation Policy] In the truncation policy section, the queue is written as "q can be represented as = [x(am+1), ..., x(n-1)]" with an extra equals sign, and the later expression "x(l)(l = am+1+ 1, .., n)" mixes an index variable with a segment endpoint; both should be rewritten for clarity.
[Analysis, Table 1] The text says "Figure 1 illustrates the results" when referring to Table 1; the table and figure references should be corrected.
[Speech CoT Argumentation] The subsection heading "Speech CoT Argumentation" should be "Speech CoT Augmentation".
[Figure 3] In Figure 3 the legends label the baselines as StreamAttFW and StreamAttP, but the text refers to StreamFW and StreamAttP; the naming should be made consistent.
[Eq. (5) and surrounding text] Equation (5) uses "yi:bm+1" and "ybm+1:i-1" with ambiguous subscripts; the notation should be clarified (e.g., y_{i:b_{m+1}} and y_{b_m+1:i-1}) and the definition of b_{m+1} should be aligned with the earlier use of b_m.
[General] There are minor typos such as "for a instance" in the Streaming CoT Training section and "V oice" in the references; a copyedit pass is needed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: StreamUni's policies are explicit functions of its intermediate transcription and are validated on external StreamST benchmarks.

full rationale

The claimed derivation chain is self-contained. Eq. (4) defines the real-time transcription x(n) as the model's own intermediate output; Eqs. (5)-(7) are explicit policy rules computed from x(n), the historical queue, and previously emitted translation words, and are evaluated by downstream SacreBLEU/COMET on MuST-C and CoVoST2, not by construction of those metrics. Eq. (8) is a standard maximum-likelihood streaming-CoT training objective built from externally available speech-transcription-translation triplets. The truncation rules (silence for two chunks or sentence-ending punctuation) and the word-count lag rule (6) are heuristics; they do not define the quantity they claim to predict. The paper's self-citations (LSG, StreamSpeech, NAST) appear as baselines or comparisons and are not load-bearing premises. Table 2's WER 20.74 indicates transcription instability is a robustness risk for any policy keyed to x(n), but that is an empirical fragility, not circular reasoning. Therefore no specific circular step can be quoted.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The ledger shows no invented entities or fitted constants. The method rests on domain assumptions about the stability and informativeness of the LSLM's partial-audio transcriptions, plus hand-set policy parameters (k, chunk size, fallback threshold). These assumptions are reasonable for a heuristic system but are only partially validated.

free parameters (3)

delay hyperparameter k = swept over {1, 3, 5, 7, 9}
Controls how many words the translation lags behind the source transcription in Eq. (6); the reported latency-quality curves are parameterized by this hand-set value.
chunk size = 320 ms (En-Zh), 640 ms (other directions)
Input speech chunk duration, chosen per language direction; changing it changes latency and context.
fallback truncation threshold = 30 chunks
If neither silence nor sentence-boundary trigger fires, the appendix sets a hand-picked 30-chunk limit before forcing a truncation.

assumptions (4)

domain assumption The real-time transcription x(n) generated from partial audio is stable and prefix-consistent across chunks.
Eq. (4) and the truncation rules compare x(n) with x(n-1) and x(n-2); if transcription is revised retroactively as more audio arrives, the comparisons are meaningless.
domain assumption The number of words C in the transcription is a valid proxy for translation progress, so O = C - k - (i-1-bm) yields correct output timing.
Eq. (6) assumes a monotonic word-level alignment between source transcription and target translation, which is not generally true for En-De or En-Zh.
ad hoc to paper Truncation when the speaker pauses (two identical transcriptions) or finishes a sentence preserves semantic integrity of both speech and translation.
The two trigger rules in the Truncation Policy section are heuristic; no analysis is provided for cases where silence occurs mid-clause or where sentence boundaries in ASR are wrong.
domain assumption The streaming CoT loss in Eq. (8), trained on uniformly truncated speech, transfers to real streaming conditions.
Training uses random truncation points; real streams have variable speaker pacing and disfluencies, and the paper only validates this on one language direction (En-Zh).

how reviews work

0 comments

Cite this review

Pith. "Pith review of StreamUni: Achieving Streaming Speech Translation with a Unified Large Speech-Language Model." pith.science (2026). https://pith.science/paper/PIBY654W

@misc{pith2026250707803,
  author       = {Pith},
  title        = {Pith review of: StreamUni: Achieving Streaming Speech Translation with a Unified Large Speech-Language Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PIBY654W}},
  note         = {Machine review of arXiv:2507.07803}
}

read the original abstract

Streaming speech translation (StreamST) requires determining appropriate timing, known as policy, to generate translations while continuously receiving source speech inputs, balancing low latency with high translation quality. However, existing StreamST methods typically operate on sentence-level speech segments, referred to as simultaneous speech translation (SimulST). In practice, they require collaboration with segmentation models to accomplish StreamST, where the truncated speech segments constrain SimulST models to make policy decisions and generate translations based on limited contextual information. Moreover, SimulST models struggle to learn effective policies due to the complexity of speech inputs and cross-lingual generation. To address these challenges, we propose StreamUni, which achieves StreamST through a unified Large Speech-Language Model (LSLM). Specifically, StreamUni incorporates speech Chain-of-Thought (CoT) in guiding the LSLM to generate multi-stage outputs. Leveraging these multi-stage outputs, StreamUni simultaneously accomplishes speech segmentation, policy decision, and translation generation, completing StreamST without requiring massive policy-specific training. Additionally, we propose a streaming CoT training method that enhances low-latency policy decisions and generation capabilities using limited CoT data. Experiments demonstrate that our approach achieves state-of-the-art performance on StreamST tasks.

Figures

Figures reproduced from arXiv: 2507.07803 by the authors.

**Figure 2.** Performance of different methods on SimulST task. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 4.** SimulST performance of different generation poli [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

Figures from the paper (1 more)

**Figure 3.** Figure 3: StreamST performance of different methods. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

HOLA: Enhancing Audio-visual Deepfake Detection via Hierarchical Contextual Aggregations and Efficient Pre-training
cs.CV 2025-07 conditional novelty 6.0 of 10

A two-stage audio-visual deepfake detector, HOLA, uses 1.81M pre-training samples and hierarchical cross-modal fusion modules to achieve first place and near-perfect AUC on AV-Deepfake1M++ video-level detection.

Reference graph

Works this paper leans on

45 extracted references · 33 canonical work pages · cited by 1 Pith paper

[1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Agostinelli, V.; Wild, M.; Raffel, M.; Fuad, K. A. A.; and Chen, L. 2024. Simul-LLM: A Framework for Exploring High-Quality Simultaneous Translation with Large Language Models. arXiv preprint arXiv:2312.04691

work page Pith review arXiv 2024
[4]

Alinejad, A.; and Sarkar, A. 2020. Effectively pretraining a speech translation decoder with Machine Translation data. In Webber, B.; Cohn, T.; He, Y.; and Liu, Y., eds., Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 8014--8020. Online: Association for Computational Linguistics

work page 2020
[5]

Bain, M.; Huh, J.; Han, T.; and Zisserman, A. 2023. WhisperX: Time-Accurate Speech Transcription of Long-Form Audio. INTERSPEECH 2023

work page 2023
[6]

Chen, J.; Ma, M.; Zheng, R.; and Huang, L. 2021. Direct Simultaneous Speech-to-Text Translation Assisted by Synchronized Streaming ASR . In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

work page 2021
[7]

Cheng, S.; Huang, Z.; Ko, T.; Li, H.; Peng, N.; Xu, L.; and Zhang, Q. 2024. Towards Achieving Human Parity on End-to-end Simultaneous Speech Translation via LLM Agent. arXiv:2407.21646

arXiv 2024
[8]

Communication, S.; Barrault, L.; Chung, Y.-A.; Meglioli, M. C.; Dale, D.; Dong, N.; Duppenthaler, M.; Duquenne, P.-A.; Ellis, B.; Elsahar, H.; Haaheim, J.; Hoffman, J.; Hwang, M.-J.; Inaguma, H.; Klaiber, C.; Kulikov, I.; Li, P.; Licht, D.; Maillard, J.; Mavlyutov, R.; Rakotoarison, A.; Sadagopan, K. R.; Ramakrishnan, A.; Tran, T.; Wenzek, G.; Yang, Y.; Y...

arXiv 2023

Show all 45 references

[9]

DeepSeek-AI; Guo, D.; Yang, D.; Zhang, H.; Song, J.; Zhang, R.; Xu, R.; Zhu, Q.; Ma, S.; Wang, P.; Bi, X.; Zhang, X.; Yu, X.; Wu, Y.; Wu, Z. F.; Gou, Z.; Shao, Z.; Li, Z.; Gao, Z.; Liu, A.; Xue, B.; Wang, B.; Wu, B.; Feng, B.; Lu, C.; Zhao, C.; Deng, C.; Zhang, C.; Ruan, C.; D...

2025 arXiv
[10]

DeepSeek-AI; Liu, A.; Feng, B.; Xue, B.; Wang, B.; Wu, B.; Lu, C.; Zhao, C.; Deng, C.; Zhang, C.; Ruan, C.; Dai, D.; Guo, D.; Yang, D.; Chen, D.; Ji, D.; Li, E.; Lin, F.; Dai, F.; Luo, F.; Hao, G.; Chen, G.; Li, G.; Zhang, H.; Bao, H.; Xu, H.; Wang, H.; Zhang, H.; Ding, H.; Xi...

2025 arXiv
[11]

A.; Cattoni, R.; Bentivogli, L.; Negri, M.; and Turchi, M

Di Gangi, M. A.; Cattoni, R.; Bentivogli, L.; Negri, M.; and Turchi, M. 2019. M u ST - C : a M ultilingual S peech T ranslation C orpus. In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics

2019
[12]

Dong, Q.; Zhu, Y.; Wang, M.; and Li, L. 2022. Learning When to Translate for Streaming Speech. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics

2022
[13]

Gangi, M. A. D.; Negri, M.; and Turchi, M. 2019. Adapting Transformer to End-to-End Spoken Language Translation. In Interspeech 2019, 1133--1137

2019
[14]

Graves, A.; Fern \'a ndez, S.; Gomez, F.; and Schmidhuber, J. 2006. Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. In ACM International Conference Proceeding Series, 369--376

2006
[15]

Guo, S.; Zhang, S.; Ma, Z.; and Feng, Y. 2025. Large Language Models Are Read/Write Policy-Makers for Simultaneous Generation. arXiv:2501.00868

2025 arXiv
[16]

Huang, Z.; Ye, R.; Ko, T.; Dong, Q.; Cheng, S.; Wang, M.; and Li, H. 2023. Speech Translation with Large Language Models: An Industrial Practice. arXiv:2312.13585

2023 arXiv
[17]

Iranzo-Sánchez, J.; Iranzo-Sánchez, J.; Giménez, A.; Civera, J.; and Juan, A. 2024. Segmentation-Free Streaming Machine Translation. arXiv:2309.14823

2024 arXiv
[18]

Lee, A.; Chen, P.-J.; Wang, C.; Gu, J.; Popuri, S.; Ma, X.; Polyak, A.; Adi, Y.; He, Q.; Tang, Y.; Pino, J.; and Hsu, W.-N. 2022. Direct Speech-to-Speech Translation With Discrete Units. In Muresan, S.; Nakov, P.; and Villavicencio, A., eds., Proceedings of the 60th Annual Mee...

2022
[19]

Li, D.; I, T.; Arivazhagan, N.; Cherry, C.; and Padfield, D. 2021. Sentence Boundary Augmentation for Neural Machine Translation Robustness. In ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 7553--7557

2021
[20]

Liu, D.; Du, M.; Li, X.; Li, Y.; and Chen, E. 2021. Cross Attention Augmented Transducer Networks for Simultaneous Translation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

2021
[21]

Ma, M.; Huang, L.; Xiong, H.; Zheng, R.; Liu, K.; Zheng, B.; Zhang, C.; He, Z.; Liu, H.; Li, X.; Wu, H.; and Wang, H. 2019. STACL: Simultaneous Translation with Implicit Anticipation and Controllable Latency using Prefix-to-Prefix Framework. In Proceedings of the 57th Conferen...

2019
[22]

J.; Wang, C.; Gu, J.; and Pino, J

Ma, X.; Dousti, M. J.; Wang, C.; Gu, J.; and Pino, J. 2020. SIMULEVAL : An Evaluation Toolkit for Simultaneous Translation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations

2020
[23]

Ma, X.; Pino, J.; and Koehn, P. 2020 a . S imul MT to S imul ST : Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translation. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10t...

2020
[24]

Ma, X.; Pino, J.; and Koehn, P. 2020 b . SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translation. arXiv preprint arXiv:2011.02048

2020 arXiv
[25]

Ma, Z.; Feng, Y.; and Zhang, M. 2025. Overcoming Non-monotonicity in Transducer-based Streaming Generation. arXiv:2411.17170

2025 arXiv
[26]

Ma, Z.; Zhang, S.; Guo, S.; Shao, C.; Zhang, M.; and Feng, Y. 2023. Non-autoregressive Streaming Transformer for Simultaneous Translation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

2023
[27]

Matusov, E.; Leusch, G.; Bender, O.; and Ney, H. 2005. Evaluating Machine Translation Output with Automatic Sentence Segmentation. In Proceedings of the Second International Workshop on Spoken Language Translation. Pittsburgh, Pennsylvania, USA

2005
[28]

Microsoft; :; Abouelenin, A.; Ashfaq, A.; Atkinson, A.; Awadalla, H.; Bach, N.; Bao, J.; Benhaim, A.; Cai, M.; Chaudhary, V.; Chen, C.; Chen, D.; Chen, D.; Chen, J.; Chen, W.; Chen, Y.-C.; ling Chen, Y.; Dai, Q.; Dai, X.; Fan, R.; Gao, M.; Gao, M.; Garg, A.; Goswami, A.; Hao, ...

2025 arXiv
[29]

Papi, S.; Gaido, M.; Negri, M.; and Bentivogli, L. 2024. S tream A tt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selection. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Proceedings of the 62nd Annual Meeting of the Association for Com...

2024
[30]

Papi, S.; Gaido, M.; Negri, M.; and Turchi, M. 2022. Over-Generation Cannot Be Rewarded: Length-Adaptive Average Lagging for Simultaneous Speech Translation. In Ive, J.; and Zhang, R., eds., Proceedings of the Third Workshop on Automatic Simultaneous Translation, 12--17. Onlin...

2022
[31]

Papi, S.; Negri, M.; and Turchi, M. 2023. Attention as a Guide for Simultaneous Speech Translation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

2023
[32]

Post, M. 2018. A Call for Clarity in Reporting BLEU Scores. In Proceedings of the Third Conference on Machine Translation: Research Papers

2018
[33]

de Souza, J

Rei, R.; C. de Souza, J. G.; Alves, D.; Zerva, C.; Farinha, A. C.; Glushkova, T.; Lavie, A.; Coheur, L.; and Martins, A. F. T. 2022. COMET -22: Unbabel- IST 2022 Submission for the Metrics Shared Task. In Koehn, P.; Barrault, L.; Bojar, O.; Bougares, F.; Chatterjee, R.; Costa-...

2022
[34]

Tang, Y.; Sun, A.; Inaguma, H.; Chen, X.; Dong, N.; Ma, X.; Tomasello, P.; and Pino, J. 2023. Hybrid Transducer and Attention based Encoder-Decoder Modeling for Speech-to-Text Tasks. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Vo...

2023
[35]

Team, S. 2024. Silero VAD: pre-trained enterprise-grade Voice Activity Detector (VAD), Number Detector and Language Classifier. https://github.com/snakers4/silero-vad

2024
[36]

N.; Kaiser, L

Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L. u.; and Polosukhin, I. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems, volume 30

2017
[37]

Wang, C.; Wu, A.; and Pino, J. 2020. CoVoST 2 and Massively Multilingual Speech-to-Text Translation. arXiv preprint 2007.10310

2020 arXiv
[38]

Wang, M.; Vu, T.-T.; Wang, Y.; Shareghi, E.; and Haffari, G. 2024. Conversational SimulMT: Efficient Simultaneous Translation with Large Language Models. arXiv:2402.10552

2024
[39]

H.; Le, Q

Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; brian ichter; Xia, F.; Chi, E. H.; Le, Q. V.; and Zhou, D. 2022. Chain of Thought Prompting Elicits Reasoning in Large Language Models. In Oh, A. H.; Agarwal, A.; Belgrave, D.; and Cho, K., eds., Advances in Neural Information Proc...

2022
[40]

Xu, J.; Guo, Z.; He, J.; Hu, H.; He, T.; Bai, S.; Chen, K.; Wang, J.; Fan, Y.; Dang, K.; Zhang, B.; Wang, X.; Chu, Y.; and Lin, J. 2025. Qwen2.5-Omni Technical Report. arXiv:2503.20215

2025 arXiv
[41]

Zeng, X.; Li, L.; and Liu, Q. 2021. R eal T ran S : End-to-End Simultaneous Speech Translation with Convolutional Weighted-Shrinking Transformer. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

2021
[42]

Zhang, D.; Li, S.; Zhang, X.; Zhan, J.; Wang, P.; Zhou, Y.; and Qiu, X. 2023. Speech GPT : Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities. In The 2023 Conference on Empirical Methods in Natural Language Processing

2023
[43]

Zhang, S.; Fang, Q.; Guo, S.; Ma, Z.; Zhang, M.; and Feng, Y. 2024 a . S tream S peech: Simultaneous Speech-to-Speech Translation with Multi-task Learning. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Proceedings of the 62nd Annual Meeting of the Association for Computat...

2024
[44]

Zhang, S.; Fang, Q.; Guo, S.; Ma, Z.; Zhang, M.; and Feng, Y. 2024 b . S tream S peech: Simultaneous Speech-to-Speech Translation with Multi-task Learning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics

2024
[45]

Zhang, S.; and Feng, Y. 2023. End-to-End Simultaneous Speech Translation with Differentiable Segmentation. In Findings of the Association for Computational Linguistics: ACL 2023

2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Agostinelli, V.; Wild, M.; Raffel, M.; Fuad, K. A. A.; and Chen, L. 2024. Simul-LLM: A Framework for Exploring High-Quality Simultaneous Translation with Large Language Models. arXiv preprint arXiv:2312.04691

work page Pith review arXiv 2024

[4] [4]

Alinejad, A.; and Sarkar, A. 2020. Effectively pretraining a speech translation decoder with Machine Translation data. In Webber, B.; Cohn, T.; He, Y.; and Liu, Y., eds., Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 8014--8020. Online: Association for Computational Linguistics

work page 2020

[5] [5]

Bain, M.; Huh, J.; Han, T.; and Zisserman, A. 2023. WhisperX: Time-Accurate Speech Transcription of Long-Form Audio. INTERSPEECH 2023

work page 2023

[6] [6]

Chen, J.; Ma, M.; Zheng, R.; and Huang, L. 2021. Direct Simultaneous Speech-to-Text Translation Assisted by Synchronized Streaming ASR . In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

work page 2021

[7] [7]

Cheng, S.; Huang, Z.; Ko, T.; Li, H.; Peng, N.; Xu, L.; and Zhang, Q. 2024. Towards Achieving Human Parity on End-to-end Simultaneous Speech Translation via LLM Agent. arXiv:2407.21646

arXiv 2024

[8] [8]

Communication, S.; Barrault, L.; Chung, Y.-A.; Meglioli, M. C.; Dale, D.; Dong, N.; Duppenthaler, M.; Duquenne, P.-A.; Ellis, B.; Elsahar, H.; Haaheim, J.; Hoffman, J.; Hwang, M.-J.; Inaguma, H.; Klaiber, C.; Kulikov, I.; Li, P.; Licht, D.; Maillard, J.; Mavlyutov, R.; Rakotoarison, A.; Sadagopan, K. R.; Ramakrishnan, A.; Tran, T.; Wenzek, G.; Yang, Y.; Y...

arXiv 2023