Pith. sign in

REVIEW 5 cited by

Unified Streaming and Non-streaming Two-pass End-to-end Model for Speech Recognition

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2012.05481 v2 pith:Z65QVI4G submitted 2020-12-10 cs.SD cs.CLeess.AS

Unified Streaming and Non-streaming Two-pass End-to-end Model for Speech Recognition

Binbin Zhang , Di Wu , Zhuoyuan Yao , Xiong Wang , Fan Yu , Chao Yang , Liyong Guo , Yaguang Hu
show 2 more authors
Lei Xie Xin Lei
This is my paper
classification cs.SD cs.CLeess.AS
keywords modelnon-streamingstreamingattentionlatencyachievesaishell-1decoder
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

In this paper, we present a novel two-pass approach to unify streaming and non-streaming end-to-end (E2E) speech recognition in a single model. Our model adopts the hybrid CTC/attention architecture, in which the conformer layers in the encoder are modified. We propose a dynamic chunk-based attention strategy to allow arbitrary right context length. At inference time, the CTC decoder generates n-best hypotheses in a streaming way. The inference latency could be easily controlled by only changing the chunk size. The CTC hypotheses are then rescored by the attention decoder to get the final result. This efficient rescoring process causes very little sentence-level latency. Our experiments on the open 170-hour AISHELL-1 dataset show that, the proposed method can unify the streaming and non-streaming model simply and efficiently. On the AISHELL-1 test set, our unified model achieves 5.60% relative character error rate (CER) reduction in non-streaming ASR compared to a standard non-streaming transformer. The same model achieves 5.42% CER with 640ms latency in a streaming ASR system.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. TRADE: Transducer-Augmented Decoder for Speech LLM

    cs.CL 2026-06 unverdicted novelty 6.0

    TRADE augments multimodal Speech LLMs with a transducer branch for streaming ASR, reporting 6.71% WER offline and 8.40% streaming on the Open ASR Leaderboard from one checkpoint.

  2. A Study of Data Selection Strategies for Pre-training Self-Supervised Speech Models

    cs.SD 2026-01 unverdicted novelty 5.0

    Prioritizing longest utterances in SSL speech pre-training data outperforms random or diversity-based sampling for ASR performance while using half the data volume.

  3. NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASR

    eess.AS 2026-04 unverdicted novelty 4.0

    NIM4-ASR delivers SOTA ASR performance on public benchmarks using a 2.3B-parameter LLM with multi-stage training, real-time streaming, and million-scale hotword customization via RAG.

  4. NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASR

    eess.AS 2026-04 conditional novelty 4.0

    A 2.3B-parameter LLM-based ASR system achieves competitive recognition accuracy and reduced hallucination through a multi-stage training paradigm with asynchronous encoder updates, ASR-specialized RL, and phoneme-leve...

  5. Online Predictive Coding for Dual-Mode Self-Supervised Speech Model

    cs.SD 2026-06 unverdicted novelty 3.0

    Proposes OPC and dual-mode LN to improve dual-mode SSL speech models, reducing WER gap at 160 ms latency on LibriSpeech from 3.65% to 3.40% (test-clean).