Pith. sign in

REVIEW 7 cited by

U2++: Unified Two-pass Bidirectional End-to-end Model for Speech Recognition

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2106.05642 v3 pith:V43UYKEY submitted 2021-06-10 cs.SD cs.CLeess.AS

Di Wu , Binbin Zhang , Chao Yang , Zhendong Peng , Wenjing Xia , Xiaoyu Chen , Xin Lei This is my paper · ORCID
classification cs.SDcs.CLeess.AS
keywords streamingmodelrecognitionaccuracyaccurateaishell-1backwarddecoding
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

The unified streaming and non-streaming two-pass (U2) end-to-end model for speech recognition has shown great performance in terms of streaming capability, accuracy, real-time factor (RTF), and latency. In this paper, we present U2++, an enhanced version of U2 to further improve the accuracy. The core idea of U2++ is to use the forward and the backward information of the labeling sequences at the same time at training to learn richer information, and combine the forward and backward prediction at decoding to give more accurate recognition results. We also proposed a new data augmentation method called SpecSub to help the U2++ model to be more accurate and robust. Our experiments show that, compared with U2, U2++ shows faster convergence at training, better robustness to the decoding method, as well as consistent 5\% - 8\% word error rate reduction gain over U2. On the experiment of AISHELL-1, we achieve a 4.63\% character error rate (CER) with a non-streaming setup and 5.05\% with a streaming setup with 320ms latency by U2++. To the best of our knowledge, 5.05\% is the best-published streaming result on the AISHELL-1 test set.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. WenetSpeech-Yue: A Large-scale Cantonese Speech Corpus with Multi-dimensional Annotation

    cs.SD 2025-09 conditional novelty 6.0 of 10

    The authors built and released the largest open-source Cantonese speech corpus (21,800 hours, 10 domains, rich metadata), and show that models trained on it match or beat existing speech recognition and synthesis systems.

  2. A Unified Speech LLM for Diarization and Speech Recognition in Multilingual Conversations

    cs.CL 2025-06 conditional novelty 6.0 of 10

    A local-window speech LLM with prompt-based speaker context and an external diarization alignment module achieves 27.25 tcpWER/tcpCER on the MLC-SLM Task II evaluation set, a 54.87% relative improvement over the offic...

  3. Unifying Streaming and Non-streaming Zipformer-based ASR

    cs.SD 2025-06 conditional novelty 5.0 of 10

    Dynamic right-context chunked attention masking lets a single zipformer ASR model cover streaming and non-streaming use, nearly closing the accuracy gap with a modest latency increase.

  4. CNVSRC 2024: The Second Chinese Continuous Visual Speech Recognition Challenge

    cs.CV 2025-05 conditional novelty 5.0 of 10

    CNVSRC 2024 lowers the baseline character error rate for Chinese visual speech recognition from 48.6% to 39.7% (single-speaker) and from 58.4% to 52.2% (multi-speaker), while adding a 200-hour dataset and documenting ...

  5. Delayed-KD: Delayed Knowledge Distillation based CTC for Low-Latency Streaming ASR

    cs.SD 2025-05 conditional novelty 4.0 of 10

    A Temporal Alignment Buffer with minimum-KL delay selection lets Delayed-KD reach 5.42% CER on AISHELL-1 at 40 ms latency, matching U2++ at 320 ms.

  6. Sample adaptive data augmentation with progressive scheduling

    cs.SD 2024-11 reject novelty 4.0 of 10

    PS-SapAug combines loss-based hybrid normalization with a progressive augmentation schedule and reports 5 to 9 percent relative WER improvements over SpecAugment on Aishell-1 and LibriSpeech-100h.

  7. WavChat: A Survey of Spoken Dialogue Models

    eess.AS 2024-11 conditional novelty 4.0 of 10

    WavChat categorizes spoken dialogue models into cascaded and end-to-end paradigms and surveys speech representations, training strategies, streaming, duplex interaction, datasets, and evaluation benchmarks.

Pith tools