Pith. sign in

REVIEW 1 cited by

Building a great multi-lingual teacher with sparsely-gated mixture of experts for speech recognition

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2112.05820 v3 pith:JPWIWD7V submitted 2021-12-10 cs.CL cs.AIcs.LGeess.AS

Kenichi Kumatani , Robert Gmyr , Felipe Cruz Salinas , Linquan Liu , Wei Zuo , Devang Patel , Eric Sun , Yu Shi This is my paper
classification cs.CL cs.AIcs.LGeess.AS
keywords networkssparsely-gatedexpertsinvestigatelanguagemixturemodemulti-lingual
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The sparsely-gated Mixture of Experts (MoE) can magnify a network capacity with a little computational complexity. In this work, we investigate how multi-lingual Automatic Speech Recognition (ASR) networks can be scaled up with a simple routing algorithm in order to achieve better accuracy. More specifically, we apply the sparsely-gated MoE technique to two types of networks: Sequence-to-Sequence Transformer (S2S-T) and Transformer Transducer (T-T). We demonstrate through a set of ASR experiments on multiple language data that the MoE networks can reduce the relative word error rates by 16.3% and 4.6% with the S2S-T and T-T, respectively. Moreover, we thoroughly investigate the effect of the MoE on the T-T architecture in various conditions: streaming mode, non-streaming mode, the use of language ID and the label decoder with the MoE.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Tight Clusters Make Specialized Experts

    cs.LG 2025-02 unverdicted novelty 6.0 of 10

    Introduces Adaptive Clustering router for MoE models that scales features to identify tight expert clusters, yielding faster convergence, robustness to corruption, and performance gains.

Pith tools