Multilingual Speech Recognition with Corpus Relatedness Sampling

Alan W. Black; Florian Metze; Siddharth Dalmia; Xinjian Li

REVIEW 3 major objections 5 minor 41 references

Multilingual Speech Recognition with Corpus Relatedness Sampling

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A multilingual acoustic model that samples whole training corpora according to their learned similarity to the target corpus cuts phone error rate on all 16 low-resource tasks, from 20.1% average to 18.5%.

desk verdict A plausible sampling trick with consistent gains over fine-tuning, but the paper never isolates whether cosine-similarity sampling or the gradual temperature schedule causes the improvement. read the letter →

arxiv 1908.01060 v1 pith:7EBG5LMD submitted 2019-08-02 cs.CL cs.SDeess.AS

Xinjian Li , Siddharth Dalmia , Alan W. Black , Florian Metze This is my paper · ORCID

classification cs.CLcs.SDeess.AS

keywords multilingualspeechrecognitionlow-resourceASRcorpusembeddingsrelatednesssamplingphoneerrorrateannealedtemperaturedomainclustering

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes that a multilingual speech-recognition model should not treat every training corpus equally. It learns a vector for each corpus, uses the cosine similarity between those vectors to decide how often each corpus is sampled while training toward a target low-resource corpus, and gradually raises the sampling temperature so training starts broad and ends focused on the target. On 16 corpora across 10 languages this schedule improves every target over the standard pretrain-then-fine-tune baseline, cutting the average phone error rate—the fraction of confused individual speech sounds—from 20.1% to 18.5%. The authors also show the learned corpus vectors cluster by language and by speech domain, which they offer as evidence that the similarity signal is meaningful.

What carries the argument

The central object is the corpus embedding: a trainable vector $e_i$ that biases input features through $h = \mathrm{Encoder}(x + e_i; W, E)$, so each corpus develops its own identity within the shared multilingual LSTM. Relatedness is then the cosine similarity score between corpus embeddings, and sampling probabilities come from a softmax over those scores scaled by a temperature $T$. The mechanism that does the work is annealing $T$ from near zero to a large value: low temperature yields near-uniform sampling (pretraining), high temperature concentrates almost all probability on the target corpus (fine-tuning), and the intermediate schedule automatically emphasizes related corpora before converging to the target.

What would settle it

Shuffle the similarity scores or replace them with random values, keep the same temperature schedule, and measure phone error rates; if the randomized model matches the real one, the relatedness signal is not what drives the improvement, and a direct check is whether cosine similarities correlate with per-corpus PER gains.

Watch

Extended reading notes

Core claim

The central claim is that an annealed corpus-sampling schedule beats both plain pretraining and pretraining-plus-fine-tuning for multilingual low-resource speech recognition. The paper learns a corpus-level embedding $e_i$ for each corpus by adding it to the acoustic input features inside the shared encoder, trains with the connectionist temporal classification (CTC) objective, and defines the probability of sampling corpus $C_i$ as a softmax over temperature-scaled cosine similarities between $e_i$ and the target corpus's embedding. As temperature goes to zero the schedule becomes uniform pretraining, and as it goes to infinity the schedule becomes target-only fine-tuning, so the method interpolates between the two extremes. In experiments the method improves on the fine-tuned baseline on all 16 corpora, with average phone error rate 18.5% versus 20.1%, with larger gains on smaller corpora which the authors attribute to the gradual schedule acting as built-in regularization.

Load-bearing premise

The load-bearing premise is that corpora with similar learned vectors are in fact the ones whose speech data most improves the target corpus, but the paper never measures whether similarity scores correlate with actual error-rate gains.

Editorial extensions

If this is right

Every one of the 16 target corpora improves over the fine-tuned baseline, with the average phone error rate falling from 20.1% to 18.5%.
Pretraining and fine-tuning are recovered as the two temperature limits, so the annealed schedule is a single training procedure that contains both standard stages.
Smaller corpora see the largest gains, which the paper explains as the gradual schedule preventing the quick overfitting that fine-tuning causes on tiny datasets.
Corpus embeddings carry both language and domain structure, meaning the sampling preference is effectively favoring same-language and same-domain speech.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper: the same learned relatedness could be used to choose which corpora to include in the multilingual pool at all, not just how to weight them during sampling.
Beyond the paper: the paper's own evidence would be stronger if it reported the correlation between cosine similarity and per-corpus phone-error gains, since the qualitative similarity tables do not by themselves establish that similarity drives transfer.
Beyond the paper: the annealed sampling idea is not limited to speech; any multitask sequence model with a known target domain could use the same corpus- or task-embedding schedule.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

This paper is worth a look if you work on multilingual low-resource ASR. The idea is clean: jointly learn an embedding for each training corpus, then use cosine similarity to the target corpus to shape a softmax sampling distribution that gradually anneals toward the target. That combination—corpus embeddings steering the sampling distribution—is not in the cited prior work, which used embeddings only as auxiliary features or sampled uniformly. The empirical claim is also strong on the surface: CRS beats a reasonable pretrain-then-finetune baseline on all 16 corpora, with an average 1.6% PER gain.

The paper does some things well. The equation setup is honest and correct: uniform sampling and pure fine-tuning are legitimate limits of the temperature parameter. The embedding analysis in Table 3 and Figure 2 convincingly shows the embeddings capture language and domain structure, which is directly relevant to the proposed mechanism.

But the central selling point is under-determined. The design compares CRS only against the standard pretrain-then-finetune pipeline, which means two things change at once: the gradual temperature schedule and the similarity-weighted sampling. There is no ablation that keeps the schedule uniform, or that keeps the sampling schedule but removes the embedding similarity. The paper itself credits the largest gain (Dutch, 3.4%) to "built-in regularization" from gradual annealing, not to relatedness. So the 1.6% average and the all-16 win pattern are fully compatible with a pure curriculum/regularization effect that has nothing to do with corpus embeddings. That is a real gap, and the stress-test note gets it right.

Training budgets, epochs, and learning-rate schedules are also not reported, so CRS might simply be spending more update steps near the target corpus. And there are no error bars, repeated seeds, or significance tests; the temperature hyperparameters T0=0.01 and a=1.5 are used without sensitivity analysis. These are standard weaknesses in this subfield, but they matter more here because the effect size is small.

None of this makes the paper worthless. If I were refereeing it, I would ask for one additional experiment: same annealing schedule with uniform sampling, and same similarity sampling with a flat temperature. That would settle whether the relatedness mechanism is the source of the gain. As it stands, the method is plausible and the numbers are consistent, but the causal claim is unevidenced.

I would send it to peer review rather than desk-reject it. A good set of reviewers can push for the ablation, and even without it, the paper is a useful data point for the subfield. If the ablation later confirms the mechanism, this becomes a genuinely useful training-time trick; if not, it still documents a simple regularization effect that can help low-resource targets.

Best, [Your name]

Referee Report

3 major / 5 minor

Summary. The paper proposes Corpus Relatedness Sampling (CRS) for multilingual speech recognition. The method trains corpus-level embeddings by adding a learnable corpus-specific vector to the acoustic input features, computes cosine similarities between each corpus and the target corpus, and uses a softmax over those similarities with an exponentially increasing temperature to anneal sampling from uniform over all corpora to target-only. The authors show that uniform pretraining and target-only fine-tuning are limiting cases of the temperature schedule, report phone error rates for 16 low-resource corpora, and claim that CRS outperforms the pretrain-then-finetune baseline on every corpus (average PER 18.5% versus 20.1%). They also present qualitative evidence that the corpus embeddings cluster by language and domain.

Significance. If the central claim is supported, CRS is a simple and practical contribution to multilingual low-resource ASR: it adds only a small embedding layer and a sampling schedule to an existing CTC architecture, and the unifying view of pretraining and fine-tuning as temperature limits is conceptually clean. The 16-corpus evaluation is a useful breadth of evidence, and the observation that embeddings capture language and domain information is interesting in its own right. However, the current experimental design does not isolate the mechanism responsible for the reported gains; the central claim therefore rests on an unevidenced causal link between corpus relatedness and the improvements. Strengths of the paper include the explicit mathematical formulation of the sampling distribution and the fact that the evaluation covers a diverse set of corpora, but the missing ablation and the absence of statistical validation prevent the claim from being fully established.

major comments (3)

[Section 3.2 and Section 4.3, Table 2] The experimental comparison conflates two changes relative to the baseline: (a) replacing the sharp pretrain-then-finetune boundary with the gradual temperature schedule of Eq. (6) and (b) biasing the sampling distribution by the cosine similarity of corpus embeddings. The paper compares CRS only against uniform pretraining followed by fine-tuning, so the average 1.6% absolute PER improvement and the win on all 16 corpora are equally compatible with a pure curriculum or regularization effect. A baseline that uses the same temperature schedule (T0=0.01, a=1.5) with uniform sampling, and a second baseline that uses randomly permuted or randomly initialized embeddings, would isolate the relatedness mechanism. Without such ablations, the paper's title claim that corpus relatedness sampling causes the improvement is not supported.
[Section 3.1 and Section 4.3, Table 3 and Figure 2] The central assumption that cosine similarity between learned corpus embeddings ranks corpora by their usefulness for transfer to the target corpus is validated only qualitatively. Table 3 and Figure 2 show that the embeddings organize by language and domain, which establishes surface identity but does not demonstrate that the similarity ordering is causally responsible for the PER gains. The authors should report a quantitative relationship, for example a correlation between the embedding similarity of each source corpus to the target and the observed PER improvement obtained by sampling that source more heavily, or an ablation in which the embedding-derived similarities are replaced with shuffled similarities. As written, the paper does not rule out the possibility that the embeddings encode features irrelevant to transfer, in which case CRS is no better than a uniform curriculum.
[Section 4.3, Table 2] No error bars, number of random seeds, or significance tests are reported for any of the PER numbers, and the training budget is not described (epochs, learning-rate schedule, optimizer, and number of target-corpus updates per model). Without this information, the claim that CRS improves on all 16 corpora could reflect noise or unequal optimization effort: CRS may simply feed the final model more target-corpus examples than the baseline's fine-tuning phase does. The authors should report mean and standard deviation over repeated runs for the baseline and CRS, and state the exact number of target-corpus updates each model receives. This is particularly important because the largest reported gains, such as Dutch (3.4%), are attributed by the authors to a 'built-in regularization' effect of the gradual temperature, which is itself a non-relatedness explanation.

minor comments (5)

[Table 3] The entry for Mandarin (hub) contains the typo 'Tkish (hub)'; this should be 'Turkish (hub)'.
[Section 4.3 and Abstract] The phrase '1.6 phone error rate' should be '1.6 percentage points of phone error rate' or 'an average PER improvement of 1.6% (absolute)', since PER is already a percentage.
[Figure 2] The t-SNE plot is described as showing 36 corpora, but the paper's experiments use only the 16 corpora in Table 1; the source, domain labels, and training details of the additional 20 corpora are not provided, making the plot hard to interpret or reproduce.
[Section 3.2, Eq. (5)] The sentence 'we can prove the statement as follows' is slightly overstated: the limit is correct for the softmax expression, but it assumes that no non-target corpus has cosine similarity exactly 1.0 to the target, and the proof should state that qualification.
[Section 4.3, second paragraph] The explanation that smaller corpora benefit from 'built-in regularization' is presented as a belief rather than as a tested hypothesis; if the authors retain this explanation, they should support it with an experiment that varies corpus size while holding the sampling mechanism fixed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the central PER comparison is empirical and the special-case limits in Eqs. 4-5 are mathematical identities.

full rationale

The paper's central claim is an empirical result: CRS achieves 18.5% average PER versus 20.1% for the fine-tuned baseline on 16 held-out test sets (Table 2). Nothing in Eqs. 1-6 forces this outcome. Eq. 3 defines the sampling distribution from cosine similarities, and Eqs. 4-5 are true softmax limits showing that uniform sampling and target-only sampling are extreme temperatures; they do not derive any improvement in PER. The corpus embeddings in Eq. 1 are fitted on the same training corpora that are later sampled, which is a mild form of data reuse, but the paper does not predict any held-out test number from that fit; the comparison with fine-tuning is an independent evaluation. The self-citations to the authors' prior architecture [5], EESEN [31], and Epitran [32] are tool citations: the architecture is used symmetrically for both the baseline and the proposed model, so the result does not rest on an unverified self-cited premise. The assumption that cosine similarity ranks transfer usefulness is a testable hypothesis supported only qualitatively, and the lack of an ablation separating the temperature schedule from the similarity weighting is a validity concern, not a circularity.

Assumptions & free parameters 4 free parameters · 3 assumptions · 1 invented entities

The free parameters are the temperature schedule and data truncation choices; the paper does not report sensitivity to them. The key axiom is that the learned corpus embeddings encode transfer-relevant relatedness, which is central to the method and only qualitatively validated. The invented entity is the corpus embedding, a trainable vector with no independent evidence beyond the paper's own experiments.

free parameters (4)

Initial temperature T0 = 0.01
Controls how close to uniform sampling the early training is; chosen as a hyperparameter without sensitivity analysis.
Temperature growth rate a = 1.5
Controls how quickly the sampling distribution shifts to target-related corpora; the paper does not study its effect on performance.
Corpus embedding dimension
The dimension of the embedding vectors is not reported in the paper; it is a model hyperparameter that must be set.
Utterance cap per corpus = 100,000
All corpora are truncated to 100,000 utterances; this preprocessing choice can affect the balance and results.

assumptions (3)

domain assumption The multilingual CTC acoustic model with shared layers is an appropriate base architecture.
Taken from prior work [5] and used for both baselines and the proposed method; not re-derived.
ad hoc to paper Adding a corpus-specific vector to the input features and optimizing via CTC yields embeddings that capture transfer-relevant corpus relatedness.
This is the central hypothesis of the paper; supported only by the observed PER gains and qualitative embedding plots, with no formal or independent validation.
ad hoc to paper Exponential temperature increase (T_{k+1}=aT_k) provides a useful curriculum from broad to focused sampling.
No ablation or theoretical justification is given; T0 and a are fixed values chosen for the experiments.

invented entities (1)

Corpus-level embedding vector ei
purpose: An additive bias to input features that encodes corpus identity and is used to compute cross-corpus similarity scores for sampling.
It is a new trainable parameter, not a physical entity; the only evidence for its usefulness is the reported experimental improvement and qualitative clustering, and no external falsifiable signature is provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Multilingual Speech Recognition with Corpus Relatedness Sampling." pith.science (2026). https://pith.science/paper/7EBG5LMD

@misc{pith2026190801060,
  author       = {Pith},
  title        = {Pith review of: Multilingual Speech Recognition with Corpus Relatedness Sampling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7EBG5LMD}},
  note         = {Machine review of arXiv:1908.01060}
}

read the original abstract

Multilingual acoustic models have been successfully applied to low-resource speech recognition. Most existing works have combined many small corpora together and pretrained a multilingual model by sampling from each corpus uniformly. The model is eventually fine-tuned on each target corpus. This approach, however, fails to exploit the relatedness and similarity among corpora in the training set. For example, the target corpus might benefit more from a corpus in the same domain or a corpus from a close language. In this work, we propose a simple but useful sampling strategy to take advantage of this relatedness. We first compute the corpus-level embeddings and estimate the similarity between each corpus. Next, we start training the multilingual model with uniform-sampling from each corpus at first, then we gradually increase the probability to sample from related corpora based on its similarity with the target corpus. Finally, the model would be fine-tuned automatically on the target corpus. Our sampling strategy outperforms the baseline multilingual model on 16 low-resource tasks. Additionally, we demonstrate that our corpus embeddings capture the language and domain information of each corpus.

Figures

Figures reproduced from arXiv: 1908.01060 by the authors.

**Figure 1.** The acoustic model to optimize corpus embeddings. Our purpose here is to compute the embedding ei for each corpus Ci where ei is expected to encode information about its corpus Ci. Those embeddings can be jointly trained with the standard multilingual model [5]. First, the embedding matrix E for all corpora is initialized, the i-th row of E is corresponding to the embedding ei of the corpus Ci. Next, during the trai… view at source ↗

**Figure 2.** Domain plot of 36 corpora, the corpus embeddings are reduced to 2 dimensions by t-SNE tionally, the bottom half of the table indicates that our embeddings are able to capture domain level information as well. For instance, the top 2 related corpus for Amharic is Bengali and Swahili. According to Table.1, those three corpora belong to the telephone domain. In addition, Dutch is a read corpus, its top 2 related corpo… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 33 canonical work pages

[1]

However, only a small portion of languages have clean speech labeled corpus

Introduction In recent years, Deep Neural Networks (DNNs) have been suc- cessfully applied to Automatic Speech Recognition (ASR) for many well-resourced languages including Mandarin and En- glish [1, 2]. However, only a small portion of languages have clean speech labeled corpus. As a result, there is an increasing interest in building speech recognition ...

work page
[2]

We propose a corpus-level embedding which can capture the language and domain information of each corpus

work page
[3]

It outperforms the pre- trained model and ﬁne-tuned model on all of our test corpora

We introduce the Corpus Relatedness Sampling strategy to train multilingual models. It outperforms the pre- trained model and ﬁne-tuned model on all of our test corpora

work page
[4]

Related Work Multilingual speech recognition has explored various models to share parameters across languages in different ways. For ex- ample, parameters can be shared by using posterior features from other languages [6], applying the same GMM components across different HMM states [7], training shared hidden layers in DNNs [3, 4] or LSTM [5], using lang...

work page Pith review arXiv 1908
[5]

Approach In this section, we describe our approach to compute the corpus embedding and our Corpus Relatedness Sampling strategy. 3.1. Corpus Embedding Suppose thatCt is the target low-resource corpus, we are in- terested in optimizing the acoustic model with a much larger training corpora setS ={C1,C2...Cn} where n is the number of corpora andCt∈S . Each ...

work page
[6]

Experiments To demonstrate that our sampling approach could improve the multilingual model, we conduct experiments on 16 corpora to compare our approach with the pretrained model and ﬁne-tuned model. 4.1. Datasets We ﬁrst describe our corpus collection. Table.1 lists all corpora we used in the experiments. There are 16 corpora from 10 lan- guages. To incr...

work page
[7]

We also introduce Corpus Relatedness Sampling approach to train multilingual speech recognition models based on those corpus embeddings

Conclusion In this work, we propose an approach to compute corpus-level embeddings. We also introduce Corpus Relatedness Sampling approach to train multilingual speech recognition models based on those corpus embeddings. Our experiment shows that our approach outperforms the ﬁne-tuned multilingual models in all 16 test corpora by 1.6 phone error rate on a...

work page
[8]

HR0011-15-C-0114

Acknowledgements This project was sponsored by the Defense Advanced Re- search Projects Agency (DARPA) Information Innovation Of- ﬁce (I2O), program: Low Resource Languages for Emergent Incidents (LORELEI), issued by DARPA/I2O under Contract No. HR0011-15-C-0114

Show all 41 references

[9]

Deep speech 2: End-to-end speech recognition in english and mandarin,

D. Amodei, S. Ananthanarayanan, R. Anubhai, J. Bai, E. Bat- tenberg, C. Case, J. Casper, B. Catanzaro, Q. Cheng, G. Chen et al., “Deep speech 2: End-to-end speech recognition in english and mandarin,” in International conference on machine learning, 2016, pp. 173–182

2016
[10]

Achieving human parity in conversational speech recognition,

W. Xiong, J. Droppo, X. Huang, F. Seide, M. Seltzer, A. Stolcke, D. Yu, and G. Zweig, “Achieving human parity in conversational speech recognition,”arXiv preprint arXiv:1610.05256, 2016

2016 arXiv
[11]

Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers,

J.-T. Huang, J. Li, D. Yu, L. Deng, and Y . Gong, “Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers,” inAcoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. IEEE, 2013, pp. 7304–7308

2013
[12]

Multilingual acoustic models using dis- tributed deep neural networks,

G. Heigold, V . Vanhoucke, A. Senior, P. Nguyen, M. Ranzato, M. Devin, and J. Dean, “Multilingual acoustic models using dis- tributed deep neural networks,” in Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on . IEEE, 2013, pp. 8619–8623

2013
[13]

Sequence- based multi-lingual low resource speech recognition,

S. Dalmia, R. Sanabria, F. Metze, and A. W. Black, “Sequence- based multi-lingual low resource speech recognition,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018, pp. 4909–4913

2018
[14]

Cross-domain and cross-language portability of acoustic features estimated by multilayer perceptrons,

A. Stolcke, F. Grezl, M.-Y . Hwang, X. Lei, N. Morgan, and D. Vergyri, “Cross-domain and cross-language portability of acoustic features estimated by multilayer perceptrons,” in Acous- tics, Speech and Signal Processing, 2006. ICASSP 2006 Proceed- ings. 2006 IEEE International...

2006
[15]

Multilingual acoustic modeling for speech recognition based on subspace gaussian mixture models,

L. Burget, P. Schwarz, M. Agarwal, P. Akyazi, K. Feng, A. Ghoshal, O. Glembek, N. Goel, M. Karaﬁ ´at, D. Povey et al., “Multilingual acoustic modeling for speech recognition based on subspace gaussian mixture models,” inAcoustics Speech and Sig- nal Processing (ICASSP), 2010 I...

2010
[16]

The language-independent bottleneck features,

K. Vesel `y, M. Karaﬁ ´at, F. Gr´ezl, M. Janda, and E. Egorova, “The language-independent bottleneck features,” in Spoken Language Technology Workshop (SLT), 2012 IEEE. IEEE, 2012, pp. 336– 341

2012
[17]

Domain robust fea- ture extraction for rapid low resource asr development,

S. Dalmia, X. Li, F. Metze, and A. W. Black, “Domain robust fea- ture extraction for rapid low resource asr development,” in 2018 IEEE Spoken Language Technology Workshop (SLT) . IEEE, 2018, pp. 258–265

2018
[18]

Language-independent and language- adaptive acoustic modeling for speech recognition,

T. Schultz and A. Waibel, “Language-independent and language- adaptive acoustic modeling for speech recognition,”Speech Com- munication, vol. 35, no. 1-2, pp. 31–51, 2001

2001
[19]

An investigation of deep neural networks for multilingual speech recognition training and adaptation,

S. Tong, P. N. Garner, and H. Bourlard, “An investigation of deep neural networks for multilingual speech recognition training and adaptation,” Tech. Rep., 2017

2017
[20]

Multilingual multilayer perceptron for rapid language adaptation between and across language families

N. T. Vu and T. Schultz, “Multilingual multilayer perceptron for rapid language adaptation between and across language families.” in Interspeech, 2013, pp. 515–519

2013
[21]

Character-aware neural language models,

Y . Kim, Y . Jernite, D. Sontag, and A. M. Rush, “Character-aware neural language models,” in Thirtieth AAAI Conference on Artiﬁ- cial Intelligence, 2016

2016
[22]

Enrich- ing word vectors with subword information,

P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, “Enrich- ing word vectors with subword information,” Transactions of the Association for Computational Linguistics , vol. 5, pp. 135–146, 2017

2017
[23]

Skip-thought vectors,

R. Kiros, Y . Zhu, R. R. Salakhutdinov, R. Zemel, R. Urtasun, A. Torralba, and S. Fidler, “Skip-thought vectors,” in Advances in neural information processing systems, 2015, pp. 3294–3302

2015
[24]

Distributed representations of sentences and documents,

Q. Le and T. Mikolov, “Distributed representations of sentences and documents,” inInternational conference on machine learning, 2014, pp. 1188–1196

2014
[25]

Googles multilingual neural machine translation system: Enabling zero- shot translation,

M. Johnson, M. Schuster, Q. V . Le, M. Krikun, Y . Wu, Z. Chen, N. Thorat, F. Vi´egas, M. Wattenberg, G. Corrado et al., “Googles multilingual neural machine translation system: Enabling zero- shot translation,” Transactions of the Association for Computa- tional Linguistics, ...

2017
[26]

Multi-dialect speech recognition with a single sequence-to-sequence model,

B. Li, T. N. Sainath, K. C. Sim, M. Bacchiani, E. Weinstein, P. Nguyen, Z. Chen, Y . Wu, and K. Rao, “Multi-dialect speech recognition with a single sequence-to-sequence model,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 20...

2018
[27]

A curriculum learning method for improved noise robustness in automatic speech recogni- tion,

S. Braun, D. Neil, and S.-C. Liu, “A curriculum learning method for improved noise robustness in automatic speech recogni- tion,” in2017 25th European Signal Processing Conference (EU- SIPCO). IEEE, 2017, pp. 548–552

2017
[28]

Active learning: Theory and ap- plications to automatic speech recognition,

G. Riccardi and D. Hakkani-Tur, “Active learning: Theory and ap- plications to automatic speech recognition,”IEEE transactions on speech and audio processing, vol. 13, no. 4, pp. 504–511, 2005

2005
[29]

Multilingual representations for low resource speech recognition and keyword search,

J. Cui, B. Kingsbury, B. Ramabhadran, A. Sethy, K. Audhkhasi, X. Cui, E. Kislal, L. Mangu, M. Nussbaum-Thom, M. Picheny et al. , “Multilingual representations for low resource speech recognition and keyword search,” in2015 IEEE Workshop on Au- tomatic Speech Recognition and Un...

2015
[30]

TED-LIUM: an auto- matic speech recognition dedicated corpus

A. Rousseau, P. Del ´eglise, and Y . Esteve, “TED-LIUM: an auto- matic speech recognition dedicated corpus.” in LREC, 2012, pp. 125–129

2012
[31]

Hkust/mts: A very large scale mandarin telephone speech cor- pus,

Y . Liu, P. Fung, Y . Yang, C. Cieri, S. Huang, and D. Graff, “Hkust/mts: A very large scale mandarin telephone speech cor- pus,” in Chinese Spoken Language Processing. Springer, 2006, pp. 724–735

2006
[32]

SWITCH- BOARD: Telephone speech corpus for research and develop- ment,

J. J. Godfrey, E. C. Holliman, and J. McDaniel, “SWITCH- BOARD: Telephone speech corpus for research and develop- ment,” in Acoustics, Speech, and Signal Processing, 1992. ICASSP-92., 1992 IEEE International Conference on , vol. 1. IEEE, 1992, pp. 517–520

1992
[33]

Thchs-30 : A free chinese speech corpus,

Z. Z. Dong Wang, Xuewei Zhang, “Thchs-30 : A free chinese speech corpus,” 2015. [Online]. Available: http: //arxiv.org/abs/1512.01882

2015 arXiv
[34]

Lib- rispeech: an ASR corpus based on public domain audio books,

V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- rispeech: an ASR corpus based on public domain audio books,” in Acoustics, Speech and Signal Processing (ICASSP), 2015 IEEE International Conference on. IEEE, 2015, pp. 5206–5210

2015
[35]

The ﬁsher corpus: a resource for the next generations of speech-to-text

C. Cieri, D. Miller, and K. Walker, “The ﬁsher corpus: a resource for the next generations of speech-to-text.” inLREC, vol. 4, 2004, pp. 69–71

2004
[36]

Primewords chi- nese corpus set 1,

L. Primewords Information Technology Co., “Primewords chi- nese corpus set 1,” 2018, https://www.primewords.cn

2018
[37]

Turkish broadcast news speech and transcripts ldc2012s06,

M. Saraclar, “Turkish broadcast news speech and transcripts ldc2012s06,” Philadelphia, Linguistic Data Consortium, Web Download, 2012

2012
[38]

Con- nectionist temporal classiﬁcation: labelling unsegmented se- quence data with recurrent neural networks,

A. Graves, S. Fern ´andez, F. Gomez, and J. Schmidhuber, “Con- nectionist temporal classiﬁcation: labelling unsegmented se- quence data with recurrent neural networks,” in Proceedings of the 23rd international conference on Machine learning . ACM, 2006, pp. 369–376

2006
[39]

EESEN: End-to-end speech recognition using deep RNN models and WFST-based decoding,

Y . Miao, M. Gowayyed, and F. Metze, “EESEN: End-to-end speech recognition using deep RNN models and WFST-based decoding,” in Automatic Speech Recognition and Understanding (ASRU), 2015 IEEE Workshop on. IEEE, 2015, pp. 167–174

2015
[40]

Epitran: Precision G2P for many languages

D. R. Mortensen, S. Dalmia, and P. Littell, “Epitran: Precision G2P for many languages.” inLREC, 2018

2018
[41]

Visualizing data using t-sne,

L. v. d. Maaten and G. Hinton, “Visualizing data using t-sne,” Journal of machine learning research, vol. 9, no. Nov, pp. 2579– 2605, 2008

2008

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

However, only a small portion of languages have clean speech labeled corpus

Introduction In recent years, Deep Neural Networks (DNNs) have been suc- cessfully applied to Automatic Speech Recognition (ASR) for many well-resourced languages including Mandarin and En- glish [1, 2]. However, only a small portion of languages have clean speech labeled corpus. As a result, there is an increasing interest in building speech recognition ...

work page

[2] [2]

We propose a corpus-level embedding which can capture the language and domain information of each corpus

work page

[3] [3]

It outperforms the pre- trained model and ﬁne-tuned model on all of our test corpora

We introduce the Corpus Relatedness Sampling strategy to train multilingual models. It outperforms the pre- trained model and ﬁne-tuned model on all of our test corpora

work page

[4] [4]

Related Work Multilingual speech recognition has explored various models to share parameters across languages in different ways. For ex- ample, parameters can be shared by using posterior features from other languages [6], applying the same GMM components across different HMM states [7], training shared hidden layers in DNNs [3, 4] or LSTM [5], using lang...

work page Pith review arXiv 1908

[5] [5]

Approach In this section, we describe our approach to compute the corpus embedding and our Corpus Relatedness Sampling strategy. 3.1. Corpus Embedding Suppose thatCt is the target low-resource corpus, we are in- terested in optimizing the acoustic model with a much larger training corpora setS ={C1,C2...Cn} where n is the number of corpora andCt∈S . Each ...

work page

[6] [6]

Experiments To demonstrate that our sampling approach could improve the multilingual model, we conduct experiments on 16 corpora to compare our approach with the pretrained model and ﬁne-tuned model. 4.1. Datasets We ﬁrst describe our corpus collection. Table.1 lists all corpora we used in the experiments. There are 16 corpora from 10 lan- guages. To incr...

work page

[7] [7]

We also introduce Corpus Relatedness Sampling approach to train multilingual speech recognition models based on those corpus embeddings

Conclusion In this work, we propose an approach to compute corpus-level embeddings. We also introduce Corpus Relatedness Sampling approach to train multilingual speech recognition models based on those corpus embeddings. Our experiment shows that our approach outperforms the ﬁne-tuned multilingual models in all 16 test corpora by 1.6 phone error rate on a...

work page

[8] [8]

HR0011-15-C-0114

Acknowledgements This project was sponsored by the Defense Advanced Re- search Projects Agency (DARPA) Information Innovation Of- ﬁce (I2O), program: Low Resource Languages for Emergent Incidents (LORELEI), issued by DARPA/I2O under Contract No. HR0011-15-C-0114