Pith. sign in

REVIEW 4 major objections 7 minor 38 references

CMT-LLM: Contextual Multi-Talker ASR Utilizing Large Language Models

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that one LLM decoder, given a filtered rare-word prompt, can jointly transcribe overlapping speakers and bias toward rare words, beating separate multi-talker and contextual-biasing baselines on LibriMix and AMI.

desk verdict A practical, well-built combination of SOT multi-talker ASR and LLM prompt biasing with a CTC filter for large lists; the gains are plausible but need stronger baselines and per-condition coverage before I'd trust the headline numbers. read the letter →

arxiv 2506.12059 v1 pith:BKPCKBDI submitted 2025-05-31 eess.AS cs.AIcs.CLcs.SD

Jiajun He , Naoki Sawada , Koichi Miyazaki , Tomoki Toda This is my paper · ORCID
classification eess.AScs.AIcs.CLcs.SD
keywords multi-talkerASRcontextualbiasinglargelanguagemodelsserializedoutputtrainingrarewordrecognitionprompt-basedtwo-stagefilteringLoRAfinetuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that overlapping-speech recognition and rare-word biasing are better solved as one task than as two stacked pipelines. It builds an ASR system from a pretrained speech encoder, a projector, and an LLM decoder trained with serialized output, and it feeds the LLM a short prompt containing the rare words that a two-stage filter selects from large biasing lists. If the claim holds, a single model can transcribe meetings or customer-service calls where several people talk over each other while still getting names and technical terms right, without a separate error-correction stage. The reported numbers, 7.9% WER on LibriMix and 32.9% on AMI SDM with 1,000 distractors, are offered as evidence that the combination works better than either traditional multi-talker models or post-processing contextual biasing.

What carries the argument

The load-bearing mechanism is the pairing of serialized output training (SOT) with a prompt-conditioned LLM decoder: overlapping speakers are transcribed in first-in-first-out order with a special speaker-change token, and a filtered list of rare words is placed in the prompt. The secondary machinery is the two-stage filter: a finetuned WavLM-Large encoder with a CTC head is decoded greedily, the hypothesis is stripped of its 5,000 most common words, substrings are matched to biasing-list entries by word-based edit distance, and the top-10 matches per candidate are merged into the final prompt. The LLM (Vicuna-7B) stays frozen apart from LoRA adapters, so the trainable parts are the projector and the LoRA weights.

What would settle it

Run CMT-LLM on a held-out set where the stage-1 CTC output is replaced by its own transcript with all rare words randomly misspelled: if B-WER stays roughly constant, the filter is not the deciding factor, but if B-WER jumps toward the no-biasing level, the filter is load-bearing. A simpler check computes B-WER with the filtered prompt versus a prompt containing the ground-truth rare words on the same utterances; the gap isolates how much rare-word accuracy the filter itself costs.

Watch

Extended reading notes

Core claim

The central discovery is that prompt-based contextual biasing survives contact with overlapping speech when the prompt list is kept small and relevant. The authors show that an LLM decoder trained with serialized output training can read speaker-change markers and, at the same time, make use of a list of rare words inserted into its instruction prompt; a coarse CTC model provides the initial hypothesis from which a word-edit-distance filter pulls the few dozen words worth biasing. Their experiments establish that this joint decoder beats the LLM baseline without a biasing list and beats a post-processing corrector (ED-CEC) at matched biasing-list sizes, while the filtering step keeps performance from collapsing when the candidate list grows to thousands of words.

Load-bearing premise

Everything hinges on the stage-1 CTC decoder's greedy hypothesis being close enough to the true rare words that a word-edit-distance match against thousands of distractors still finds them; the paper's own coverage numbers (87.40% at 1,000 distractors, dropping to 83.07% at 5,000) expose that assumption, and if the first pass degrades on harder acoustics the prompt loses the very words it should bias.

Editorial extensions

If this is right

  • Meeting transcription systems can drop the separate error-correction stage: one decoder handles speaker order, overlap, and rare vocabulary together.
  • Contextual biasing by prompt is more effective than post-processing: at 100 distractors CMT-LLM reaches 8.7% B-WER on LibriMix versus 15.3% for ED-CEC, and the gap persists across list sizes.
  • Large biasing lists of thousands of words are usable if a cheap first-pass CTC filters them; without filtering, prompt lists beyond roughly 300 words degrade performance below the no-biasing baseline.
  • Target-word coverage from the filter, not just list size, drives B-WER: coverage falls from 87.40% to 83.07% as distractors grow from 1,000 to 5,000, and the paper shows B-WER tracks coverage directly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same filtering-plus-prompt pattern could be borrowed by other LLM-based speech tasks, such as keyword spotting or named-entity tagging in meetings, where the first-pass hypothesis selects the prompt content.
  • Because the paper reports that phoneme-based and semantic matching were slower and less effective, a testable extension is a phonetic-distance variant that might recover target words when greedy CTC output is misspelled but phonetically close.
  • The ground-truth-rare-words upper bound of 2.6% B-WER on LibriMix suggests most remaining rare-word errors come from the filter missing targets, so improving coverage should matter more than enlarging the prompt.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes CMT-LLM, a unified framework for multi-talker ASR and contextual biasing. The architecture consists of a pretrained WavLM speech encoder, a downsampler/projector, and a Vicuna-7B LLM decoder trained with SOT-style transcriptions, where the LLM receives a prompt containing rare words from a biasing list. To handle large lists, the authors introduce a two-stage filtering approach: a CTC-finetuned WavLM produces greedy hypotheses, from which common words are removed, sub-word segments are generated, and word-level edit distance against the biasing list selects the Top-10 most relevant entries per candidate. Experiments on LibriMix and AMI (IHM-Mix, SDM, MDM) report lower WER than several baselines, including a no-biasing LLM baseline and an ED-CEC post-processing baseline, at biasing sizes up to 5,000 distractors.

Significance. If the reported results hold, the paper makes a useful contribution by demonstrating that a single LLM-based decoder can jointly address overlapping speech and rare-word biasing, moving beyond separate multi-talker and contextual-biasing pipelines. The construction of AMI biasing lists from OCR-extracted slide text is a practical design choice, and the oracle (GT Rare Words) and anti-context controls provide a clear way to isolate the effect of the biasing prompt. The sensitivity analysis in Fig. 3 is a valuable sanity check on list size and coverage. The manuscript is, however, an empirical systems paper whose central claims rest on a small number of benchmark numbers; those numbers are not yet reported with the statistical care needed to establish the claimed improvements, and one headline number is internally inconsistent.

major comments (4)
  1. [Table 1 vs. Table 3 and Abstract] There is a direct conflict in the headline LibriMix test-set number. Table 1 reports CMT-LLM (ours) at 7.3% WER under the caption 'with 1,000 distractors,' while Table 3 reports CMT-LLM + 1,000 distractors at 7.9% WER, and the Abstract states 7.9% on LibriMix. These cannot both be correct for the same setup. Please clarify which configuration Table 1 refers to; if Table 1 is a no-biasing or different-biasing condition, the caption is misleading, and if it is the same condition, the discrepancy must be resolved before the paper can be accepted.
  2. [Tables 2 and 3; Section 3.3] The central empirical claim that CMT-LLM 'significantly outperforms' the LLM Baseline is not supported by any measure of uncertainty or significance. For example, on AMI SDM test, the difference is 32.9 vs. 34.2 WER (Table 3), and on MDM it is 30.4 vs. 31.2; these differences are small and may be within run-to-run or seed-to-seed variation. The paper reports no error bars, no multiple seeds, and no significance test (e.g., matched-pair test over test utterances). Please provide variance estimates over at least a few training seeds or a significance test to substantiate the word 'significantly' in Section 3.3.
  3. [Section 3.3, coverage numbers; Section 2.2] The two-stage filtering pipeline is load-bearing for the large-list claims, but the paper reports only aggregate target-word coverage (87.40%, 85.07%, 83.07% for 1,000, 2,000, and 5,000 distractors) without any breakdown by dataset or microphone condition. This matters specifically for AMI SDM, where the stage-1 WavLM-CTC model has 44.0% WER (Table 2); if its greedy hypotheses are corrupted, the edit-distance matching in Section 2.2 may fail to recover target rare words, so the small SDM gain (32.9 vs. 34.2) may not come from successful biasing. Please report coverage separately for LibriMix, IHM-Mix, SDM, and MDM, and, if possible, the distribution of edit distances between filtered candidates and true rare words for each condition.
  4. [Table 3; Section 1] The contextual-biasing comparison in Table 3 is limited to ED-CEC and the authors' own LLM baselines. The introduction cites CTC-assisted LLM-based contextual ASR [21] and deep biasing [6], both of which are closely related to the proposed two-stage filtering and LLM-based decoding; yet these are not compared in Table 3 or elsewhere. Given the abstract's claim of outperforming 'traditional contextual biasing methods,' the absence of these stronger cited baselines makes the comparison incomplete. Please either add these baselines under the same setup or state clearly why they are excluded.
minor comments (7)
  1. [Abstract] There is a missing space in 'recognition.Experiments' in the full-text version of the Abstract; please fix the typesetting.
  2. [Figure 1] The caption describes 'visual-grounded contextual multi-talker ASR pipeline,' but the visual grounding (the slide image and OCR) is not described in Section 2.2 and the model itself does not appear to use visual features. Please clarify the role of the visual content in the proposed method or adjust the wording.
  3. [Section 2.2] The description of the training-time biasing list is under-specified: 'The biasing list size in the LLM prompt is limited to 100 words' — is this 100 words taken from the ground-truth rare-word set, or from filtered hypotheses? Clarify how the 100-word training prompt is constructed to avoid confusing it with the inference-time filtering.
  4. [Section 2.2] The statement that phoneme-based edit distance and text semantic similarity are 'slower and less effective' than word-based edit distance is an empirical claim with no supporting experiment or reference; please either provide evidence or soften the claim.
  5. [Figure 3] The legend in Figure 3 combines WER and B-WER curves with 'Avg Bias List Size' and 'Bias Word Coverage (%)' on what appears to be a single axis, which is confusing because these quantities have different units. Please split the panels or use separate axes and clearly label which panel each legend entry applies to.
  6. [References] Reference [25] is cited as 'Decoupled weight decay regularization' by Loshchilov; this should cite the AdamW paper (ICLR 2019) with the full author list and publication venue.
  7. [References] The GitHub URL in the reference for deep biasing contains a space ('is21 deep bias'), which will likely break the link; please fix it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims are empirical and externally anchored; no derivation step reduces to its own inputs.

full rationale

CMT-LLM is an empirical systems paper. The central claims—that an LLM-based SOT decoder with prompt-based contextual biasing outperforms separate multi-talker and contextual-bias pipelines—are supported by comparisons against external baselines and by experiments on LibriMix and AMI. The biasing lists are constructed from external sources (the LibriSpeech-derived full list and OCR-extracted AMI lecture slides), not from the model's own outputs. The GT-rare-words condition is explicitly presented as an oracle upper bound, not as a fitted parameter or as the reported system. The two-stage filtering algorithm uses a separately finetuned WavLM CTC model with greedy decoding and word-level edit distance; its outputs are evaluated by target-bias-word coverage and B-WER, and they are not used to train the CMT-LLM or to set its parameters. The citation of the authors' own ED-CEC work appears only as a baseline comparison, not as load-bearing justification for the method. No uniqueness theorem, no ansatz smuggled via self-citation, and no prediction that is identical by construction to an input are present. The operational definition of rare words by membership in a list is standard for contextual biasing evaluation and does not constitute circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on an empirical pipeline rather than on mathematical derivation, so the ledger is dominated by modeling assumptions and hyperparameters. The free parameters are optimization and filtering choices; the axioms are background assumptions about the SOT label format, the transferability of the pretrained LLM and SSL encoder, and the reliability of the CTC hypotheses and OCR-derived context. None of these is independently machine-checked in the paper.

free parameters (6)
  • LoRA rank and alpha = r=8, alpha=32
    Section 3.1; controls the size of LLM adaptation; no ablation or tuning analysis provided.
  • Learning rate and warmup = 1e-4, 1000 steps
    Section 3.1; standard AdamW choices, not justified from data.
  • Downsampling factor n = 5
    Section 2.2; changes speech frame rate from 50 Hz to 10 Hz; hand-chosen to match LLM input length.
  • Training-time prompt list size cap = 100 words
    Section 2.2; explicitly capped to minimize training computation; affects how the model learns to use context.
  • Common-word removal threshold = 5,000
    Section 2.2; words among the most common 5,000 are discarded before edit-distance matching; threshold selected without reported sensitivity analysis.
  • Top-N related words per segment = 10
    Section 2.2; each segment matches its Top-10 list words; the paper's own Figure 3(c) shows Top-N strongly affects error rate, so this choice matters.
assumptions (5)
  • domain assumption SOT FIFO ordering with <sc> tokens adequately represents overlapping speech transcripts
    Invoked in Section 2.1 and Figure 1(a); if speakers start nearly simultaneously or the order is ambiguous, the target labels become ill-defined.
  • domain assumption Vicuna-7B pretrained knowledge transfers to ASR decoding from speech embeddings
    Section 2.2 and 3.1; no analysis of what the LLM contributes beyond conditional language modeling.
  • domain assumption Stage-1 finetuned WavLM CTC greedy decoding preserves enough acoustic fragments of target rare words
    Section 2.2; this is the premise of the two-stage filter and is only indirectly supported by coverage numbers.
  • domain assumption Tesseract OCR of AMI slides yields a valid biasing context list
    Section 3.2; OCR errors would change the target rare-word set for evaluation.
  • ad hoc to paper Rare words are those in the full 209.2K list or with corpus frequency below 100
    Section 3.2; this threshold defines the evaluation target and is not motivated independently.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CMT-LLM: Contextual Multi-Talker ASR Utilizing Large Language Models." pith.science (2026). https://pith.science/paper/BKPCKBDI

@misc{pith2026250612059,
  author       = {Pith},
  title        = {Pith review of: CMT-LLM: Contextual Multi-Talker ASR Utilizing Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BKPCKBDI}},
  note         = {Machine review of arXiv:2506.12059}
}
read the original abstract

In real-world applications, automatic speech recognition (ASR) systems must handle overlapping speech from multiple speakers and recognize rare words like technical terms. Traditional methods address multi-talker ASR and contextual biasing separately, limiting performance in complex scenarios. We propose a unified framework that combines multi-talker overlapping speech recognition and contextual biasing into a single task. Our ASR method integrates pretrained speech encoders and large language models (LLMs), using optimized finetuning strategies. We also introduce a two-stage filtering algorithm to efficiently identify relevant rare words from large biasing lists and incorporate them into the LLM's prompt input, enhancing rare word recognition. Experiments show that our approach outperforms traditional contextual biasing methods, achieving a WER of 7.9% on LibriMix and 32.9% on AMI SDM when the biasing size is 1,000, demonstrating its effectiveness in complex speech scenarios.

Figures

Figures reproduced from arXiv: 2506.12059 by the authors.

Figure 1
Figure 1. Illustration of the visual-grounded contextual multi-talker ASR pipeline. LLM Use the rare words provided to improve the accuracy of ASR if they are relevant. The rare words are []. Speech Encoder Downsampler Projector S1 S2 S3 S4 P1 P2 Pn T1 T2 LoRA Audio Prompt Sm ... ... LLM Tokenizer LLM Tokenizer ASR Input <BOS> Text 1 <sc> Text 2 <sc> Text 3 1 1 ... T1 T2 2 2 ... T1 T2 3 3 P ... 3 T1 T2 1 1 ... T1 T2 2 2 ... T… view at source ↗
Figure 2
Figure 2. Overall architecture of the CMT-LLM model. (T 1 1 , T 1 2 , · · · , <sc>, T 2 1 , T 2 2 , · · · , <sc>, T 3 1 , T 3 2 , · · ·) ∈ R N is the token sequence. In T, T b a represents the a-th token of the b-th speaker and N denotes the length of the transcript. Follow￾ing previous works [16, 3], we use the SOT method to address the multi-talker ASR problem. Specifically, assuming the num￾ber of speakers is 3, the transc… view at source ↗
Figure 3
Figure 3. Impact of Biasing List Size, Coverage, and Filtering on ASR Performance. out explicit context. ED-CEC [8], a SOTA contextual ASR post-processing method, improves B-WER with a small biasing list (+100 distractors) but becomes less effective as the list size increases, owing to growing ambiguity from additional distrac￾tors. In contrast, CMT-LLM, which integrates the biasing list directly into the prompt, performs sig… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 29 canonical work pages

  1. [21]

    Seri- alized output training for end-to-end overlapped speech recogni- tion,

    N. Kanda, Y . Gaur, X. Wang, Z. Meng, and T. Yoshioka, “Seri- alized output training for end-to-end overlapped speech recogni- tion,” in Proc. Interspeech, 2020, pp. 2797–2801

    2020

  2. [15]

    PMF-CEC: Phoneme-augmented multimodal fusion for context-aware ASR error correction with error-specific selective decoding,

    J. He and T. Toda, “PMF-CEC: Phoneme-augmented multimodal fusion for context-aware ASR error correction with error-specific selective decoding,” arXiv preprint, 2025

    work page 2025

  3. [6]

    Multitalker speech separation with utterance-level permutation invariant training of deep recurrent neural networks,

    M. Kolbæk, D. Yu, Z.-H. Tan, and J. Jensen, “Multitalker speech separation with utterance-level permutation invariant training of deep recurrent neural networks,”IEEE/ACM Trans. Audio, Speech and Lang. Proc., vol. 25, no. 10, pp. 1901–1913, 2017

    work page 1901

  4. [1]

    Existing methods include permutation invariant training [1], heuristic error assignment [2], and serialized output training (SOT) [3]

    Introduction Multi-talker automatic speech recognition (ASR), particularly in overlapping speech scenarios, remains a major challenge. Existing methods include permutation invariant training [1], heuristic error assignment [2], and serialized output training (SOT) [3]. Among these, SOT has gained attention for resolving speaker arrangement uncertainty by ...

    work page

  5. [2]

    CMT-LLM: Contextual Multi-Talker ASR Utilizing Large Language Models

    Proposed Method 2.1. Problem Formulation The contextual multi-talker ASR problem can be formalized as the mapping function f (S, C) = T , where the speech S = ( S1, S2, · · ·, SM ) contains M acoustic frames, the con- text C = ( C1, C2, · · ·, CL) ∈ RL denotes the biasing list containing L rare words, and the ground truth transcript T = arXiv:2506.12059v1...

    work page Pith review arXiv 2025

  6. [3]

    Transcribe speech to text

    Experimental Evaluation 3.1. Implementation Details Our method was trained on 4 NVIDIA A100 80 GB GPUs and the batch size was set to 2. We employed the finetuned WavLM-Large [22] as the speech encoder, processing 16 kHz sampled audio into feature embeddings with a frame rate of 50 Hz and a dimension of 1,024. These embeddings underwent downsampling (n = 5...

    work page 2021

  7. [4]

    Conclusion This paper for the first time introduces an LLM-based SOT method CMT-LLM for multi-talker contextual ASR. Leverag- ing its strong decoding capabilities, deep comprehension of long-range context, and cross-speaker modeling ability, LLM excels in handling complex multi-talker speech scenarios. In addition, CMT-LLM effectively incorporates context...

    work page

  8. [5]

    Acknowledgments This work was partly supported by JST CREST Grant Number JPMJCR22D1, Japan

    work page

Show all 38 references
  1. [7]

    Streaming end-to-end multi-talker speech recognition,

    L. Lu, N. Kanda, J. Li, and Y . Gong, “Streaming end-to-end multi-talker speech recognition,” IEEE Signal Processing Letters, vol. 28, pp. 803–807, 2021

    2021

  2. [8]

    Serialized speech infor- mation guidance with overlapped encoding separation for multi- speaker automatic speech recognition,

    H. Shi, Y . Gao, Z. Ni, and T. Kawahara, “Serialized speech infor- mation guidance with overlapped encoding separation for multi- speaker automatic speech recognition,” in Proc. SLT, 2024, pp. 1–7

    2024

  3. [9]

    Enhancing recognition of rare words in ASR through error detection and context-aware error correc- tion,

    J. He, Z. Yang, and T. Toda, “Enhancing recognition of rare words in ASR through error detection and context-aware error correc- tion,” IEICE Tech. Rep., vol. 123, no. 292, pp. 13–18, 2023

    2023

  4. [10]

    Deep shallow fusion for RNN-T personalization,

    D. Le, G. Keren, J. Chan, J. Mahadeokar, C. Fuegen, and M. L. Seltzer, “Deep shallow fusion for RNN-T personalization,” in Proc. SLT, 2021, pp. 251–257

    2021

  5. [11]

    Graph neural networks for contextual ASR with the tree-constrained pointer generator,

    G. Sun, C. Zhang, and P. C. Woodland, “Graph neural networks for contextual ASR with the tree-constrained pointer generator,” IEEE/ACM Trans. Audio, Speech and Lang. Proc. , vol. 32, pp. 2407–2417, 2024

    2024

  6. [13]

    ED-CEC: Improving rare word recognition using ASR postprocessing based on error detection and context-aware error correction,

    J. He, Z. Yang, and T. Toda, “ED-CEC: Improving rare word recognition using ASR postprocessing based on error detection and context-aware error correction,” in Proc. ASRU, 2023, pp. 1– 6

    2023

  7. [14]

    MF-AED-AEC: Speech emo- tion recognition by leveraging multimodal fusion, ASR error de- tection, and ASR error correction,

    J. He, X. Shi, X. Li, and T. Toda, “MF-AED-AEC: Speech emo- tion recognition by leveraging multimodal fusion, ASR error de- tection, and ASR error correction,” in Proc. ICASSP, 2024, pp. 11 066–11 070

    2024

  8. [16]

    Seed-ASR: Understanding di- verse speech and contexts with LLM-based speech recognition,

    Y . Bai, J. Chen, J. Chen, W. Chen, Z. Chen, C. Ding, L. Dong, Q. Dong, Y . Du, K. Gao et al. , “Seed-ASR: Understanding di- verse speech and contexts with LLM-based speech recognition,” in arXiv preprint arXiv:2407.04675, 2024

    2024 arXiv

  9. [17]

    An embarrassingly simple approach for LLM with strong ASR capacity,

    Z. Ma, G. Yang, Y . Yang, Z. Gao, J. Wang, Z. Du, F. Yu, Q. Chen, S. Zheng, S. Zhang et al. , “An embarrassingly simple approach for LLM with strong ASR capacity,” in arXiv preprint arXiv:2402.08846, 2024

    2024 arXiv

  10. [18]

    Large language model can transcribe speech in multi-talker scenarios with versatile instructions,

    L. Meng, S. Hu, J. Kang, Z. Li, Y . Wang, W. Wu, X. Wu, X. Liu, and H. Meng, “Large language model can transcribe speech in multi-talker scenarios with versatile instructions,” in Proc. ICASSP, 2025, pp. 1–5

    2025

  11. [19]

    Advancing multi-talker ASR performance with large language models,

    M. Shi, Z. Jin, Y . Xu, Y . Xu, S.-X. Zhang, K. Wei, Y . Shao, C. Zhang, and D. Yu, “Advancing multi-talker ASR performance with large language models,” in Proc. SLT, 2024, pp. 14–21

    2024

  12. [20]

    Mala- ASR: Multimedia-assisted LLM-based ASR,

    G. Yang, Z. Ma, F. Yu, Z. Gao, S. Zhang, and X. Chen, “Mala- ASR: Multimedia-assisted LLM-based ASR,” Proc. Interspeech, pp. 2405–2409, 2024

    2024

  13. [22]

    WavLLM: Towards ro- bust and adaptive speech large language model,

    S. Hu, L. Zhou, S. Liu, S. Chen, L. Meng, H. Hao, J. Pan, X. Liu, J. Li, S. Sivasankaran, L. Liu, and F. Wei, “WavLLM: Towards ro- bust and adaptive speech large language model,” inProc. EMNLP, 2024, pp. 4552–4572

    2024

  14. [23]

    Two stage contextual word filtering for context bias in unified stream- ing and non-streaming transducer,

    Z. Yang, S. Sun, X. Wang, Y . Zhang, L. Ma, and L. Xie, “Two stage contextual word filtering for context bias in unified stream- ing and non-streaming transducer,” inProc. Interspeech, 2023, pp. 3257–3261

    2023

  15. [24]

    Contextualized end-to-end speech recognition with contextual phrase prediction network,

    K. Huang, A. Zhang, Z. Yang, P. Guo, B. Mu, T. Xu, and L. Xie, “Contextualized end-to-end speech recognition with contextual phrase prediction network,” inProc. Interspeech, 2023, pp. 4933– 4937

    2023

  16. [25]

    Adaptive contextual biasing for transducer based streaming speech recognition,

    T. Xu, Z. Yang, K. Huang, P. Guo, A. Zhang, B. Li, C. Chen, C. Li, and L. Xie, “Adaptive contextual biasing for transducer based streaming speech recognition,” in Proc. Interspeech, 2023, pp. 1668–1672

    2023

  17. [26]

    CTC-assisted LLM-based contextual ASR,

    G. Yang, Z. Ma, Z. Gao, S. Zhang, and X. Chen, “CTC-assisted LLM-based contextual ASR,” in Proc. SLT, 2024, pp. 126–131

    2024

  18. [27]

    WavLM: Large-scale self- supervised pre-training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiaoet al., “WavLM: Large-scale self- supervised pre-training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  19. [28]

    Vicuna: An open- source chatbot impressing gpt-4 with 90%* chatgpt quality,

    W.-L. Chiang, Z. Li, Z. Lin, Y . Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y . Zhuang, J. E. Gonzalez et al., “Vicuna: An open- source chatbot impressing gpt-4 with 90%* chatgpt quality,” See https://vicuna. lmsys. org (accessed 14 April 2023) , vol. 2, no. 3, p. 6, 2023

    2023

  20. [29]

    LLaMa: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar et al. , “LLaMa: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  21. [30]

    Decoupled weight decay regularization,

    I. Loshchilov, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  22. [31]

    Multi-speaker ASR combining non-autoregressive conformer CTC and conditional speaker chain,

    P. Guo, X. Chang, S. Watanabe, and L. Xie, “Multi-speaker ASR combining non-autoregressive conformer CTC and conditional speaker chain,” in Proc. Interspeech, 2021, pp. 3720–3724

    2021

  23. [32]

    SURT 2.0: Advances in transducer-based multi-talker speech recognition,

    D. Raj, D. Povey, and S. Khudanpur, “SURT 2.0: Advances in transducer-based multi-talker speech recognition,” IEEE/ACM Trans. Audio, Speech and Lang. Proc. , vol. 31, p. 3800–3813, 2023

    2023

  24. [33]

    Librimix: An open-source dataset for generalizable speech separation,

    J. Cosentino, M. Pariente, S. Cornell, A. Deleforge, and E. Vin- cent, “Librimix: An open-source dataset for generalizable speech separation,” in arXiv preprint arXiv:2005.11262, 2020

    2005 arXiv

  25. [34]

    The AMI meeting corpus: A pre-announcement,

    J. Carletta, S. Ashby, S. Bourban, M. Flynn, M. Guillemot, T. Hain, J. Kadlec, V . Karaiskos, W. Kraaij, M. Kronenthalet al., “The AMI meeting corpus: A pre-announcement,” in Interna- tional workshop on machine learning for multimodal interaction . Springer, 2005, pp. 28–39

    2005

  26. [35]

    Lib- rispeech: an ASR corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- rispeech: an ASR corpus based on public domain audio books,” in Proc. ICASSP, 2015, pp. 5206–5210

    2015

  27. [36]

    Wham!: Extending speech separation to noisy environments,

    G. Wichern, J. Antognini, M. Flynn, L. R. Zhu, E. McQuinn, D. Crow, E. Manilow, and J. L. Roux, “Wham!: Extending speech separation to noisy environments,” inProc. Interspeech, 2019, pp. 1368–1372

    2019

  28. [37]

    Acoustic beamform- ing for speaker diarization of meetings,

    X. Anguera, C. Wooters, and J. Hernando, “Acoustic beamform- ing for speaker diarization of meetings,”IEEE/ACM Trans. Audio, Speech and Lang. Proc., vol. 15, no. 7, pp. 2011–2022, 2007

    2011

  29. [38]

    Contextual- ized streaming end-to-end speech recognition with trie-based deep biasing and shallow fusion,

    D. Le, M. Jain, G. Keren, S. Kim, Y . Shi, J. Mahadeokar, J. Chan, Y . Shangguan, C. Fuegen, O. Kalinliet al., “Contextual- ized streaming end-to-end speech recognition with trie-based deep biasing and shallow fusion,” arXiv preprint arXiv:2104.02194 , 2021

    2021 arXiv

  30. [39]

    Tree-constrained pointer generator with graph neural network encodings for contextual speech recognition,

    G. Sun, C. Zhang, and P. C. Woodland, “Tree-constrained pointer generator with graph neural network encodings for contextual speech recognition,” in Proc. Interspeech, 2022, pp. 2043–2047

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.