Pith. sign in

REVIEW 2 major objections 5 minor 42 references

Effective Context in Neural Speech Models

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Neural speech models use less context than they are given, and HuBERT can stream without retraining.

desk verdict A solid measurement paper with a somewhat oversold streaming claim; the core finding that SSL speech models use short effective context is credible and useful. read the letter →

arxiv 2505.22487 v1 pith:MO2HPKCA submitted 2025-05-28 cs.SD cs.CLeess.AS

Yen Meng , Sharon Goldwater , Hao Tang This is my paper
classification cs.SDcs.CLeess.AS
keywords effectivecontextself-supervisedspeechHuBERTwav2vec2.0WavLMstreamingprobingJacobianinfluence
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to measure how much context neural speech models actually use, as opposed to how much they are given, and proposes two model-agnostic methods for doing so: truncating input windows and computing Jacobian influence. Applied to supervised Transformers, the methods show that fundamental-frequency tracking, phone classification, and word classification use increasing amounts of effective context. Applied to self-supervised models (wav2vec 2.0, HuBERT, WavLM), they show that effective context grows mostly in the early layers and stays short, around 2.0 seconds on either side of the center for HuBERT. Because the effective context is short, the paper argues that HuBERT can be run in streaming mode with only a small lookahead and history, without architecture changes or fine-tuning, with only a small degradation in phone probing accuracy.

What carries the argument

The paper's machinery is a pair of measurement tools rather than a new model. The truncation approach removes all input frames outside a window of size $2W+1$ centered on frame $t$ and measures the $\ell^2$ distance or task error between the resulting output and the full-context output. The Jacobian approach computes the Frobenius norm of the Jacobian of a hidden vector with respect to each input frame, aggregated over a dataset as the relative influence $S(\sigma)$ for each time shift $\sigma$; the center value is summarized by a single statistic they call contextualization, $1 - S(0)$. These tools let the authors compare effective context across layers, models, and supervised tasks, and they supply the evidence that short context suffices for streaming.

What would settle it

Run a full HuBERT-based automatic speech recognition system in streaming mode with a 400 ms lookahead and 2 s history, and compare word error rate against the same system seeing full utterances on a held-out set. A word error rate increase substantially larger than the phone-error increase reported in the paper would show that the phone probe understates the context needed by downstream tasks.

Watch

Extended reading notes

Core claim

The central discovery is that the context a speech Transformer uses can be measured, and that for self-supervised models it is much shorter than the context available. Using truncation, the paper finds HuBERT's effective context is about 2.0 seconds on either side of the center frame; using the Jacobian-based relative influence, it finds that contextualization rises mainly in the first layers and then plateaus, roughly matching a supervised phone classifier and falling well below a supervised word classifier. From this, the paper concludes that HuBERT can be run as a streaming representation extractor, sliding a 400 ms lookahead and roughly 2 seconds of history over the input, with a phone probing error rate of 13.7% versus 11.9% in the full-context setting.

Load-bearing premise

The streaming conclusion rests on the assumption that the phone-classification probe used to measure errors is a faithful stand-in for real downstream tasks; if actual speech recognition needs more context than the probe does, the streaming claim would not hold.

Editorial extensions

If this is right

  • Pretrained HuBERT can be deployed as a low-latency streaming feature extractor by truncating its context, with no additional training or architecture changes.
  • The measured effective context, not the designer-provided context, is what determines whether a model can run in streaming mode.
  • For self-supervised speech models, most contextualization happens in the early layers; later layers add less context, which may guide where to prune or compress.
  • Phone and word probing accuracy correlate with contextualization, suggesting that a minimum effective context is necessary for good performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct testable extension is to attach a streaming ASR decoder to HuBERT with the same 400 ms lookahead and 2 s history and compare word error rate to a full-context decoder; the paper's phone-probe result suggests the gap should be small.
  • The same measurement tools could be applied to long-context models such as Whisper to check whether their extended context is actually used, which would reframe 'long context' as an empirical property rather than an architectural feature.
  • The finding that untrained Transformers gain contextualization through layers without training suggests that architecture alone creates some context mixing, so controlled comparisons need untrained baselines.
  • If effective context is a bottleneck for downstream tasks, training objectives that explicitly reward contextualization could improve self-supervised representations; the paper's correlation results imply such a mechanism.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes two model-agnostic methods for measuring the effective context of neural speech models: a truncation approach that removes input frames outside a centered window and measures the effect on layer outputs, and a Jacobian approach that computes the Frobenius norm of the derivative of a hidden vector with respect to each input frame. These methods are applied to supervised Transformers trained for f0, phone, and word classification, and to self-supervised models (wav2vec 2.0, HuBERT, WavLM). The main findings are that effective context increases in the order f0 < phone < word, that self-supervised models have relatively short effective context similar to the supervised phone model, and that contextualization correlates with phone and word probing performance. As an application, the paper simulates a streaming HuBERT by feeding sliding windows of history and lookahead, reporting small phone-probing error degradations compared with full-context input.

Significance. If the results hold, the paper offers simple, architecture-agnostic tools for quantifying context usage, which is useful for designing efficient streaming speech systems and for understanding what self-supervised models learn. The two measurement approaches are complementary, and their consistency with each other and with prior layer-wise probing studies (e.g., the idiosyncratic last layers of wav2vec 2.0) strengthens the empirical case. The paper does not provide code or release the evaluation scripts, but the methods are straightforward to implement. The streaming demonstration is practically motivated, though its validation is currently limited to phone probing. Overall, the measurement methodology and the cross-model comparisons are valuable contributions, but the headline streaming claim requires additional support.

major comments (2)
  1. [Section 2.2, Eq. (4)] Eq. (4) defines S(σ) as a sum over utterances and frames of the influence s_n(t, t+σ). For a fixed utterance of length T, the number of valid pairs (t, t+σ) is approximately T − |σ|, so S(σ) is systematically smaller for larger |σ| purely because fewer frames contribute, even when the per-frame influence is flat. Since the paper normalizes only by the total sum over σ, the plotted decay in Fig. 2 conflates this triangular boundary mask with true context usage. The contextualization statistic 1−S(0) is also affected: S(0) has the maximum number of contributing frames, so its value is inflated and contextualization appears lower. This is load-bearing because the shape of the relative influence curves, the comparison of effective context across tasks and models, and the conclusion that self-supervised models have short effective context all rely on the Jacobian measurements. The paper should correct this by averaging over the number of valid (t, t+σ) pairs (or by restricting the inner sum to a central segment of frames where all time shifts are valid) and then re-examining the affected figures and conclusions.
  2. [Section 4 and Abstract] The abstract and Section 4 claim that HuBERT can be run in streaming mode without modification to the architecture and without further fine-tuning, with only small degradation (e.g., 13.7% vs 11.9% phone error). This claim is supported only by a linear phone probe on layer 9, using a classifier trained on full-context features and not retrained on the streamed features. Phone probing is a frame-wise, independently classified metric; it does not involve temporal decoding, a lexicon, or a language model, whereas the cited streaming applications [32–35] are ASR systems. The paper's own Fig. 4 shows that word probing requires larger effective context than phone probing, so phone-only results cannot bound the degradation of higher-level tasks. Moreover, because the probe is not retrained, the reported 13.7% is not a clean estimate of what a downstream model trained on streamed features would achieve. To support the streaming claim as stated, the paper should evaluate a downstream word-level task or an actual streaming ASR system under the same history/lookahead windows, or the claim should be qualified to phone probing specifically.
minor comments (5)
  1. [Abstract and Section 4] The Abstract reports a 1.5% degradation for a 2s history, but Section 4 reports 13.7% vs 11.9%, which is a 1.8 percentage-point absolute increase. Please correct the abstract or clarify whether these are relative or absolute changes.
  2. [Section 3.1] The paper states that the symmetry assumption for the truncation approach is 'later verified', but it would be helpful to explicitly point to the verification (e.g., the roughly symmetric shapes in Fig. 2(b)) at the point of the assertion.
  3. [Section 4] The streaming simulation is described only qualitatively as 'run it as a sliding window over the input frames'. Please specify the exact construction of each input window (history length + current frame + lookahead length), how the output frame is selected from the window, and whether the probe is applied to the layer-9 representation at that position, so the experiment is reproducible.
  4. [Figure 3] The x-axis of Fig. 3 is not labeled in the text or caption; from the description the reader must infer that it is the layer index. Please add axis labels and describe the line styles/colours in the caption.
  5. [Section 4] The choice to evaluate streaming on layer 9 only is not justified. Please state why layer 9 was chosen (e.g., it is a typical layer used for phone probing or a representative high-performance layer).

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: effective-context measurements and the streaming simulation are direct empirical evaluations; self-citations are minor and non-load-bearing.

full rationale

I traced the main derivation chain. The effective-context definitions are operational: the truncation measure (Eq. 1) and the Jacobian measure (Eq. 3) are direct sensitivity quantities, not fitted parameters later relabeled as predictions. The reported effective-context values in Section 3 are read off measured curves, and the streaming section in Section 4 explicitly runs HuBERT in sliding-window mode and reports measured phone probing error rates (Fig. 5), rather than predicting those rates from previously fitted constants. The conclusion that HuBERT 'can be run in streaming mode' is therefore an empirical demonstration, not a reduction to the paper's own inputs by construction. Self-citations to [12], [28], and [31] are used as methodological precedent or as prior observations of layer-wise behavior; they are not load-bearing premises on which the central claim depends. The main weaknesses are the reliance on a frame-wise linear phone probe as the only downstream validation of the streaming claim, and the numeric mismatch between the Abstract (1.5% degradation with 2s history) and Section 4 (13.7% vs 11.9%, i.e., 1.8 percentage points); these are evidence-strength and internal-consistency concerns, not circularity. Accordingly, no specific circular step is identified and the score is low.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities or fitted physical constants. The main assumptions are about the validity of the measurement procedures and the transferability of probing results to downstream tasks. The single free parameter listed is a methodological window choice, not a fitted constant in a derivation.

free parameters (1)
  • Relative influence window W = 10s (5s each side)
    The contextualization statistic 1-S(0) depends on the window size used to compute relative influence; the authors choose 10s as sufficiently large for the models studied (Section 3.2). Comparisons across models rely on the same window size.
assumptions (4)
  • domain assumption Symmetric effective context assumption
    Section 2.1 assumes the model's effective context is symmetric around the center frame to justify truncation-based windowing; the paper says it will later verify this.
  • domain assumption Jacobian norm as measure of influence
    Section 2.2 equates the Frobenius norm of the Jacobian of output frame t with respect to input frame tau to the influence of that input, assuming first-order sensitivity captures usage.
  • domain assumption Phone probing accuracy reflects model utility
    Sections 3.1 and 4 use phone error rates from linear probes to assess both effective context and streaming degradation, assuming this predicts downstream performance.
  • domain assumption Absolute position encoding works in sliding windows
    Section 4's streaming runs feed fixed-size windows to HuBERT, whose sinusoidal position encodings encode absolute position; the model must tolerate the center frame appearing at a different absolute position than in full-utterance processing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Effective Context in Neural Speech Models." pith.science (2026). https://pith.science/paper/MO2HPKCA

@misc{pith2026250522487,
  author       = {Pith},
  title        = {Pith review of: Effective Context in Neural Speech Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MO2HPKCA}},
  note         = {Machine review of arXiv:2505.22487}
}
read the original abstract

Modern neural speech models benefit from having longer context, and many approaches have been proposed to increase the maximum context a model can use. However, few have attempted to measure how much context these models actually use, i.e., the effective context. Here, we propose two approaches to measuring the effective context, and use them to analyze different speech Transformers. For supervised models, we find that the effective context correlates well with the nature of the task, with fundamental frequency tracking, phone classification, and word classification requiring increasing amounts of effective context. For self-supervised models, we find that effective context increases mainly in the early layers, and remains relatively short -- similar to the supervised phone model. Given that these models do not use a long context during prediction, we show that HuBERT can be run in streaming mode without modification to the architecture and without further fine-tuning.

Figures

Figures reproduced from arXiv: 2505.22487 by the authors.

Figure 1
Figure 1. Examples illustrating influence. Left: The x-axis is the time point τ of the input utterance x, and the y-axis is the calculated influence value s(t, τ ) for the timepoint t = 300 (red dot). Right: The relative influence S(σ) (normalized), where the x-axis is the time shift to the center frame. . the input to f. Removing single frames is not very meaningful as, in speech, a lot can be inferred from neighboring frame… view at source ↗
Figure 2
Figure 2. (a) Relative influence in the final layers of supervised 6-layer Transformer models trained for different tasks. The y-axis is on a log scale and the x-axis is only shown between ±0.7s, although the relative influence values were computed with a window size of 5 seconds on both sides. Dots show the heights of the center peaks. (b) As in (a) but for different layers of HuBERT. (c) Contextualization of different model… view at source ↗
Figure 3
Figure 3. The change of output in terms of the ℓ2 distance (left) and the phone error rates (right), as we vary the window size of input to HuBERT (different coloured lines). 3.2. Results with the Jacobian approach We follow the same setting as the truncation approach. However, instead of immediately comparing results on HuBERT layers, we first analyze a set of supervised Transformers for predicting f0, phones, and words. We … view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Relation between contextualization and probing perfor￾mance on phones (left) and words (right). Each point represents a layer of a model. The last layer of the supervised 6-layer Transformer is annotated. as all values are based on the same window size (as here).2 [PI…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 36 canonical work pages

  1. [1]

    Various speech tasks benefit from contextualized speech embeddings learned with self-supervision [1–5]

    Introduction The recent success of speech and language processing systems can be largely attributed to better modeling of context. Various speech tasks benefit from contextualized speech embeddings learned with self-supervision [1–5]. Automatic speech recogni- tion, summarization, question answering, and language modeling have all been shown to improve wh...

    work page

  2. [2]

    Effective Context in Neural Speech Models

    Measuring Effective Context We define effective context based on the following principle. For an input utterance of T frames x1, . . . , xT , we are interested in how a function f changes as we change the frames. At a high level, if changes made to frame xt do not change the output of f by much, then xt is not part of the effective context of f. The funct...

    work page Pith review arXiv 2025

  3. [3]

    In this section, we select a few models to analyze and showcase the utility of our approach

    Experiments Since there isn’t any prior work on measuring effective context in speech models, our goal is to provide a foundation for future work to measure the effective context for a broader range of models and architectures. In this section, we select a few models to analyze and showcase the utility of our approach. 3.1. Pilot experiments with truncati...

    work page

  4. [4]

    This would give us a streaming representation extractor

    Simulating a Streaming HuBERT Given that the effective context of pretrained Transformers is not long, we should be able to truncate their context and run them in a low-latency streaming mode without much performance loss. This would give us a streaming representation extractor. In contrast to other streaming models [32 –35], this approach does not requir...

    work page

  5. [5]

    Discussion and Conclusion In this work, we designed two complementary approaches to measuring effective context from first principles. The truncation approach is a direct modification to the input and can be easily verified by task performance difference, but how and how much we can truncate requires empirical verification. The Jacobian approach is less c...

    work page

  6. [6]

    An unsuper- vised autoregressive model for speech representation learning,

    Y .-A. Chung, W.-N. Hsu, H. Tang, and J. Glass, “An unsuper- vised autoregressive model for speech representation learning,” in INTERSPEECH, 2019

    work page 2019

  7. [7]

    Representation learning with contrastive predictive coding,

    A. v. d. Oord, Y . Li, and O. Vinyals, “Representation learning with contrastive predictive coding,”arXiv preprint arXiv:1807.03748, 2018

    arXiv 2018

  8. [8]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in NeurIPS, 2020

    work page 2020

Show all 42 references
  1. [9]

    Hubert: Self-supervised speech representa- tion learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representa- tion learning by masked prediction of hidden units,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021

    2021

  2. [10]

    Wavlm: Large-scale self-supervised pre-training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiaoet al., “Wavlm: Large-scale self-supervised pre-training for full stack speech processing,”IEEE Journal of Selected Topics in Signal Processing, 2022

    2022

  3. [11]

    Transformer- Based Long-Context End-to-End Speech Recognition,

    T. Hori, N. Moritz, C. Hori, and J. L. Roux, “Transformer- Based Long-Context End-to-End Speech Recognition,” inINTER- SPEECH, 2020

    2020

  4. [12]

    Large context end-to-end automatic speech recognition via extension of hierarchical recurrent encoder-decoder models,

    R. Masumura, T. Tanaka, T. Moriya, Y . Shinohara, T. Oba, and Y . Aono, “Large context end-to-end automatic speech recognition via extension of hierarchical recurrent encoder-decoder models,” inICASSP, 2019

    2019

  5. [13]

    How much context does my attention- based asr system need?

    R. Flynn and A. Ragni, “How much context does my attention- based asr system need?” inINTERSPEECH, 2024

    2024

  6. [14]

    Transformer-xl: Attentive language models beyond a fixed- length context,

    Z. Dai, Z. Yang, Y . Yang, J. Carbonell, Q. V . Le, and R. Salakhut- dinov, “Transformer-xl: Attentive language models beyond a fixed- length context,” inACL, 2019

    2019

  7. [15]

    Leave no context behind: Efficient infinite context transformers with infini-attention,

    T. Munkhdalai, M. Faruqui, and S. Gopal, “Leave no context behind: Efficient infinite context transformers with infini-attention,” arXiv preprint arXiv:2404.07143, 2024

    2024 arXiv

  8. [16]

    Landmark attention: Random- access infinite context length for transformers,

    A. Mohtashami and M. Jaggi, “Landmark attention: Random- access infinite context length for transformers,”Advances in NeurIPS, 2023

    2023

  9. [17]

    On training recurrent networks with trun- cated backpropagation through time in speech recognition,

    H. Tang and J. Glass, “On training recurrent networks with trun- cated backpropagation through time in speech recognition,” inSLT, 2018

    2018

  10. [18]

    On the usefulness of self-attention for automatic speech recognition with transformers,

    S. Zhang, E. Loweimi, P. Bell, and S. Renals, “On the usefulness of self-attention for automatic speech recognition with transformers,” inSLT, 2021

    2021

  11. [19]

    Branchformer: Parallel mlp-attention architectures to capture local and global context for speech recognition and understanding,

    Y . Peng, S. Dalmia, I. Lane, and S. Watanabe, “Branchformer: Parallel mlp-attention architectures to capture local and global context for speech recognition and understanding,” inICML, 2022

    2022

  12. [20]

    Sum- former: A linear-complexity alternative to self-attention for speech recognition,

    T. Parcollet, R. van Dalen, S. Zhang, and S. Bhattacharya, “Sum- former: A linear-complexity alternative to self-attention for speech recognition,” inINTERSPEECH, 2024

    2024

  13. [21]

    Understanding the role of self attention for efficient speech recognition,

    K. Shim, J. Choi, and W. Sung, “Understanding the role of self attention for efficient speech recognition,” inICLR, 2022

    2022

  14. [22]

    Layer-wise analysis of a self-supervised speech representation model,

    A. Pasad, J.-C. Chou, and K. Livescu, “Layer-wise analysis of a self-supervised speech representation model,” inASRU, 2021

    2021

  15. [23]

    Comparative layer-wise analysis of self-supervised speech models,

    A. Pasad, B. Shi, and K. Livescu, “Comparative layer-wise analysis of self-supervised speech models,” inICASSP, 2023

    2023

  16. [24]

    What do self- supervised speech models know about words?

    A. Pasad, C.-M. Chien, S. Settle, and K. Livescu, “What do self- supervised speech models know about words?”TACL, 2023

    2023

  17. [25]

    Probing phoneme, language and speaker information in unsupervised speech representations,

    M. de Seyssel, M. Lavechin, Y . Adi, E. Dupoux, and G. Wis- niewski, “Probing phoneme, language and speaker information in unsupervised speech representations,” inINTERSPEECH, 2022

    2022

  18. [26]

    Self-supervised speech representations are more phonetic than semantic,

    K. Choi, A. Pasad, T. Nakamura, S. Fukayama, K. Livescu, and S. Watanabe, “Self-supervised speech representations are more phonetic than semantic,” inINTERSPEECH, 2024

    2024

  19. [27]

    Sharp nearby, fuzzy far away: How neural language models use context,

    U. Khandelwal, H. He, P. Qi, and D. Jurafsky, “Sharp nearby, fuzzy far away: How neural language models use context,” inACL, 2018

    2018

  20. [28]

    What context features can trans- former language models use?

    J. O’Connor and J. Andreas, “What context features can trans- former language models use?” inACL, 2021

    2021

  21. [29]

    Same task, more tokens: the impact of input length on the reasoning performance of large language models,

    M. Levy, A. Jacoby, and Y . Goldberg, “Same task, more tokens: the impact of input length on the reasoning performance of large language models,”ACL, 2024

    2024

  22. [30]

    Deep inside con- volutional networks: Visualising image classification models and saliency maps,

    K. Simonyan, A. Vedaldi, and A. Zisserman, “Deep inside con- volutional networks: Visualising image classification models and saliency maps,” inICLR, 2014

    2014

  23. [31]

    ” why should i trust you?

    M. T. Ribeiro, S. Singh, and C. Guestrin, “” why should i trust you?” explaining the predictions of any classifier,” inProceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 2016, pp. 1135–1144

    2016

  24. [32]

    A unified approach to interpreting model predic- tions,

    S. Lundberg, “A unified approach to interpreting model predic- tions,”NeurIPS, 2017

    2017

  25. [33]

    Autore- gressive predictive coding: A comprehensive study,

    G.-P. Yang, S.-L. Yeh, Y .-A. Chung, J. Glass, and H. Tang, “Autore- gressive predictive coding: A comprehensive study,”IEEE Journal of Selected Topics in Signal Processing, 2022

    2022

  26. [34]

    Pyin: A fundamental frequency estima- tor using probabilistic threshold distributions,

    M. Mauch and S. Dixon, “Pyin: A fundamental frequency estima- tor using probabilistic threshold distributions,” inICASSP, 2014

    2014

  27. [35]

    fairseq: A fast, extensible toolkit for sequence modeling,

    M. Ott, S. Edunov, A. Baevski, A. Fan, S. Gross, N. Ng, D. Grang- ier, and M. Auli, “fairseq: A fast, extensible toolkit for sequence modeling,” inProceedings of NAACL-HLT 2019: Demonstrations, 2019

    2019

  28. [36]

    Orthogo- nality and isotropy of speaker and phonetic information in self- supervised speech representations,

    M. Mohamed, O. D. Liu, H. Tang, and S. Goldwater, “Orthogo- nality and isotropy of speaker and phonetic information in self- supervised speech representations,” inINTERSPEECH, 2024

    2024

  29. [37]

    Dual-mode asr: Unify and improve streaming asr with full-context modeling,

    J. Yu, W. Han, A. Gulati, C.-C. Chiu, B. Li, T. N. Sainath, Y . Wu, and R. Pang, “Dual-mode asr: Unify and improve streaming asr with full-context modeling,” inICLR, 2020

    2020

  30. [38]

    Efficient cascaded streaming asr system via frame rate reduction,

    X. Cai, D. Qiu, S. Ding, D. Hwang, W. W. A. Bruguier, R. Prab- havalkar, T. Sainath, and Y . He, “Efficient cascaded streaming asr system via frame rate reduction,” inASRU, 2023

    2023

  31. [39]

    Improving stream- ing automatic speech recognition with non-streaming model distil- lation on unsupervised data,

    T. Doutre, W. Han, M. Ma, Z. Lu, C.-C. Chiu, R. Pang, A. Narayanan, A. Misra, Y . Zhang, and L. Cao, “Improving stream- ing automatic speech recognition with non-streaming model distil- lation on unsupervised data,” inICASSP, 2021

    2021

  32. [40]

    Xlsr-transducer: Streaming asr for self-supervised pretrained models,

    S. Kumar, S. Madikeri, J. Zuluaga-Gomez, E. Villatoro-Tello, I. Thorbecke, P. Motlicek, A. Ganapathirajuet al., “Xlsr-transducer: Streaming asr for self-supervised pretrained models,”arXiv preprint arXiv:2407.04439, 2024

    2024 arXiv

  33. [41]

    Ecapa-tdnn: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification,

    B. Desplanques, J. Thienpondt, and K. Demuynck, “Ecapa-tdnn: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification,”INTERSPEECH, 2020

    2020

  34. [42]

    Robust speech recognition via large-scale weak su- pervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak su- pervision,” inICML, 2023

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.