pith. sign in

arxiv: 2606.09357 · v1 · pith:GJ5SZB55new · submitted 2026-06-08 · 📡 eess.AS

Rethinking Depth: A study of the Recursive-Transformer for Speech Recognition

Thomas Rolland , Carlos Carvalho , Alberto Abad This is my paper

Pith reviewed 2026-06-27 14:59 UTC · model grok-4.3

classification 📡 eess.AS
keywords Recursive-TransformerAutomatic Speech RecognitionASRTransformer architectureLayer recursionModel efficiencySpeech encoder
0
0 comments X

The pith

The Recursive-Transformer matches standard Transformer performance in speech recognition while using 66% fewer parameters when recursion is applied in the latent space.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper investigates using the same layers repeatedly in Transformer encoders for automatic speech recognition instead of adding new layers each time. This approach, called the Recursive-Transformer, is tested by varying how deep the recursion goes and where the shared layers are placed. Results show it can deliver similar accuracy to larger models, especially with limited recursion loops in the latent space. A reader would care because it suggests a way to build effective ASR systems with much smaller memory and compute footprints.

Core claim

By repeatedly applying the same transformer layers within the encoder, the Recursive-Transformer achieves comparable word error rates to conventional stacked-layer Transformers on ASR tasks while reducing the total parameter count by 66 percent, with the best results obtained when recurrence operates in the latent space using a restricted number of loops.

What carries the argument

The Recursive-Transformer, which reuses the same set of layers multiple times according to a chosen recursion depth rather than stacking distinct layers.

If this is right

  • Recurrence applied in the latent space with few loops yields the strongest performance-parameter trade-off.
  • Overall parameter count drops by 66% while maintaining comparable recognition accuracy.
  • Systematic variation of recursion depth and layer allocation reveals optimal configurations for ASR encoders.
  • The technique serves as a practical alternative for resource-constrained ASR deployments.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar recursion might reduce model size in other sequence modeling domains such as language or translation.
  • Effective model depth could increase without adding parameters if loops are tuned carefully.
  • Future work could test whether the approach scales to much larger base models or different speech datasets.

Load-bearing premise

The comparisons assume that baseline models and recursive variants were trained and evaluated under equivalent conditions without selective reporting of favorable recursion settings.

What would settle it

An independent run on the same speech datasets where the recursive model shows higher error rates or requires more parameters than the non-recursive baseline at matched sizes.

Figures

Figures reproduced from arXiv: 2606.09357 by Alberto Abad, Carlos Carvalho, Thomas Rolland.

Figure 1
Figure 1. Figure 1: Cosine similarity matrix of layer-wise outputs from the Whisper-medium encoder, computed using identical input across all layers. Each element (i, j) in the matrix represents the cosine similarity between the outputs of layers i and j. 2. Related work The Universal Transformer [18] was the first work to intro￾duce recurrence in Transformer-based architectures [5] by it￾eratively applying a single layer to … view at source ↗
Figure 2
Figure 2. Figure 2: Diagram of the Latent-Recursive-Transformer architecture. Each block is composed of several sub-layers. The Prelude block (P) encodes the inputs into a latent representation. The Recurrent block (R), shared across multiple steps, iteratively refines this latent representation. Finally, the Coda block (C) decodes the latent state to produce the output. 4.2. Implementation details All experiments were conduc… view at source ↗
Figure 3
Figure 3. Figure 3: WER change relative to the baseline model for ut￾terance duration, transcript length (word-count), and baseline difficulty (per-utterance baseline WER bins) for test-other. Bars show the mean improvement ∆WER = WERL11−WERL1loop for L1loop variants. Positive values indicate lower WER than baseline. The shaded region highlights negative improvement (degradation). 5.3. Influence of the number of recurrence lo… view at source ↗
read the original abstract

Transformer-based architectures have led to significant improvements in Automatic Speech Recognition (ASR), often at the cost of substantially increased model sizes. A promising approach to address this issue is layer sharing through depth recursion, commonly referred to as the Recursive-Transformer, which involves repeatedly applying the same layers within the model. Despite its potential shown in other fields, this technique remains relatively unexplored in ASR. In this paper, we present an experimental study of the Recursive-Transformer applied to ASR encoder architectures. We systematically investigate the impact of recursion depth and layer allocation within the Recursive-based Transformer. Our results demonstrate that the Recursive-Transformer is a viable alternative, especially when recurrence is applied in the latent space with a restricted number of loops, obtaining comparable performance while reducing the parameter count by 66%.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The manuscript presents an experimental study of the Recursive-Transformer applied to ASR encoder architectures. It systematically investigates the impact of recursion depth and layer allocation, claiming that the Recursive-Transformer is a viable alternative—particularly when recurrence is applied in the latent space with a restricted number of loops—yielding comparable performance while reducing the parameter count by 66%.

Significance. If the empirical results hold under fair and fully reported conditions, the demonstration of substantial parameter reduction via latent-space recursion would be a useful contribution to efficient Transformer design for ASR, addressing the common tradeoff between model size and accuracy.

major comments (1)
  1. [Abstract] Abstract: the headline claim of comparable ASR performance with a 66% parameter reduction is stated without any information on datasets, baselines, error bars, training procedures, or statistical tests. This omission is load-bearing for the central empirical claim because the abstract supplies no verifiable evidence that the data actually support the stated result.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their review and constructive comment. We address the major comment below.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the headline claim of comparable ASR performance with a 66% parameter reduction is stated without any information on datasets, baselines, error bars, training procedures, or statistical tests. This omission is load-bearing for the central empirical claim because the abstract supplies no verifiable evidence that the data actually support the stated result.

    Authors: We agree that the abstract is concise and does not include supporting details on datasets, baselines, error bars, training procedures, or statistical tests. The full manuscript reports these elements in the experimental setup and results sections. To address the concern, we will revise the abstract to briefly reference the primary dataset, the key baseline, and the restricted recursion setting while preserving length constraints. Error bars and statistical details remain in the body of the paper as they are too detailed for the abstract. revision: yes

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper is an empirical experimental study comparing Recursive-Transformer variants to baselines in ASR, reporting performance and parameter reductions. No derivation chain, equations, fitted parameters, or mathematical claims exist that could reduce to inputs by construction. No self-citations are invoked as load-bearing uniqueness theorems or ansatzes. The central results rest on experimental outcomes, which are externally falsifiable and not internally circular.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available; no explicit free parameters, axioms, or invented entities are described.

pith-pipeline@v0.9.1-grok · 5657 in / 971 out tokens · 21607 ms · 2026-06-27T14:59:05.489997+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

53 extracted references · 16 canonical work pages · 7 internal anchors

  1. [1]

    Rethinking Depth: A study of the Recursive-Transformer for Speech Recognition

    Introduction Recent Automatic Speech Recognition (ASR) advances are driven by scaling model size and datasets, with state-of-the-art systems now containing configurations that exceed one billion parameters and can be trained on millions of hours of speech data [1, 2, 3]. As both model size and training data have contin- ued to scale, performance improveme...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  2. [2]

    initial,

    Related work The Universal Transformer [18] was the first work to intro- duce recurrence in Transformer-based architectures [5] by it- eratively applying a single layer to refine representations, ef- fectively decoupling computational depth from the number of parameters. While recurrence is an established concept, it has seen a resurgence in recent litera...

  3. [3]

    We argue that such choices should be guided by a deeper understanding of the model’s in- ternal dynamics

    Latent-Recursive-Transformer for ASR This work is motivated by the observation that prior research in Recursive-Transformers for ASR selects the number of loops and the shared layers arbitrarily. We argue that such choices should be guided by a deeper understanding of the model’s in- ternal dynamics. To this end, as our work focuses on the ASR encoder, we...

  4. [4]

    Corpora Our experiments primarily use the LibriSpeech corpus [40], a widely adopted ASR benchmark comprising roughly 1,000 hours of English audiobook speech from LibriV ox [41]

    Experimental setting 4.1. Corpora Our experiments primarily use the LibriSpeech corpus [40], a widely adopted ASR benchmark comprising roughly 1,000 hours of English audiobook speech from LibriV ox [41]. We follow the standard partitions with 960 hours for training and two test set, test-clean and test-other, each containing 5 hours. To evaluate the gener...

  5. [5]

    Recursive-Transformer configurations We evaluate the influence of different Recursive-Transformer configurations, parameterised by the tuple{n r, L, np, nc}

    Results 5.1. Recursive-Transformer configurations We evaluate the influence of different Recursive-Transformer configurations, parameterised by the tuple{n r, L, np, nc}. To ensure fair comparison, all configurations process the same number of layers, such thatn p +L×n r +n c = 24, unless stated otherwise. The baseline configuration (B) consists of 24 uni...

  6. [6]

    Our experi- ments highlight the critical role of recurrence depth

    Conclusion and future work In this work, we investigated Recursive- and Latent-Recursive- Transformer encoders for ASR, showing that they preserve per- formance with substantially fewer parameters and can outper- form standard Transformers under parameter-matched settings, particularly for the Latent-Recursive architecture. Our experi- ments highlight the...

  7. [7]

    Acknowledgements This work was funded/supported by Portuguese national funds through Fundac ¸˜ao para a Ciˆencia e a Tecnologia, I.P. (FCT) un- der projects UID/50021/2025 and UID/PRR/50021/2025, and by the Portuguese Recovery and Resilience Plan and NextGen- erationEU European Union funds under project C644865762- 00000008 (Accelerat.AI)

    2025

  8. [8]

    Gra- nary: Speech recognition and translation dataset in 25 european languages, 2025

    N. R. Koluguri, M. Sekoyan, G. Zelenfroynd, S. Meister, S. Ding, S. Kostandian, H. Huang, N. Karpov, J. Balam, V . Lavrukhin et al., “Granary: Speech recognition and translation dataset in 25 european languages,”arXiv:2505.13404, 2025

    work page arXiv 2025

  9. [9]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” inInternational conference on machine learning. PMLR, 2023, pp. 28 492–28 518

    2023

  10. [10]

    Owls: Scaling laws for multilingual speech recognition and translation models,

    W. Chen, J. Tian, Y . Peng, B. Yan, C.-H. H. Yang, and S. Watan- abe, “Owls: Scaling laws for multilingual speech recognition and translation models,”arXiv:2502.10373, 2025

    work page arXiv 2025

  11. [11]

    Scaling Laws for Neural Language Models

    J. Kaplan, S. McCandlish, T. J. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,”arXiv:2001.08361, vol. abs/2001.08361, 2020. [Online]. Available: https://api. semanticscholar.org/CorpusID:210861095

    work page internal anchor Pith review Pith/arXiv arXiv 2001

  12. [12]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017

    2017

  13. [13]

    Performance and efficiency evaluation of asr inference on the edge,

    S. Gondi and V . Pratap, “Performance and efficiency evaluation of asr inference on the edge,”Sustainability, vol. 13, no. 22, p. 12392, 2021

    2021

  14. [14]

    Ai and memory wall,

    A. Gholami, Z. Yao, S. Kim, C. Hooper, M. W. Mahoney, and K. Keutzer, “Ai and memory wall,”IEEE Micro, vol. 44, no. 3, pp. 33–39, 2024

    2024

  15. [15]

    Impact of joint heat and memory constraints of mobile device in edge-assisted on-device artificial intelligence,

    P. Choi, J. Kim, and J. Kwak, “Impact of joint heat and memory constraints of mobile device in edge-assisted on-device artificial intelligence,” inProceedings of the 2nd International Workshop on Networked AI Systems, 2024, pp. 31–36

    2024

  16. [16]

    A survey of model compression and acceleration for deep neural networks,

    Y . Cheng, D. Wang, P. Zhou, and T. Zhang, “A survey of model compression and acceleration for deep neural networks,” arXiv:1710.09282, 2017

    work page arXiv 2017

  17. [17]

    Improving End-to-end Mixed-case ASR with Knowledge Distillation and Integration of V oice Activity Cues ,

    S. Novitasari, T. Fukuda, and G. Kurata, “Improving End-to-end Mixed-case ASR with Knowledge Distillation and Integration of V oice Activity Cues ,” inInterspeech 2025, 2025, pp. 3334–3338

    2025

  18. [18]

    Context- Driven Dynamic Pruning for Large Speech Foundation Models,

    M. Someki, S. Bharadwaj, A. A. Joshi, C.-J. Lin, J. Tian, J. weon Jung, M. M ¨uller, N. Susanj, J. Liu, and S. Watanabe, “Context- Driven Dynamic Pruning for Large Speech Foundation Models,” inInterspeech 2025, 2025, pp. 1993–1997

    2025

  19. [19]

    Towards One-bit ASR: Extremely Low-bit Conformer Quantization Using Co-training and Stochastic Preci- sion,

    Z. Li, H. Xu, Z. Jin, L. Meng, T. Wang, H. Wang, Y . Chen, M. Cui, S. Hu, and X. Liu, “Towards One-bit ASR: Extremely Low-bit Conformer Quantization Using Co-training and Stochastic Preci- sion,” inInterspeech 2025, 2025, pp. 1973–1977

    2025

  20. [20]

    Llm in a flash: Efficient large language model inference with limited mem- ory,

    K. Alizadeh, S. I. Mirzadeh, D. Belenko, S. Khatamifard, M. Cho, C. C. Del Mundo, M. Rastegari, and M. Farajtabar, “Llm in a flash: Efficient large language model inference with limited mem- ory,” inProceedings of the 62nd Annual Meeting of the Associ- ation for Computational Linguistics (Volume 1: Long Papers), 2024, pp. 12 562–12 584

    2024

  21. [21]

    Efficient neu- ral architecture search via parameters sharing,

    H. Pham, M. Guan, B. Zoph, Q. Le, and J. Dean, “Efficient neu- ral architecture search via parameters sharing,” inInternational conference on machine learning. PMLR, 2018, pp. 4095–4104

    2018

  22. [22]

    ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

    Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Sori- cut, “Albert: A lite bert for self-supervised learning of language representations,”arXiv:1909.11942, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1909

  23. [23]

    One wide feedforward is all you need,

    T. Pires, A. Vilarinho Lopes, Y . Assogba, and H. Setiawan, “One wide feedforward is all you need,” inProceedings of the Eighth Conference on Machine Translation, P. Koehn, B. Haddow, T. Kocmi, and C. Monz, Eds. Singapore: Association for Computational Linguistics, Dec. 2023, pp. 1031–1044. [Online]. Available: https://aclanthology.org/2023.wmt-1.98/

    2023

  24. [24]

    Shared-Adapters: A Novel Transformer-based Parameter Efficient Transfer Learning Ap- proach For Children’s Automatic Speech Recognition,

    T. Rolland and A. Abad, “Shared-Adapters: A Novel Transformer-based Parameter Efficient Transfer Learning Ap- proach For Children’s Automatic Speech Recognition,” inInter- speech 2024, 2024, pp. 2370–2374

    2024

  25. [25]

    Universal transformers,

    M. Dehghani, S. Gouws, O. Vinyals, J. Uszkoreit, and L. Kaiser, “Universal transformers,” inICLR 2019, 2019. [Online]. Available: https://openreview.net/pdf?id=HyzdRiR9Y7

    2019

  26. [26]

    Sliced recursive transformer,

    Z. Shen, Z. Liu, and E. Xing, “Sliced recursive transformer,” in Computer Vision – ECCV 2022, S. Avidan, G. Brostow, M. Ciss´e, G. M. Farinella, and T. Hassner, Eds. Cham: Springer Nature Switzerland, 2022, pp. 727–744

    2022

  27. [27]

    Audio albert: A lite bert for self-supervised learning of audio representation,

    P.-H. Chi, P.-H. Chung, T.-H. Wu, C.-C. Hsieh, Y .-H. Chen, S.- W. Li, and H.-y. Lee, “Audio albert: A lite bert for self-supervised learning of audio representation,” in2021 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2021, pp. 344–350

    2021

  28. [28]

    Sim-t: Simplify the transformer network by multiplexing technique for speech recognition,

    G. Wei, Z. Duan, S. Li, G. Yang, X. Yu, and J. Li, “Sim-t: Simplify the transformer network by multiplexing technique for speech recognition,”arXiv:2304.04991, 2023

    work page arXiv 2023

  29. [29]

    Residualtransformer: Residual low-rank learning with weight-sharing for transformer layers,

    Y . Wang and J. Li, “Residualtransformer: Residual low-rank learning with weight-sharing for transformer layers,” ICASSP. IEEE, April 2024

    2024

  30. [30]

    SpiralFormer: Looped Transformers Can Learn Hierarchical Dependencies via Multi-Resolution Recursion

    C. Yu, X. Shu, Y . Wang, Y . Zhang, H. Wu, Y . Wu, R. Long, Z. Chen, Y . Xu, W. Suet al., “Spiralformer: Looped transform- ers can learn hierarchical dependencies via multi-resolution recur- sion,”arXiv preprint arXiv:2602.11698, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  31. [31]

    Relaxed recursive transformers: Effective parameter sharing with layer-wise lora,

    S. Bae, A. Fisch, H. Harutyunyan, Z. Ji, S. Kim, and T. Schus- ter, “Relaxed recursive transformers: Effective parameter sharing with layer-wise lora,” inInternational Conference on Learning Representations, 2025

    2025

  32. [32]

    Looping back to move forward: Recursive trans- formers for efficient and flexible large multimodal models,

    R. Xu, Y . Gao, L. Wang, J. Li, W. Chen, Q. Guo, M. Yang, and S. Zhang, “Looping back to move forward: Recursive trans- formers for efficient and flexible large multimodal models,”arXiv preprint arXiv:2602.09080, 2026

    work page arXiv 2026

  33. [33]

    Looped trans- formers for length generalization,

    Y . Fan, Y . Du, K. Ramchandran, and K. Lee, “Looped trans- formers for length generalization,” inICLR 2025, 2025. [Online]. Available: https://openreview.net/forum?id=2edigk8yoU

    2025

  34. [34]

    Go wider instead of deeper,

    F. Xue, Z. Shi, F. Wei, Y . Lou, Y . Liu, and Y . You, “Go wider instead of deeper,” inProceedings of the AAAI Conference on Ar- tificial Intelligence, vol. 36, no. 8, 2022, pp. 8779–8787

    2022

  35. [35]

    Leveraging adapter for parameter- efficient asr encoder,

    K. Shim, J. Lee, and H. Kim, “Leveraging adapter for parameter- efficient asr encoder,” inInterspeech 2024, 2024, pp. 2380–2384

    2024

  36. [36]

    Beyond universal trans- former: block reusing with adaptor in transformer for automatic speech recognition,

    H. Tang, Z. Liu, C. Zeng, and X. Li, “Beyond universal trans- former: block reusing with adaptor in transformer for automatic speech recognition,” inInternational Symposium on Neural Net- works. Springer, 2024, pp. 69–79

    2024

  37. [37]

    Exploring shared-weight mechanisms in transformer and conformer architectures for automatic speech recognition,

    T. Rolland and A. Abad, “Exploring shared-weight mechanisms in transformer and conformer architectures for automatic speech recognition,” inProc. Interspeech 2025, 2025, pp. 2885–2889

    2025

  38. [38]

    Scaling up test-time compute with latent reasoning: A recurrent depth approach,

    J. Geiping, S. M. McLeish, N. Jain, J. Kirchenbauer, S. Singh, B. R. Bartoldson, B. Kailkhura, A. Bhatele, and T. Goldstein, “Scaling up test-time compute with latent reasoning: A recurrent depth approach,” inES-FoMo III: 3rd Workshop on Efficient Systems for Foundation Models, 2025. [Online]. Available: https://openreview.net/forum?id=D6o6Bwtq7h

    2025

  39. [39]

    Scaling Latent Reasoning via Looped Language Models

    R.-J. Zhu, Z. Wang, K. Hua, T. Zhang, Z. Li, H. Que, B. Wei, Z. Wen, F. Yin, H. Xinget al., “Scaling latent reasoning via looped language models,”arXiv preprint arXiv:2510.25741, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  40. [40]

    Reasoning with latent thoughts: On the power of looped transformers,

    N. Saunshi, N. Dikkala, Z. Li, S. Kumar, and S. J. Reddi, “Reasoning with latent thoughts: On the power of looped transformers,” inInternational Conference on Learning Rep- resentations, Y . Yue, A. Garg, N. Peng, F. Sha, and R. Yu, Eds., vol. 2025, 2025, pp. 14 855–14 881. [Online]. Available: https://proceedings.iclr.cc/paper files/paper/2025/file/ 2676...

    2025

  41. [41]

    Transformer layers as painters,

    Q. Sun, M. Pickett, A. K. Nain, and L. Jones, “Transformer layers as painters,”arXiv:2407.09298, 2024

    work page arXiv 2024

  42. [42]

    Subformer: Explor- ing weight sharing for parameter efficiency in generative trans- formers,

    M. Reid, E. Marrese-Taylor, and Y . Matsuo, “Subformer: Explor- ing weight sharing for parameter efficiency in generative trans- formers,”arXiv:2101.00234, 2021

    work page arXiv 2021

  43. [43]

    Adaptive Computation Time for Recurrent Neural Networks

    A. Graves, “Adaptive computation time for recurrent neural net- works,”arXiv preprint arXiv:1603.08983, 2016

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  44. [44]

    Sangmin Bae, Yujin Kim, Reza Bayat, Sungnyun Kim, Jiyoun Ha, Tal Schuster, Adam Fisch, Hrayr Harutyunyan, Ziwei Ji, Aaron Courville, and Se-Young Yun

    S. Bae, Y . Kim, R. Bayat, S. Kim, J. Ha, T. Schuster, A. Fisch, H. Harutyunyan, Z. Ji, A. Courvilleet al., “Mixture-of-recursions: Learning dynamic recursive depths for adaptive token-level com- putation,”arXiv preprint arXiv:2507.10524, 2025

    work page arXiv 2025

  45. [45]

    MeSH: Memory-as-State-Highways for Recursive Transformers

    C. Yu, X. Shu, Y . Wang, Y . Zhang, H. Wu, J. Li, R. Long, Z. Chen, Y . Xu, W. Suet al., “Mesh: Memory-as-state-highways for recur- sive transformers,”arXiv preprint arXiv:2510.07739, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  46. [46]

    Reasoning with latent thoughts: On the power of looped transformers,

    N. Saunshi, N. Dikkala, Z. Li, S. Kumar, and S. J. Reddi, “Reasoning with latent thoughts: On the power of looped transformers,” inThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025. [Online]. Available: https://openreview.net/forum?id=din0lGfZFd

    2025

  47. [47]

    Lib- rispeech: an asr corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- rispeech: an asr corpus based on public domain audio books,” inICASSP. IEEE, 2015, pp. 5206–5210

    2015

  48. [48]

    Librivox: Free public domain audiobooks,

    J. Kearns, “Librivox: Free public domain audiobooks,” 2014

    2014

  49. [49]

    Aishell-1: An open- source mandarin speech corpus and a speech recognition base- line,

    H. Bu, J. Du, X. Na, B. Wu, and H. Zheng, “Aishell-1: An open- source mandarin speech corpus and a speech recognition base- line,” in2017 20th conference of the oriental chapter of the inter- national coordinating committee on speech databases and speech I/O systems and assessment (O-COCOSDA). IEEE, 2017, pp. 1–5

    2017

  50. [50]

    Open-source conversational ai with speechbrain 1.0,

    M. Ravanelli, T. Parcollet, A. Moumen, S. de Langen, C. Subakan, P. Plantinga, Y . Wang, P. Mousavi, L. D. Libera, A. Ploujnikov, F. Paissan, D. Borra, S. Zaiem, Z. Zhao, S. Zhang, G. Karakasidis, S.-L. Yeh, P. Champion, A. Rouhe, R. Braun, F. Mai, J. Zuluaga- Gomez, S. M. Mousavi, A. Nautsch, H. Nguyen, X. Liu, S. Sagar, J. Duret, S. Mdhaffar, G. Laperri...

    2024

  51. [51]

    Connectionist temporal classification,

    A. Graves, “Connectionist temporal classification,” inSupervised sequence labelling with recurrent neural networks. Springer, 2012, pp. 61–93

    2012

  52. [52]

    Do language models use their depth efficiently?arXiv preprint arXiv:2505.13898, 2025

    R. Csord ´as, C. D. Manning, and C. Potts, “Do language models use their depth efficiently?”arXiv:2505.13898, 2025

    work page arXiv 2025

  53. [53]

    Branchformer: Parallel mlp-attention architectures to capture local and global context for speech recognition and understanding,

    Y . Peng, S. Dalmia, I. Lane, and S. Watanabe, “Branchformer: Parallel mlp-attention architectures to capture local and global context for speech recognition and understanding,” inInterna- tional Conference on Machine Learning. PMLR, 2022, pp. 17 627–17 643

    2022