Pith. sign in

REVIEW 4 major objections 5 minor 29 references

This paper claims that small discrete speech vocabularies (k ≤ 1,000) outperform larger ones in speech language model pre-training, with WavLM codebooks working best and larger base models adding robustness.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Smaller discrete vocabularies (k = 125 to 1,000), WavLM units, and larger models give the lowest negative log-likelihood in speech language model pre-training.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A useful empirical sweep of discrete unit choices for speech LMs, but the headline 'small k is best' rests on raw NLL comparisons across vocabulary sizes that are confounded by codebook entropy. the 4 major comments →

arxiv 2509.05359 v1 pith:4X5HR6PP submitted 2025-09-03 cs.CL cs.AIeess.AS

An Empirical Analysis of Discrete Unit Representations in Speech Language Modeling Pre-training

Yanis Labrak , Richard Dufour , Micka\"el Rouvier This is my paper
classification cs.CL cs.AIeess.AS
keywords speech language modelsdiscrete speech unitsself-supervised speech encodersk-means clusteringcontinual pre-trainingphoneme alignmentacoustic robustnessmodel scaling
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Speech language models are text LMs taught to read audio tokens, and this paper asks which audio-token recipe makes that adaptation work best. After training 51 models that extend pre-trained SmolLM text LMs to LibriSpeech audio, the authors find that codebook size is the dominant factor: k-means vocabularies of 1,000 or fewer units give the lowest negative log-likelihood, and larger vocabularies degrade sharply. WavLM-based units perform best overall, and the 1.7B base model is less sensitive to vocabulary size and acoustic perturbations than smaller models. The paper also shows that units align with phonemes without phonetic supervision and that matching the k-means corpus to the target domain matters more than training on noisy data. If correct, the practical recipe for speech LM pre-training is a small, domain-matched WavLM codebook plus a large base model.

Core claim

The paper's central claim is that discrete unit granularity dominates how well a language model learns speech during adaptation. Across four encoders and six codebook sizes (125 to 5,000 clusters), k ≤ 1,000 yields NLL about 1.8-2.4 on LibriSpeech test-clean; k ≥ 2,500 degrades sharply. WavLM gives the best units, and scaling the base model to 1.7B improves NLL and robustness to noise and pitch shifts. Cluster-usage and forced-phoneme alignment show units are used efficiently and specialize in small phoneme sets without phonetic labels. The k-means corpus matters: LibriHeavy units, matched to the LibriSpeech domain, stay stable under perturbations, while noisy corpora do not.

What carries the argument

The central object is the discrete speech unit codebook: frame-level representations at 50 Hz are taken from the final hidden layer of a frozen self-supervised encoder and quantized by k-means into k tokens, which are then added to the LLM tokenizer and trained with the standard next-token objective. Varying k changes how finely acoustic variation is discretized, and the paper uses three diagnostics on top of NLL—cluster usage entropy, phoneme-alignment matrices from forced alignment, and perturbation tests—to separate codebook efficiency from phonetic content and robustness.

Load-bearing premise

The claim that the hours used to build the speech codebooks do not overlap with the hours used to train the language model is asserted in Section 2.2 without evidence; if the two sets overlap, the LibriHeavy advantage in robustness could be a data-leakage artifact rather than a domain-matching effect.

What would settle it

Search the 2,000 LibriHeavy hours used for k-means for overlapping speakers or recordings with LibriSpeech train/test; if overlap exists, re-run the Table 3 perturbation comparison with a strictly disjoint clustering corpus. If the LibriHeavy NLL advantage disappears, the domain-matching conclusion is a leakage artifact.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Pre-training SLMs should prefer codebooks of at most 1,000 units; larger codebooks make the units harder to learn, especially for small models.
  • WavLM is a safer default encoder than HuBERT, XLS-R, or Wav2Vec 2 when the goal is semantic speech modeling at this stage.
  • Scaling the text backbone is a substitute for codebook tuning: the 1.7B model holds its NLL across k up to 1,000 and tolerates acoustic perturbations better.
  • Discrete units carry phoneme-level information by default, so downstream speech understanding may inherit phonetic structure without a separate ASR or alignment module.
  • Clustering data should be chosen to match the target domain; adding noisy or mismatched data to k-means training does not buy robustness and can hurt.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the paper evaluates only NLL, the optimal codebook size could move once downstream semantic benchmarks (sWUGGY, spoken QA, SLU) are scored; NLL favors stable token distributions, not necessarily paralinguistic richness.
  • The model-scale result suggests a scaling law between parameters and codebook size; fitting NLL as a function of k and model size across more capacity points would make the trade-off explicit and testable.
  • The phoneme-alignment result implies discrete units from a frozen encoder could be used as an unsupervised phonetic supervision signal, e.g., to bootstrap alignments or lexicons for low-resource languages.
  • The domain-matching result suggests codebook training should be viewed as a domain-adaptation step: train k-means on unlabeled target-domain audio instead of a large diverse corpus.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper reports a systematic empirical study of discrete speech-unit representations for continual pre-training of speech language models. The authors encode speech with four self-supervised encoders (WavLM, HuBERT, XLS-R, Wav2Vec2), discretize features with k-means codebooks of sizes k in {125, 250, 500, 1000, 2500, 5000} trained on different unlabeled corpora, and adapt SmolLM models (135M, 360M, 1.7B) via LoRA. All models are trained for 300 steps on LibriSpeech and evaluated by negative log-likelihood (NLL) on LibriSpeech test-clean. The headline findings are that smaller vocabularies (k <= 1000) perform best, WavLM units are the most promising, larger models are more robust to vocabulary size, and domain-matched k-means training data (LibriHeavy) improves robustness. The paper also analyzes cluster utilization and phoneme alignment of discrete units.

Significance. If the NLL-based comparisons were valid, the paper would provide a practically useful design rule for SLM pre-training: a small WavLM codebook combined with a large base model and domain-matched clustering data. The sweep is broad - 51 trained models, four encoders, six codebook sizes, three model scales, and four clustering corpora - and the authors state they will release tokenized datasets and clustering models, which is a concrete reproducibility contribution. The phoneme-alignment analysis is also a useful qualitative check on what the units encode. The paper is not circular: k-means codebooks are trained on separate corpora and NLL is measured on a held-out set. However, the central cross-k claim is threatened by the use of raw NLL across different vocabulary sizes, and several robustness conclusions rest on single runs with very small NLL differences.

major comments (4)
  1. [Section 2.4, Tables 1-2] The central claim that smaller k is 'consistently superior' is based on raw NLL, but NLL is not comparable across codebooks of different size. With a fixed frame rate, the per-token entropy of the speech token stream grows with k; for a uniform codebook the minimal NLL is log k, so the observed gap between k=125 (about 2.15) and k=5,000 (about 4.21) may largely reflect the number of classes, not 'excessive speech unit granularity' or 'noisier token distributions.' The sentence in Section 2.4 that frame rate is held constant does not address this confound. I recommend comparing NLL against a normalized baseline (e.g., NLL minus the empirical marginal entropy of each codebook, or bits per second) or, preferably, evaluating downstream semantic metrics such as sWUGGY at each k. Comparisons within a fixed k (encoder or scale effects) are much less affected.
  2. [Section 2.1, Section 3.3, Tables 1-3] Every number in the paper comes from a single training run with seed 42. Several key conclusions are based on very small NLL differences - for example, Table 3 shows LibriHeavy Noise-H at 2.692 vs. Noise-L at 2.678, and People's Speech Pitch Shift at 2.866 vs. Noise-L at 2.860. Without multiple seeds, confidence intervals, or paired significance tests, claims such as 'superior performance and stability' and 'People's Speech stands out' are not established. This is especially important because the paper's three headline patterns (small k, WavLM, larger models) are all rank-order claims over NLL values that differ by as little as 0.01-0.26 in the robustness section.
  3. [Section 2.2, Section 3.3] The claim that 'None of the data selected to build the k-means overlaps with the speech modeling dataset' is load-bearing for the LibriHeavy advantage, but no evidence is provided. LibriHeavy and LibriSpeech are both derived from LibriVox audiobooks, so the selected 2,000-hour subset may include the same recordings or speakers as LibriSpeech test-clean. If overlap exists, the lower NLL for LibriHeavy-trained codebooks in Table 3 and the domain-matching conclusion in Section 4 are confounded by leakage at the clustering level. Please report the exact overlap check (e.g., utterance or speaker IDs excluded) or use a clustering corpus with documented disjointness.
  4. [Section 2.4, Section 4] The paper uses NLL as the sole evaluation metric, but the introduction and conclusion make claims about 'semantic information' and 'semantic speech understanding.' The only support for the NLL-semantics link is a citation to prior work; the paper itself does not run sWUGGY or any downstream task. Thus the statement that 'smaller SLMs struggle to capture semantic information from higher discretization granularity units' goes beyond what NLL measures. A direct downstream evaluation, or at least an explicit limitation paragraph separating speech-modeling quality from semantic capability, is needed before the design recommendations can be accepted as stated.
minor comments (5)
  1. [Section 3.3] The naming 'Noise-H (SNR 15-20dB)' and 'Noise-L (SNR 5-10dB)' is confusing: higher SNR corresponds to less noise, so Noise-H should be less noisy if H refers to SNR, but the text calls it 'high-intensity Gaussian noise.' Please clarify the naming convention.
  2. [Table 1, Section 3.1] For k=5,000 at Step 100, Wav2Vec reaches NLL 8.179, so the text '7.8-8.1' should be '7.8-8.2.'
  3. [Section 3.3, Table 3] People's Speech is worse than LibriHeavy on every condition in Table 3 (e.g., clean 2.739 vs. 2.621); the phrase 'stands out' should be qualified to mean 'relative to GigaSpeech and CommonVoice,' not overall.
  4. [Figure 3] The caption says all matrices represent LibriSpeech test-clean, but subfigure (b) is labeled test-other. Please correct.
  5. [Section 3.4, Section 3.5] 'CommonV oice' appears with a spurious space throughout; check LaTeX rendering. Also, Section 3.5 says 'see Figure 3, but wasn't displayed due to a lack of space'; if the figure is not displayed, the reference should say 'not shown' and the claim 'similar pattern across all the granularities' should be supported by the included k=250 figures or explicitly deferred to supplementary material.

Circularity Check

0 steps flagged

No significant circularity: empirical comparisons are externally benchmarked and no self-citation chain is load-bearing.

full rationale

The paper's central claims are empirical rankings over NLL computed on held-out LibriSpeech test-clean. The k-means discretizers are trained on separate corpora (LibriHeavy, GigaSpeech, People's Speech, CommonVoice) and the SLMs are LoRA-finetuned on LibriSpeech with a fixed autoregressive NLL objective; no parameter is fitted to the NLL values used for comparison, and no evaluation result is fed back into the model or clustering configuration. The paper does not rely on self-citations: all cited prior work is external (encoder architectures, SpeechGPT pipeline, SmolLM, prior NLL-to-semantics correlations). Comparing NLL across different vocabulary sizes k is statistically confounded by class-count/entropy differences, and the asserted non-overlap between LibriHeavy k-means training data and LibriSpeech is unverified and could be a leakage risk; however these are correctness/validity concerns, not circularity. No equation or definition reduces any prediction to its input; the conclusions are inductive summaries of the measured NLL tables.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

The paper introduces no new entities or fitted constants. Its claims rest on standard self-supervised speech encoders, k-means clustering, an autoregressive LM objective, and several domain assumptions about proxy metrics and data separation.

axioms (4)
  • domain assumption NLL on LibriSpeech test-clean is a valid proxy for speech modeling capability
    Section 2.4 invokes prior work [15,7] claiming NLL correlates with semantic speech tasks; the paper's 'capabilities' claims therefore depend on unmeasured downstream correlation.
  • domain assumption Final hidden layer features provide the best balance of acoustic and linguistic information
    Section 2.2 fixes the encoder layer to the final hidden layer citing [26,23] without an ablation in this setup.
  • domain assumption The 2,000 hours selected for k-means do not overlap with LibriSpeech
    Section 2.2 asserts no overlap but provides no speaker/book-level verification; LibriHeavy and LibriSpeech share LibriVox origin, so leakage would confound Table 3.
  • domain assumption Montreal Forced Aligner phoneme alignments are accurate enough for unit-to-phoneme assignment
    Section 3.5 builds all phoneme alignment matrices on MFA alignments without reporting alignment error or human verification.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of An Empirical Analysis of Discrete Unit Representations in Speech Language Modeling Pre-training." pith.science (2026). https://pith.science/paper/4X5HR6PP

@misc{pith2026250905359,
  author       = {Pith},
  title        = {Pith review of: An Empirical Analysis of Discrete Unit Representations in Speech Language Modeling Pre-training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4X5HR6PP}},
  note         = {Machine review of arXiv:2509.05359}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

This paper investigates discrete unit representations in Speech Language Models (SLMs), focusing on optimizing speech modeling during continual pre-training. In this paper, we systematically examine how model architecture, data representation, and training robustness influence the pre-training stage in which we adapt existing pre-trained language models to the speech modality. Our experiments highlight the role of speech encoders and clustering granularity across different model scales, showing how optimal discretization strategies vary with model capacity. By examining cluster distribution and phonemic alignments, we investigate the effective use of discrete vocabulary, uncovering both linguistic and paralinguistic patterns. Additionally, we explore the impact of clustering data selection on model robustness, highlighting the importance of domain matching between discretization training and target applications.

Figures

Figures reproduced from arXiv: 2509.05359 by Micka\"el Rouvier, Richard Dufour, Yanis Labrak.

Figure 1
Figure 1. Figure 1: Overview of a Speech Language Model. This stage is critical, as it determines how models initially learn to process speech input and serves as the foundation for all subsequent speech-related capabilities [18, 15]. In this work, we systematically investigate discrete speech unit representations in language modeling. Through extensive experiments across model scales (135M to 1.7B parameters), encoder archit… view at source ↗
Figure 2
Figure 2. Figure 2: Phoneme confusion matrices showing the relationship between predicted dis [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Phoneme confusion matrices showing the relationship between predicted dis [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references · 10 canonical work pages

  1. [1]

    Ardila, R., Branson, M., Davis, K., Kohler, M., Meyer, J., Henretty, M., Morais, R., Saun- ders, L., Tyers, F., Weber, G.: Common voice: A massively-multilingual speech corpus. In: Calzolari, N., B ´echet, F., Blache, P., Choukri, K., Cieri, C., Declerck, T., Goggi, S., Isa- hara, H., Maegaard, B., Mariani, J., Mazo, H., Moreno, A., Odijk, J., Piperidis, ...

    work page

  2. [2]

    In: Interspeech 2022

    Babu, A., Wang, C., Tjandra, A., Lakhotia, K., Xu, Q., Goyal, N., Singh, K., von Platen, P., Saraf, Y ., Pino, J., Baevski, A., Conneau, A., Auli, M.: Xls-r: Self-supervised cross- lingual speech representation learning at scale. In: Interspeech 2022. pp. 2278–2282 (2022). https://doi.org/10.21437/Interspeech.2022-143

    doi:10.21437/interspeech.2022-143 2022

  3. [3]

    In: Proceedings of the 34th International Conference on Neural Information Processing Systems

    Baevski, A., Zhou, H., Mohamed, A., Auli, M.: wav2vec 2.0: a framework for self-supervised learning of speech representations. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. NIPS ’20, Curran Associates Inc., Red Hook, NY , USA (2020)

    work page 2020

  4. [4]

    Ben Allal, L., Lozhkov, A., Bakouch, E.: Smollm - blazingly fast and remarkably powerful (July 2024), https://huggingface.co/blog/smollm, accessed: 2025-02-06

    work page 2024

  5. [5]

    In: Interspeech 2021

    Chen, G., Chai, S., Wang, G.B., Du, J., Zhang, W.Q., Weng, C., Su, D., Povey, D., Tr- mal, J., Zhang, J., Jin, M., Khudanpur, S., Watanabe, S., Zhao, S., Zou, W., Li, X., Yao, X., Wang, Y ., You, Z., Yan, Z.: Gigaspeech: An evolving, multi-domain asr cor- pus with 10,000 hours of transcribed audio. In: Interspeech 2021. pp. 3670–3674 (2021). https://doi.o...

    doi:10.21437/interspeech.2021-1965 2021

  6. [6]

    IEEE Journal of Selected Topics in Signal Processing 16(6), 1505–1518 (2022)

    Chen, S., Wang, C., Chen, Z., Wu, Y ., Liu, S., Chen, Z., Li, J., Kanda, N., Yoshioka, T., Xiao, X., Wu, J., Zhou, L., Ren, S., Qian, Y ., Qian, Y ., Wu, J., Zeng, M., Yu, X., Wei, F.: Wavlm: Large-scale self-supervised pre-training for full stack speech process- ing. IEEE Journal of Selected Topics in Signal Processing 16(6), 1505–1518 (2022). https://do...

    arXiv 2022

  7. [7]

    In: Al-Onaizan, Y ., Bansal, M., Chen, Y .N

    Cuervo, S., Marxer, R.: Scaling properties of speech language models. In: Al-Onaizan, Y ., Bansal, M., Chen, Y .N. (eds.) Proceedings of the 2024 Conference on Empirical Meth- ods in Natural Language Processing. pp. 351–361. Association for Computational Linguis- tics, Miami, Florida, USA (Nov 2024). https://doi.org/10.18653/v1/2024.emnlp-main.21, https:/...

    work page doi:10.18653/v1/2024.emnlp-main.21 2024

  8. [8]

    CoRR abs/2111.09344 (2021), https://arxiv

    Galvez, D., Diamos, G., Ciro, J., Cer ´on, J.F., Achorn, K., Gopi, A., Kanter, D., Lam, M., Mazumder, M., Reddi, V .J.: The people’s speech: A large-scale diverse english speech recog- nition dataset for commercial usage. CoRR abs/2111.09344 (2021), https://arxiv. org/abs/2111.09344

    Pith/arXiv arXiv 2021

  9. [9]

    Hassid, M., Remez, T., Nguyen, T.A., Gat, I., Conneau, A., Kreuk, F., Copet, J., Defossez, A., Synnaeve, G., Dupoux, E., Schwartz, R., Adi, Y .: Textually pretrained speech language models (2024), https://arxiv.org/abs/2305.13009

    Pith/arXiv arXiv 2024

  10. [10]

    IEEE/ACM Trans

    Hsu, W.N., Bolte, B., Tsai, Y .H.H., Lakhotia, K., Salakhutdinov, R., Mohamed, A.: Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Trans. Audio, Speech and Lang. Proc. 29, 3451–3460 (Oct 2021). https://doi.org/10.1109/TASLP.2021.3122291,https://doi.org/10.1109/TASLP. 2021.3122291

    arXiv 2021

  11. [11]

    Hu, E.J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., Chen, W.: Lora: Low- rank adaptation of large language models (2021), https://arxiv.org/abs/2106. 09685

    work page 2021

  12. [12]

    Kang, W., Yang, X., Yao, Z., Kuang, F., Yang, Y ., Guo, L., Lin, L., Povey, D.: Libriheavy: a 50,000 hours asr corpus with punctuation casing and context (2023)

    work page 2023

  13. [13]

    Transactions of the Association for Computational Linguistics 9, 1336–1354 (2021)

    Lakhotia, K., Kharitonov, E., Hsu, W.N., Adi, Y ., Polyak, A., Bolte, B., Nguyen, T.A., Copet, J., Baevski, A., Mohamed, A., Dupoux, E.: On generative spoken language mod- eling from raw audio. Transactions of the Association for Computational Linguistics 9, 1336–1354 (2021). https://doi.org/10.1162/tacl˙a˙00430, https://aclanthology. org/2021.tacl-1.79/

    doi:10.1162/tacl 2021

  14. [14]

    In: International Confer- ence on Learning Representations (2019), https://openreview.net/forum?id= Bkg6RiCqY7

    Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: International Confer- ence on Learning Representations (2019), https://openreview.net/forum?id= Bkg6RiCqY7

    work page 2019

  15. [15]

    Maiti, S., Peng, Y ., Choi, S., weon Jung, J., Chang, X., Watanabe, S.: V oxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text con- tinuation tasks (2024), https://arxiv.org/abs/2309.07937

    Pith/arXiv arXiv 2024

  16. [16]

    In: Interspeech 2017

    McAuliffe, M., Socolof, M., Mihuc, S., Wagner, M., Sonderegger, M.: Montreal forced aligner: Trainable text-speech alignment using kaldi. In: Interspeech 2017. pp. 498–502 (2017). https://doi.org/10.21437/Interspeech.2017-1386

    doi:10.21437/interspeech.2017-1386 2017

  17. [17]

    Mousavi, P., Libera, L.D., Duret, J., Ploujnikov, A., Subakan, C., Ravanelli, M.: Dasb - dis- crete audio and speech benchmark (2024), https://arxiv.org/abs/2406.14294

    Pith/arXiv arXiv 2024

  18. [18]

    Nguyen, T.A., Muller, B., Yu, B., Costa-jussa, M.R., Elbayad, M., Popuri, S., Ropers, C., Duquenne, P.A., Algayres, R., Mavlyutov, R., Gat, I., Williamson, M., Synnaeve, G., Pino, J., Sagot, B., Dupoux, E.: Spirit lm: Interleaved spoken and written language model (2024), https://arxiv.org/abs/2402.05755

    Pith/arXiv arXiv 2024

  19. [19]

    https://doi.org/10.1109/JSTSP.2022.3200909

    Nguyen, T.A., Sagot, B., Dupoux, E.: Are discrete units necessary for spoken language modeling? IEEE Journal of Selected Topics in Signal Processing 16(6), 1415–1423 (2022). https://doi.org/10.1109/JSTSP.2022.3200909

    work page arXiv 2022

  20. [20]

    Nguyen, T.A., de Seyssel, M., Roz ´e, P., Rivi`ere, M., Kharitonov, E., Baevski, A., Dunbar, E., Dupoux, E.: The zero resource speech benchmark 2021: Metrics and baselines for unsuper- vised spoken language modeling (2020), https://arxiv.org/abs/2011.11588

    Pith/arXiv arXiv 2021

  21. [21]

    org/abs/2303.08774

    OpenAI, Achiam, J., Adler, S., et al.: Gpt-4 technical report (2024), https://arxiv. org/abs/2303.08774

    Pith/arXiv arXiv 2024

  22. [22]

    In: 2015 IEEE International Confer- ence on Acoustics, Speech and Signal Processing (ICASSP)

    Panayotov, V ., Chen, G., Povey, D., Khudanpur, S.: Librispeech: An asr cor- pus based on public domain audio books. In: 2015 IEEE International Confer- ence on Acoustics, Speech and Signal Processing (ICASSP). pp. 5206–5210 (2015). https://doi.org/10.1109/ICASSP.2015.7178964

    arXiv 2015

  23. [23]

    In: 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

    Pasad, A., Chou, J.C., Livescu, K.: Layer-wise analysis of a self-supervised speech represen- tation model. In: 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). pp. 914–921 (2021). https://doi.org/10.1109/ASRU51503.2021.9688093 12 Yanis Labrak et al

    arXiv 2021

  24. [24]

    In: ICASSP 2023 - 2023 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP)

    Pasad, A., Shi, B., Livescu, K.: Comparative layer-wise analysis of self- supervised speech models. In: ICASSP 2023 - 2023 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP). pp. 1–5 (2023). https://doi.org/10.1109/ICASSP49357.2023.10096149

    arXiv 2023

  25. [25]

    Touvron, H., Martin, L., et al.: Llama 2: Open foundation and fine-tuned chat models (2023), https://arxiv.org/abs/2307.09288

    Pith/arXiv arXiv 2023

  26. [26]

    In: Interspeech 2023

    Yang, H., Zhao, J., Haffari, G., Shareghi, E.: Investigating pre-trained audio en- coders in the low-resource condition. In: Interspeech 2023. pp. 1498–1502 (2023). https://doi.org/10.21437/Interspeech.2023-343

    work page doi:10.21437/interspeech.2023-343 2023

  27. [27]

    org/abs/2412.02612

    Zeng, A., Du, Z., Liu, M., Wang, K., Jiang, S., Zhao, L., Dong, Y ., Tang, J.: Glm-4-voice: Towards intelligent and human-like end-to-end spoken chatbot (2024),https://arxiv. org/abs/2412.02612

    Pith/arXiv arXiv 2024

  28. [28]

    Zhang, D., Li, S., Zhang, X., Zhan, J., Wang, P., Zhou, Y ., Qiu, X.: Speechgpt: Empowering large language models with intrinsic cross-modal conversational abilities (2023), https: //arxiv.org/abs/2305.11000

    Pith/arXiv arXiv 2023

  29. [4222]

    European Language Resources Association, Marseille, France (May 2020), https: //aclanthology.org/2020.lrec-1.520/

    work page 2020

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.