Pith. sign in

REVIEW 3 major objections 5 minor 33 references

Spoken Language Modeling with Duration-Penalized Self-Supervised Units

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper establishes that the optimal coarseness of discrete speech units for spoken language models depends on the task: coarser units harm phone and word discrimination but improve whole-sentence resynthesis and lexical and syntactic…

desk verdict Useful first systematic look at codebook size × coarseness for SLM units, but the LM gains are confounded by unequal training exposure; the phone/word findings stand. read the letter →

arxiv 2505.23494 v1 pith:XS2QFMVU submitted 2025-05-29 cs.CL eess.AS

Nicol Visser , Herman Kamper This is my paper
classification cs.CLeess.AS
keywords spokenlanguagemodelingduration-penalizeddynamicprogrammingdiscretespeechunitsunitcoarsenesscodebooksizezeroresourcechallengeresynthesisself-supervisedrepresentations
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks how the duration and vocabulary size of discrete speech units affect a spoken language model that is trained entirely without text. It uses duration-penalized dynamic programming (DPDP) to make individual units coarser while holding the codebook size fixed, and it sweeps both codebook size and coarseness over tasks that climb the linguistic hierarchy: phone discrimination, word discrimination, whole-sentence resynthesis, and lexical and syntactic scoring. The central finding is that unit granularity should be matched to the task: for telling phones and words apart, fine K-means units remain best, but for regenerating whole sentences and for lexical and syntactic language modeling, coarser DPDP units beat fine units at the same bitrate as long as the codebook is large enough. If this is right, SLM designers should select unit granularity by the downstream task rather than optimizing only for phonetic fidelity, and DPDP provides a simple, efficient way to obtain coarser units where they help.

What carries the argument

The load-bearing mechanism is duration-penalized dynamic programming (DPDP): a dynamic program over a sequence of frame-wise self-supervised speech features that minimizes the sum of a quantization cost plus a duration reward, concretely $\sum_t (\|x_t - c_{u_t}\|^2 - \lambda \mathbb{1}[u_t = u_{t-1}])$, so raising $\lambda$ merges consecutive frames into longer, coarser units while leaving the K-means codebook untouched. This allows the paper to decouple unit duration from vocabulary size, which earlier systems had varied together. A secondary contribution is a speedup that restricts DPDP's search to the nearest 5% of codebook entries at each frame; the authors report that it matches full-search behavior in development experiments.

What would settle it

Retrain the same unit language model configurations with a matched convergence criterion, for instance training each model until its validation loss stops improving or until every model has seen the same number of distinct utterances, and re-measure sWUGGY and sBLIMP; if the advantage of coarser DPDP units disappears, the reported per-bit gains come from the fixed-step protocol rather than from the units themselves.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that codebook size and unit coarseness interact, and neither is uniformly good. With codebooks of 100 to 1,000 entries and bitrates swept from the full K-means rate down to roughly half of it, increasing the DPDP duration reward makes units longer and more context-dependent. At the phone level and the word level, those coarser units never outperform fine K-means units at a fixed bitrate: they lose the fine phonetic and lexical detail those discriminative tasks need. In whole-sentence resynthesis, however, coarser units from a large codebook yield lower word error rates at low bitrates than fine K-means units, and in the zero-resource lexical and syntactic tasks, coarser units from codebooks above 100 entries give higher accuracies per bit, with the best syntactic system cutting sequence length by 40 percent. The paper concludes that coarser units are beneficial exactly where the model must reason over whole sentences or longer context, and that DPDP is a simple and efficient way to obtain them for those tasks.

Load-bearing premise

The load-bearing premise is that training every unit language model for exactly 10,000 steps with 100,000-token batches is a fair comparison, even though coarser units make each batch contain more distinct utterances, so the models with coarser units may simply have seen more content and converged differently.

Editorial extensions

If this is right

  • For phone and word discrimination, there is no performance-per-bit gain from coarser units; the right codebook size with K-means units is sufficient.
  • For whole-sentence resynthesis, coarser units from a large codebook (K=500) keep word error rates low even around 190 bits per second, while the same coarseness with a small codebook (K=100) causes mispronunciations.
  • For lexical (sWUGGY) and syntactic (sBLIMP) language modeling, coarser units with codebooks above 100 entries improve per-bit accuracy; the best sBLIMP configuration uses K=500 at 190 bits per second and shortens input sequences by 40%.
  • Because the unit language model trains on fixed-length context windows, coarser units reduce the number of training steps proportionally to sequence length, giving faster training at the same context budget.
  • Scaled to 60,000 hours of audio, the DPDP-based system approaches the scores of larger or more complex models on the zero-resource benchmark, and a phone-level model trained on only 1,000 hours still outperforms all the speech-trained models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Task-dependence of the result implies that a single benchmark, such as lexical accuracy alone, can camouflage degradations at the phone level; SLM evaluation should report several linguistic levels at once.
  • If the fixed-step training protocol is not at matched convergence across coarseness levels, the reported gains from coarser units could partly be a training-dynamics artifact; training each model to a convergence criterion would settle this.
  • Since DPDP changes granularity while keeping the codebook fixed, it can be combined with learned or hierarchical codebooks; the trade-offs observed here might shift if the codebook itself adapts to longer contexts.
  • The fact that a 1,000-hour phone-topped model beats 60,000-hour speech models hints that the main bottleneck for SLMs is supervision rather than unit granularity, and coarseness may matter most in low-resource settings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies how codebook size and unit coarseness interact in spoken language models. It builds an SLM using WavLM features, K-means codebooks with K in {100, 200, 500, 1000}, DPDP to increase unit duration, a Mistral-style transformer ULM, and ELLA-V/WavTokenizer for resynthesis. The paper evaluates ABX and same-different discrimination, resynthesis word error rate, and sWUGGY/sBLIMP language modeling at different bitrates. The main finding is that coarser DPDP units hurt phone/word discrimination but, with sufficiently large codebooks, improve whole-sentence resynthesis and lexical/syntactic language modeling at lower bitrates. The paper concludes that DPDP is a simple way to obtain coarser units for tasks where they help, and it includes a scaled comparison to AudioLM and SyllableLM.

Significance. If the interaction between codebook size and coarseness is real, the paper provides a practically useful design rule: choose unit granularity according to the downstream task rather than maximizing phonetic fidelity. The study is systematic across four linguistic levels and the code is public. However, the central quantitative claims currently lack uncertainty estimates, and the fixed training budget may confound coarseness with training exposure; these issues must be resolved before the empirical conclusions can be taken as established.

major comments (3)
  1. [§2.2 and §3.3] The fixed training budget conflates unit coarseness with the number of training epochs. Section 2.2 states that every ULM is trained for 10k steps with 100k-token batches on the same 960-hour corpus. Because coarser units yield shorter sequences, the same token budget corresponds to more passes over the training data: at half the bitrate, the model sees roughly twice as many epochs. Section 3.3 even notes that "the number of steps required reduces proportionally with the sequence length," which is a direct acknowledgment that the 10k-step budget is not epoch-matched. The sWUGGY and sBLIMP gains attributed to coarser units may therefore be driven by increased exposure to the in-domain training set rather than by unit quality. Please retrain or re-report with matched epochs, or add convergence checks and learning curves.
  2. [Figures 3–5] All quantitative comparisons rest on single runs without error bars or significance tests. The differences in sBLIMP accuracy in Figure 5(b) are a few points, and the WER curves in Figure 4 use median WER with no spread. Since a central claim is that coarser units benefit language modeling at larger codebook sizes, the reader cannot tell whether these differences are reliable. Please run multiple seeds (or provide bootstrap/interval estimates) and state significance where claims are made.
  3. [§3.1, footnote 2] The paper selects the any-context ABX variant while acknowledging in footnote 2 that within-context ABX favors coarser representations. Because the summary conclusion "coarseness provides little benefit" at the phone level depends on this choice, omitting the within-context numbers makes the claim difficult to evaluate. Report both variants, or bound the claim explicitly to the any-context setting.
minor comments (5)
  1. [§2.2] The heading "Spoken lanugage model" contains a typo; it should be "Spoken language model."
  2. [§6, reference [11]] The venue "ICCASP" should be "ICASSP."
  3. [§2.1] The claim that the 5% nearest-neighbor search speeds up inference "without affecting performance" is only supported by an allusion to development experiments; please include the quantitative comparison or a short ablation.
  4. [Figure 1] The phone labels such as /S/ and /ER/ would benefit from a broader notation explanation or a legend, since readers may not expect ARPABET-ish symbols without context.
  5. [§2.2] The phrase "unique SLM framework" is overstated because the components (WavLM, K-means, Mistral, ELLA-V, WavTokenizer) are existing; consider replacing it with "our SLM setup."

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the claims are assessed against external benchmarks with an explicitly stated DPDP objective.

full rationale

The paper's central comparisons are made against external benchmarks (ABX, same-different, ASR WER, sWUGGY, sBLIMP), not derived from the DPDP cost function itself. DPDP is defined explicitly in Eq. (1), with lambda as an independent intervention controlling coarseness and codebook size K as an independent variable. No result is obtained by inverting the objective or by fitting a parameter to the target benchmark. The self-citations to Kamper's DPDP papers [8,12] are provenance for a method that is re-implemented here and stated in the paper's own equations; the citation is therefore not load-bearing. The fixed 10k-step training protocol and the selection of K=500 at 190 bps based on earlier dev results are experimental-design concerns (possible training-exposure confounding and mild selection optimism), but they do not make any claimed result equivalent to its inputs by construction. Consequently, no specific circular reduction can be exhibited, and the paper is self-contained against external evaluation.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper is an empirical study: the central claim depends on a chain of chosen hyperparameters (K, lambda grid, WavLM layer, training budgets) and on benchmark assumptions, plus a post hoc ABX variant choice. No new theoretical entities are introduced; DPDP units come from a previously published algorithm, and WavTokenizer, ELLA-V, WavLM, and HuBERT are existing systems reused as components.

free parameters (5)
  • Duration penalty lambda grid = not listed; six values per codebook
    Chosen by hand for each K to span full bitrate down to half; exact values omitted, making replication harder.
  • Codebook size K = 100, 200, 500, 1000
    Four hand-picked sizes, not derived; the paper concludes K=500 works best for resynthesis and language modeling.
  • WavLM layer = layer 11 of WavLM Large
    Selected for best phone discriminability in development experiments; all downstream results depend on this choice.
  • ULM training budget = 10k steps, 100k tokens per step
    Fixed across conditions, but coarser units alter sequence lengths and the number of utterances per batch; no convergence check is reported.
  • Nearest-neighbor search fraction = 5%
    Added to speed up DPDP; the authors claim no performance difference, tested only in development experiments.
assumptions (4)
  • standard math Dynamic programming exactly optimizes Eq. (1) with the duration reward.
    Inherited from Kamper (2023); the current paper changes the formulation from penalty to reward but calls it exactly equivalent.
  • domain assumption WavLM layer-11 features are a sufficient representation for phone, word, sentence, lexical, and syntactic evaluation.
    A single feature layer is used for all tasks; if another layer changed the ranking, the coarseness conclusions could differ.
  • ad hoc to paper The fixed training budget yields comparable model quality across bitrates.
    Coarser units shorten sequences, so each step covers more utterances; the paper does not establish that all models have converged equally.
  • domain assumption ABX, same-different, WER, sWUGGY, and sBLIMP are valid proxies for SLM quality.
    These are standard benchmarks, but ABX variant selection is post hoc and WER depends on the HuBERT ASR model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Spoken Language Modeling with Duration-Penalized Self-Supervised Units." pith.science (2026). https://pith.science/paper/XS2QFMVU

@misc{pith2026250523494,
  author       = {Pith},
  title        = {Pith review of: Spoken Language Modeling with Duration-Penalized Self-Supervised Units},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XS2QFMVU}},
  note         = {Machine review of arXiv:2505.23494}
}
read the original abstract

Spoken language models (SLMs) operate on acoustic units obtained by discretizing self-supervised speech representations. Although the characteristics of these units directly affect performance, the interaction between codebook size and unit coarseness (i.e., duration) remains unexplored. We investigate SLM performance as we vary codebook size and unit coarseness using the simple duration-penalized dynamic programming (DPDP) method. New analyses are performed across different linguistic levels. At the phone and word levels, coarseness provides little benefit, as long as the codebook size is chosen appropriately. However, when producing whole sentences in a resynthesis task, SLMs perform better with coarser units. In lexical and syntactic language modeling tasks, coarser units also give higher accuracies at lower bitrates. We therefore show that coarser units aren't always better, but that DPDP is a simple and efficient way to obtain coarser units for the tasks where they are beneficial.

Figures

Figures reproduced from arXiv: 2505.23494 by the authors.

Figure 1
Figure 1. Coarser and coarser speech units obtained with DPDP. The units are labeled with the most likely phone from the forced alignments. The codebook size is K = 500. encourages contiguous features to be assigned to the same code. Suppose we have a sequence of SSL features x1, x2, . . . , xT that we want to quantize to a set of K codebook vectors {ck} K k=1, with each xt, ck ∈ R D. We want a sequence of code assign￾ments u… view at source ↗
Figure 3
Figure 3. Phone and word discrimination scores of the DPDP units (colored curves) compared to K-means units (black mark￾ers). (a) ABX error rates. (b) Same-different average precision (AP). Both tasks are evaluated on LibriSpeech dev-clean. Same-different task: At a higher linguistic level, the same￾different task [24] evaluates how well the discretized features perform in identifying whether two spoken segments correspond to… view at source ↗
Figure 4
Figure 4. WERs when resynthesizing the dev-clean subset of LibriSpeech using increasingly coarser DPDP units (colored curves) compared to the K-means units (black markers). size K = 500 result in a much lower WER compared to using the K-means units with K = 100. While the larger context￾dependent units are detrimental to phone discrimination, they are beneficial to speech synthesis – provided that the codebook size is large e… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 30 canonical work pages

  1. [1]

    In contrast to models that combine speech and text [2, 3], SLMs do not use text data in any of their components

    Introduction Spoken language models (SLMs) aim to learn language directly from speech audio [1]. In contrast to models that combine speech and text [2, 3], SLMs do not use text data in any of their components. Although this could enable speech technology in settings where text cannot be collected, SLMs still lag far behind text language models [4]. One po...

    work page

  2. [2]

    Spoken Language Modeling with Duration-Penalized Self-Supervised Units

    Model and methodology An SLM starts by tokenizing a raw waveform into a sequence of discrete units. This typically involves quantizing frame- wise features from a self-supervised learning (SSL) model such as HuBERT [13] or WavLM [14] and removing consecutive repeated codes (deduplication) to obtain varying-length units. These units are seen as pseudo-text...

    work page Pith review arXiv 2025

  3. [3]

    Experiments We want to know how codebook size and unit coarseness (how long the units are) affect SLM performance. Do coarser represen- tations aid language modeling? How large can we make the units while still being able to obtain the original content from them, in intelligible synthesized speech? To answer these questions, we conduct evaluations across ...

    work page

  4. [4]

    Tak- ing the results together, it appears that coarser units are not bene- ficial at the lowest levels in discriminating between isolated units like phones or words

    Discussion How do codebook size and unit coarseness affect spoken language models (SLMs), particularly when using duration- penalized dynamic programming (DPDP) for unit extraction? We investigated the interaction between these two crucial aspects in a range of evaluations moving up the linguistic hierarchy. Tak- ing the results together, it appears that ...

    work page

  5. [5]

    Acknowledgements This research was partly supported by the Harry Crossley Foun- dation and the Google PhD fellowship program

    work page

  6. [6]

    AudioLM: a Language Modeling Approach to Audio Generation,

    Z. Borsos, R. Marinier, D. Vincent, E. Kharitonov, O. Pietquin, M. Sharifi, D. Roblek, O. Teboul, D. Grangier, M. Tagliasacchi, and N. Zeghidour, “AudioLM: a Language Modeling Approach to Audio Generation,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 2523–2533, 2023

    work page 2023

  7. [7]

    The Zero Resource Speech Challenge 2021: Spoken language modelling,

    E. Dunbar, M. Bernard, N. Hamilakis, T. A. Nguyen, M. de Seyssel, P. Roz´e, M. Rivi `ere, E. Kharitonov, and E. Dupoux, “The Zero Resource Speech Challenge 2021: Spoken language modelling,” inProc. Interspeech, 2021, pp. 1574–1578

    work page 2021

  8. [8]

    Spirit LM: In- terleaved Spoken and Written Language Model,

    T. A. Nguyen, B. Muller, B. Yu, M. R. Costa-juss`a, M. Elbayad, S. Popuri, P.-A. Duquenne, R. Algayres, R. Mavlyutov, I. Gat, G. Synnaeve, J. Pino, B. Sagot, and E. Dupoux, “Spirit LM: In- terleaved Spoken and Written Language Model,”Transactions of the Association for Computational Linguistics, vol. 13, pp. 30–52, 2024

    work page 2024

Show all 33 references
  1. [9]

    Textually Pretrained Speech Language Models,

    M. Hassid, T. Remez, T. A. Nguyen, I. Gat, A. Conneau, F. Kreuk, J. Copet, A. Defossez, G. Synnaeve, E. Dupoux, R. Schwartz, and Y . Adi, “Textually Pretrained Speech Language Models,” inProc. NeurIPS, 2023, pp. 63 483–63 501

    2023

  2. [10]

    Improving Spoken Language Modeling with Phoneme Classification: A Simple Fine-tuning Approach,

    M. Poli, E. Chemla, and E. Dupoux, “Improving Spoken Language Modeling with Phoneme Classification: A Simple Fine-tuning Approach,” inProc. EMNLP, 2024, pp. 5284–5292

    2024

  3. [11]

    Generative Spoken Language Modeling From Raw Audio,

    K. Lakhotia, E. Kharitonov, W. N. Hsu, Y . Adi, A. Polyak, B. Bolte, T. A. Nguyen, J. Copet, A. Baevski, A. Mohamed, and E. Dupoux, “Generative Spoken Language Modeling From Raw Audio,”Trans- actions of the Association for Computational Linguistics, vol. 9, pp. 1336–1354, 2021

    2021

  4. [12]

    Word Segmentation on Discovered Phone Units With Dynamic Programming and Self-Supervised Scoring,

    H. Kamper, “Word Segmentation on Discovered Phone Units With Dynamic Programming and Self-Supervised Scoring,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 684–694, 2023

    2023

  5. [13]

    Generative Spoken Language Model based on continuous word-sized audio tokens,

    R. Algayres, Y . Adi, T. A. Nguyen, J. Copet, G. Synnaeve, B. Sagot, and E. Dupoux, “Generative Spoken Language Model based on continuous word-sized audio tokens,” inProc. EMNLP, 2023, pp. 3008–3028

    2023

  6. [14]

    Towards Unsupervised Phone and Word Segmentation Using Self-Supervised Vector-Quantized Neural Networks,

    H. Kamper and B. van Niekerk, “Towards Unsupervised Phone and Word Segmentation Using Self-Supervised Vector-Quantized Neural Networks,” inProc. Interspeech, 2021, pp. 1539–1543

    2021

  7. [15]

    Unsupervised Speech Segmentation and Variable Rate Represen- tation Learning Using Segmental Contrastive Predictive Coding,

    S. Bhati, J. Villalba, P. ˙Zelasko, L. Moro-Velazquez, and N. Dehak, “Unsupervised Speech Segmentation and Variable Rate Represen- tation Learning Using Segmental Contrastive Predictive Coding,” IEEE/ACM Transactions on Audio, Speech, and Language Pro- cessing, vol. 30, p. 200...

    2002

  8. [16]

    SyllableLM: Learning Coarse Semantic Units for Speech Language Models,

    A. Baade, P. Peng, and D. Harwath, “SyllableLM: Learning Coarse Semantic Units for Speech Language Models,” inProc. ICLR, 2025

    2025

  9. [17]

    Acoustic BPE for Speech Generation with Discrete tokens,

    F. Shen, Y . Guo, C. Du, X. Chen, and K. Yu, “Acoustic BPE for Speech Generation with Discrete tokens,” inProc. ICCASP, 2024, pp. 11 746–11 750

    2024

  10. [18]

    RoFormer: Enhanced Transformer with Rotary Position Embedding,

    J. Su, Y . Lu, S. Pan, A. Murtadha, B. Wen, and Y . Liu, “RoFormer: Enhanced Transformer with Rotary Position Embedding,”Neuro- computing, vol. 568, 2024

    2024

  11. [19]

    HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units,

    W. Hsu, B. Bolte, Y . H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 3451–3460, 2021

    2021

  12. [20]

    Wavlm: Large-Scale Self- Supervised Pre-training for Full Stack Speech Processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L. Zhou, S. Ren, Y . Qian, Y . Qian, J. Wu, M. Zeng, X. Yu, and F. Wei, “Wavlm: Large-Scale Self- Supervised Pre-training for Full Stack Speech Processing,”IEEE Journal of Selecte...

    2021

  13. [21]

    LibriSpeech: An ASR corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “LibriSpeech: An ASR corpus based on public domain audio books,” inProc. ICASSP, 2015, pp. 5206–5210

    2015

  14. [22]

    The Faiss library,

    M. Douze, A. Guzhva, C. Deng, J. Johnson, G. Szilvasy, P.-E. Mazar´e, M. Lomeli, L. Hosseini, and H. J´egou, “The Faiss library,”

  15. [23]

    Evaluating context- invariance in unsupervised speech representations,

    M. Hallap, E. Dupoux, and E. Dunbar, “Evaluating context- invariance in unsupervised speech representations,” inProc. Inter- speech, 2023, pp. 2973–2977

    2023

  16. [24]

    Mistral 7B,

    A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. de las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M.-A. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed, “Mistral 7B,”

  17. [25]

    Libri-Light: A Benchmark for ASR with Limited or No Supervi- sion,

    J. Kahn, M. Rivi`ere, W. Zheng, E. Kharitonov, Q. Xu, P. E. Mazar´e, J. Karadayi, V . Liptchinsky, R. Collobert, C. Fuegen, T. Likhoma- nenko, G. Synnaeve, A. Joulin, A. Mohamed, and E. Dupoux, “Libri-Light: A Benchmark for ASR with Limited or No Supervi- sion,” inProc. ICASSP...

    2020

  18. [27]

    WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling,

    S. Ji, Z. Jiang, W. Wang, Y . Chen, M. Fang, J. Zuo, Q. Yang, X. Cheng, Z. Wang, R. Liet al., “WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling,” inProc. ICLR, 2025

    2025

  19. [28]

    ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reordering,

    Y . Song, Z. Chen, X. Wang, Z. Ma, and X. Chen, “ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reordering,” inProc. AAAI Conf. Artif. Intell., vol. 39, no. 24, 2025, pp. 25 174–25 182

    2025

  20. [29]

    Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers,

    C. Wang, S. Chen, Y . Wu, Z. Zhang, L. Zhou, S. Liu, Z. Chen, Y . Liu, H. Wang, J. Li, L. He, S. Zhao, and F. Wei, “Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers,” IEEE/ACM Transactions on Audio, Speech, and Language Pro- cessing, vol. 33, pp. 705–718, 2023

    2023

  21. [30]

    Evaluating Speech Features with the Minimal-Pair ABX Task: Analysis of the Classical MFC/PLP Pipeline,

    T. Schatz, V . Peddinti, F. Bach, A. Jansen, H. Hermansky, and E. Dupoux, “Evaluating Speech Features with the Minimal-Pair ABX Task: Analysis of the Classical MFC/PLP Pipeline,” inProc. Interspeech, 2013, pp. 1781–1785

    2013

  22. [32]

    Rapid Evaluation of Speech Representations for Spoken Term Discovery,

    M. A. Carlin, S. Thomas, A. Jansen, and H. Hermansky, “Rapid Evaluation of Speech Representations for Spoken Term Discovery,” inProc. Interspeech, 2011, pp. 821–824

    2011

  23. [100]

    manu- facture

    Other systems that use DPDP to get coarser units from K >100 achieve either the same or higher error rates. Thus, for phone discrimination, coarser units prove detrimental to performance. The coarser units are context-dependent and lack the fine phonetic distinctions required ...

  24. [2023]

    Available: https://arxiv.org/abs/2310.06825

    [Online]. Available: https://arxiv.org/abs/2310.06825

    arXiv

  25. [2024]

    Available: https://arxiv.org/abs/2401.08281

    [Online]. Available: https://arxiv.org/abs/2401.08281

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.