Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

A codec running at 12.5 frames per second can match or beat higher-frame-rate codecs on reconstruction quality while sharply cutting the autoregressive steps a speech LLM must take.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

NanoCodec achieves competitive speech quality at 12.5 frames per second and 0.6-1.78 kbps, with a causal decoder for low-latency speech LLM inference.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection NanoCodec is a solid, incremental codec paper with real ablations and a downstream TTS test, but the headline accuracy claims rest on point estimates without variance and the 12.5 FPS trade-off is more muted than the 'high-quality ultra fast' framing suggests. the 4 major comments →

arxiv 2508.05835 v1 pith:75C7JM4L submitted 2025-08-07 eess.AS cs.CLcs.SD

NanoCodec: Towards High-Quality Ultra Fast Speech LLM Inference

Edresson Casanova , Paarth Neekhara , Ryan Langman , Shehzeen Hussain , Subhankar Ghosh , Xuesong Yang , Ante Juki\'c , Jason Li
show 1 more author
Boris Ginsburg
This is my paper
classification eess.AS cs.CLcs.SD
keywords neural audio codeclow frame ratespeech LLM inferencespeech codingfinite scalar quantizationcausal decoderzero-shot text-to-speechintelligibility
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that low frame rate is not a quality sacrifice if the codec architecture is adjusted properly, and introduces NanoCodec, a neural speech codec that runs at 12.5 frames per second. At 1.78 kbps, NanoCodec matches or exceeds the 21.5 FPS LFSC codec on most reconstruction metrics (for example SQMOS 4.441 vs 4.432 and SECS 0.862 vs 0.841) while using a causal decoder and about 45% fewer parameters. At 1.1 kbps it reports better scores than Mimi on every metric, with especially large gains in intelligibility (CER 3.617 vs 7.221). The paper also shows that a TTS model trained on NanoCodec tokens produces comparable or better intelligibility with roughly 2.3x faster real-time factor and much shorter time-to-first-audio. A sympathetic reader would care because fewer frames per second means fewer autoregressive steps, which is the main bottleneck in speech LLM training and streaming inference.

Core claim

The central discovery is that a codec designed for 12.5 FPS can preserve the reconstruction quality of a 21.5 FPS codec while enabling streaming-friendly decoding, provided the right architectural choices are made. NanoCodec keeps LFSC's FSQ-based quantization and HiFi-GAN-style generator, but adds dilated residual blocks with rates 1, 3, and 5, switches decoder activations from Leaky ReLU to Snake, uses a multi-band multi-scale STFT discriminator, and adds a speaker-consistency loss that pushes reconstructed speech toward the same speaker embedding as the ground truth. These modifications let the 1.78 kbps, 12.5 FPS, partially causal model match or beat LFSC on SQMOS, mel distance, speaker

What carries the argument

The load-bearing component is a partially causal, fully convolutional encoder-decoder with FSQ (Finite Scalar Quantization) vector quantization running at 12.5 frames per second, paired with a causal decoder. At 12.5 FPS, one second of speech becomes 12.5 code sequences rather than 21.5, so an autoregressive speech LLM needs about 42% fewer steps; the causal decoder means those steps can be decoded streamingly without a five-frame lookahead cache. The ablation design isolates frame rate, causality, and bitrate, and the speaker-consistency loss plus the discriminators carry the quality improvements.

Load-bearing premise

The headline comparisons assume the five automatic metrics (SQMOS, PESQ, mel distance, SECS, and CER) capture what human listeners care about, since the codec evaluation reports no human listening test and no confidence intervals.

What would settle it

Run a controlled human listening test on the same MLS and DAPS samples, comparing NanoCodec at 1.78 kbps/12.5 FPS and 1.1 kbps/12.5 FPS against LFSC at 1.89 kbps/21.5 FPS and Mimi at 1.1 kbps. If listeners rate the 12.5 FPS reconstructions as worse, or if the reported metric gaps fall inside confidence intervals, the claim that frame rate can be halved without quality loss fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A speech LLM trained on NanoCodec tokens at 12.5 FPS reached CER 0.49 on the LibriTTS test-clean set, beating the LFSC-trained TTS's CER 0.85, while real-time factor improved 2.33x and time-to-first-audio dropped by roughly 2.5x compared with the 21.5 FPS LFSC model.
  • At 1.1 kbps, NanoCodec reports better SQMOS, PESQ, mel distance, SECS, and CER than Mimi, with CER 3.617 vs 7.221, so the 12.5 FPS format does not force an intelligibility penalty at the lowest bitrate the paper compares.
  • Frame-rate reduction from 21.5 to 12.5 FPS at nearly the same bitrate leaves SQMOS, speaker similarity, and CER comparable, suggesting the quality bottleneck is not the frame rate itself but the amount of speech generated per autoregressive step.
  • Lower bitrates hurt intelligibility far more than other metrics: CER rises from 2.423 at 1.78 kbps to 3.617 at 1.1 kbps and 6.519 at 0.8 kbps, so naive bitrate reduction for simpler LLM token distributions carries a measurable intelligibility cost.
  • Lengthening the TTS context from 5 to 10 seconds lifted speaker similarity from 0.635 to 0.691 with the 12.5 FPS codec, indicating that longer context can partially compensate for the reduced number of tokens available per utterance.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the automatic-metric results hold up in human listening, the main remaining bottleneck for fast speech LLMs is not codec quality but the autoregressive model's ability to work with fewer, longer tokens; architectural changes like explicit duration modeling or longer conditioning context may be more productive than pushing frame rate even lower.
  • The success of a noncausal encoder plus causal decoder suggests that other codecs could adopt the same split to gain streaming capability at low latency, since only the decoder is used during TTS or speech-to-speech inference; this is a testable recipe for future codec designs.
  • The sharp CER degradation below 1.1 kbps points to a practical floor for pure reconstruction-based codecs; hybrid approaches that inject semantic or phonetic information into the quantized tokens might push intelligible bitrates lower than 1 kbps, which the paper does not explore.
  • Because the paper reports point estimates without confidence intervals for the codec metrics, a direct re-run with multiple seeds or significance testing would clarify which small SQMOS gaps (for example 4.441 vs 4.432) are robust; this is a check available to any group with the public checkpoints.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces NanoCodec, a low-frame-rate neural speech codec operating at 12.5 frames per second (FPS) and 1.78 kbps, built by modifying the LFSC architecture with a partially causal configuration (noncausal encoder, causal decoder), Speaker Consistency Loss, Snake activation, and a multi-band multi-scale STFT discriminator. The paper reports ablation studies on frame rate, causality, and bitrate; compares reconstruction quality against LFSC, Mimi, WavTokenizer, TS3-Codec, and TAAE using objective metrics (SQMOS, PESQ, Mel distance, SECS, CER); and evaluates NanoCodec in the Koel-TTS downstream task with human MOS, CER, SECS, RTF, and TTFA measurements. The central claim is that NanoCodec outperforms related codecs across bitrate ranges while enabling faster Speech LLM inference.

Significance. If the stated results hold, NanoCodec is a useful contribution: it demonstrates that 12.5 FPS with a causal decoder is feasible for speech LLM inference, releases code and checkpoints, and includes a downstream TTS evaluation as a realistic stress test. The ablation study is systematic, and the direct comparisons against public checkpoints are falsifiable. The main value is in showing that low frame rate does not necessarily degrade reconstruction quality on objective metrics. However, the significance currently depends on point estimates of automated predictors without variance or human listening for the codec itself, and the downstream TTS experiment shows clear trade-offs in quality and speaker similarity at the headline 12.5 FPS setting. These issues must be addressed before the superiority claims can be accepted.

major comments (4)
  1. [Section 2.3.2, Table 2] The SOTA claim is supported only by point estimates of SQMOS, PESQ, Mel distance, SECS, and CER, with no confidence intervals, error bars, or significance tests. Several key differences are extremely small (e.g., SQMOS 4.441 vs 4.432 for NanoCodec 1.78kbps vs LFSC; PESQ 2.760 vs 2.831, where LFSC is higher; DAPS CER 0.758 vs 0.655, where LFSC is better). These margins could easily be within evaluation noise, especially since SQMOS is a learned predictor rather than a subjective MOS. The paper should report variance across evaluation subsets or repeated decoding, perform significance tests, or explicitly temper the claim to 'comparable on objective metrics' where appropriate. Without this, the headline 'outperforms related works' is not robustly established.
  2. [Section 3.2, Table 3] The intended ultra-fast 12.5 FPS configuration shows degraded downstream quality and speaker similarity relative to the 21.5 FPS LFSC baseline: MOS 4.01±0.04 vs 4.17±0.04 and SECS 0.635 vs 0.719. With a 10s context, MOS drops further to 3.84. The paper acknowledges this and attributes it to the AR step generating more speech and shorter context, but these are hypotheses, and the conclusion 'fully harnessing the potential... may require architectural refinements' is a significant limitation. As it stands, the 'high-quality' part of the central claim is not supported in the target use case. The authors should either provide evidence that the 12.5 FPS quality gap can be closed (e.g., fine-tuning, architecture changes, or longer context) or reframe the contribution as a latency/efficiency improvement with a measured quality trade-off.
  3. [Section 2.3.1, Table 1] The decision to adopt the partially causal model (noncausal encoder, causal decoder) is based on the 1.1 kbps ablation, where the partially causal model is compared to the fully noncausal model. However, the final headline 1.78 kbps 12.5 FPS model is only evaluated in the partially causal configuration; there is no noncausal or fully causal ablation at 1.78 kbps. Thus the claim that partial causality is 'beneficial' at the operating point of interest is not directly demonstrated. Additionally, the justification that 'only the codec decoder is used at inference in TTS/S2S' is questionable: in zero-shot TTS, the reference/prompt utterance must also be tokenized by the codec encoder during inference. This should be qualified; noncausal encoding of a pre-recorded reference is acceptable, but 'only the decoder runs' is an oversimplification.
  4. [Section 2.3.2, Table 2] The comparisons across bitrates are not fully controlled: NanoCodec 0.8 kbps is compared to WavTokenizer 0.9 kbps, and 0.6 kbps to TAAE 0.7 kbps; TS3-Codec and TAAE operate at 16 kHz while NanoCodec is 22.05 kHz. These differences affect metrics and make the 'across bitrate ranges' claim less clear. The authors should at least discuss the effect of bitrate and sampling-rate mismatches, or provide matched-bitrate comparisons. Also, the 'ultra fast' claim refers to Speech LLM inference, but no direct runtime or complexity measurement is provided for the codec itself (only TTS RTF/TTFA); reporting codec encoder/decoder inference time would strengthen the contribution.
minor comments (4)
  1. [Table 1] Table 1 contains duplicated rows (e.g., the 1.1 kbps/12.5 FPS Noncausal/Causal row appears twice, and the 1.78 kbps/12.5 FPS row appears twice). Please remove redundant entries and ensure each configuration is listed once.
  2. [Section 2.1] The dataset name 'Common V oice' is a typo; it should be 'Common Voice'. Also, the paper defines 'Frames/Sec' inconsistently in Table 1 (e.g., token/sec and Frames/sec are both listed); please unify the terminology.
  3. [Eq. (1)] The Speaker Consistency Loss uses a fixed scaling factor α=0.1, but no sensitivity analysis or ablation for α is reported. Since this is a hyperparameter in the loss, a brief ablation or at least a discussion of its effect would improve reproducibility.
  4. [Abstract and Section 2.3.2] The phrase 'state-of-the-art' and 'outperforms related works' should be qualified with 'on the evaluated objective metrics' or similar, because the codec reconstruction is not evaluated with human listeners and several comparisons are within small margins. This would align the claims with the evidence.

Circularity Check

0 steps flagged

No significant circularity: NanoCodec's claims are empirical measurements against external baselines and public checkpoints, not derivations from its own outputs.

full rationale

The paper is an empirical systems paper; there is no derivation chain in which a predicted quantity is defined in terms of the input or in which a fitted parameter is renamed as a prediction. The central comparisons (Tables 1-3) are direct measurements of reconstruction and TTS quality using external metrics (SQMOS, PESQ, Mel Dist., SECS, CER) computed on held-out test sets (MLS 44.1kHz, DAPS) and, for TTS, human MOS ratings plus external ASR/speaker models (Parakeet-TDT, Titanet-Small). References to the authors' own prior work (LFSC [2], Koel-TTS [4], Spectral Codecs [9]) are used to justify architecture choices, baselines, and evaluation protocols, but the claims do not reduce to those citations: NanoCodec's performance is measured against public checkpoints of LFSC, Mimi, WavTokenizer, TAAE, and TS3-Codec, and the results are falsifiable numbers. The paper honestly discloses limitations of the 12.5 FPS TTS model (lower speaker similarity and MOS, Table 3) and hypothesizes causes, which further shows the reported SOTA claim is not forced by construction. The lack of error bars/significance testing and absence of human listening for codec reconstructions are statistical-validity concerns, not circularity. No step in the paper equates an output with an input by definition, so the circularity score is 0.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

No new physical or conceptual entities are introduced; NanoCodec is a specific system built from established components. The listed free parameters are hand-chosen design values that shape the reported results.

free parameters (4)
  • SCL scaling factor alpha = 0.1
    Hand-chosen weight for the speaker consistency loss; directly influences speaker similarity results in Table 1.
  • Codebook configuration (levels per codebook) = [8,7,6,6] with 8 codebooks and 4 dims
    Design choice that sets the total code count (2016) and bitrate; varied in ablations for different bitrates.
  • Stride patterns per frame rate = [2,3,6,7,7] for 12.5 FPS
    Selected to achieve target frame rates; affects temporal resolution and quality.
  • Initial channel counts = 24 encoder, 864 decoder
    Chosen to reduce parameters by 45% vs LFSC; impacts model capacity and quality.
axioms (4)
  • domain assumption SQMOS, PESQ, Mel Dist, SECS, and CER are valid proxies for perceptual quality and intelligibility of reconstructed speech.
    Underpins all quality comparisons and the SOTA claim; no human listening test for codec outputs in the paper.
  • domain assumption The MLS 44.1kHz test set and DAPS F10/M10 speakers are representative samples for evaluating multilingual codec quality.
    All ablations and comparisons use these subsets; generalization to other conditions is assumed.
  • ad hoc to paper A non-causal encoder with a causal decoder is an acceptable configuration for streaming TTS because only the decoder runs at inference in such systems.
    Stated in Section 2.3.1 to justify adopting the partially causal model; specific to the TTS use-case.
  • domain assumption English speech contains roughly 10-12 phonemes per second, so 12.5 FPS captures about one phoneme per frame.
    Used to explain the 6.25 FPS quality drop; phoneme rate is cited from a textbook but not measured on the datasets.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of NanoCodec: Towards High-Quality Ultra Fast Speech LLM Inference." pith.science (2026). https://pith.science/paper/75C7JM4L

@misc{pith2026250805835,
  author       = {Pith},
  title        = {Pith review of: NanoCodec: Towards High-Quality Ultra Fast Speech LLM Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/75C7JM4L}},
  note         = {Machine review of arXiv:2508.05835}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large Language Models (LLMs) have significantly advanced audio processing by leveraging audio codecs to discretize audio into tokens, enabling the application of language modeling techniques to speech data. However, existing audio codecs often operate at high frame rates, leading to slow training and inference, particularly for autoregressive models. To address this, there is growing interest in low frame-rate audio codecs, which reduce the number of autoregressive steps required to generate one second of audio. In this paper, we conduct ablation studies to examine the impact of frame rate, bitrate, and causality on codec reconstruction quality. Based on our findings, we introduce NanoCodec, a state-of-the-art audio codec that achieves high-quality compression at just 12.5 frames per second (FPS). NanoCodec outperforms related works across various bitrate ranges, establishing a new benchmark for low-latency and efficient Speech LLM training and inference.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. IRAF: Interference-Resilient Adaptive Fusion for Noise-Robust End-to-End Full-Duplex Spoken Dialogue Systems

    cs.SD 2026-06 unverdicted novelty 4.0

    IRAF introduces an adaptive fusion module that uses a predicted scalar reliability gate to reduce the impact of interfering speakers on user audio representations in end-to-end full-duplex spoken dialogue systems, wit...

Reference graph

Works this paper leans on

40 extracted references · 25 canonical work pages · cited by 1 Pith paper · 5 internal anchors

  1. [1]

    These technologies have been central to fields such as audio transmission and commu- nication [1, 2]

    Introduction Audio coding is an important signal processing technique, that compresses audio signals into discrete codes and then uses these codes to reconstruct the original audio. These technologies have been central to fields such as audio transmission and commu- nication [1, 2]. Recently, audio codecs have been applied to various downstream tasks. For...

    Pith/arXiv arXiv 2025

  2. [2]

    The model consists of a fully convolutional generator network and three discriminators

    NanoCodec NanoCodec builds upon the LFSC [2], incorporating architec- tural modifications to enhance audio quality and intelligibility, while reducing the model number of parameters. The model consists of a fully convolutional generator network and three discriminators. The generator includes an encoder, vector quantization, and a HiFi-GAN-based decoder [...

    work page 2016

  3. [3]

    Experiments setup To assess the performance of our codec in comparison to the LFSC and explore the effects of bitrate reduction, we adopted Koel-TTS [4], a SOTA LLM-based TTS model

    Zero-Shot TTS study 3.1. Experiments setup To assess the performance of our codec in comparison to the LFSC and explore the effects of bitrate reduction, we adopted Koel-TTS [4], a SOTA LLM-based TTS model. Among Koel- TTS architectures, we use the Decoder Context variant with Classifier-Free Guidance (CFG) as it achieves the best perfor- mance in zero-sh...

    work page

  4. [4]

    Ex- perimental results demonstrate that NanoCodec surpasses exist- ing approaches at the same bitrate, offering significantly higher intelligibility and speaker similarity

    Conclusions and future work In this work, we introduced NanoCodec, a state-of-the-art au- dio codec that achieves high-quality compression at a bitrate of 1.78 kbps and a frame rate of 12.5 frames per second. Ex- perimental results demonstrate that NanoCodec surpasses exist- ing approaches at the same bitrate, offering significantly higher intelligibility...

    work page

  5. [5]

    Apcodec: A neural audio codec with parallel amplitude and phase spectrum encoding and decoding,

    Y . Ai, X.-H. Jiang, Y .-X. Lu, H.-P. Du, and Z.-H. Ling, “Apcodec: A neural audio codec with parallel amplitude and phase spectrum encoding and decoding,” IEEE/ACM Trans. Audio, Speech and Lang. Proc., vol. 32, p. 3256–3269, jun 2024. [Online]. Available: https://doi.org/10.1109/TASLP.2024.3417347

    arXiv 2024

  6. [6]

    Low frame-rate speech codec: a codec designed for fast high-quality speech llm training and in- ference,

    E. Casanova, R. Langman, P. Neekhara, S. Hussain, J. Li, S. Ghosh, A. Juki´c, and S.-g. Lee, “Low frame-rate speech codec: a codec designed for fast high-quality speech llm training and in- ference,” ICASSP, 2025

    work page 2025

  7. [7]

    Speechtok- enizer: Unified speech tokenizer for speech large language mod- els,

    X. Zhang, D. Zhang, S. Li, Y . Zhou, and X. Qiu, “Speechtok- enizer: Unified speech tokenizer for speech large language mod- els,” arXiv preprint arXiv:2308.16692, 2023

    Pith/arXiv arXiv 2023

  8. [8]

    Koel- tts: Enhancing llm based speech generation with preference alignment and classifier free guidance,

    S. Hussain, P. Neekhara, X. Yang, E. Casanova, S. Ghosh, M. T. Desta, R. Fejgin, R. Valle, and J. Li, “Koel- tts: Enhancing llm based speech generation with preference alignment and classifier free guidance,” 2025. [Online]. Available: https://arxiv.org/abs/2502.05236

    Pith/arXiv arXiv 2025

  9. [9]

    Textless direct speech-to-speech translation with discrete speech representation,

    X. Li, Y . Jia, and C.-C. Chiu, “Textless direct speech-to-speech translation with discrete speech representation,” in ICASSP 2023- 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    work page 2023

  10. [10]

    Textless Unit-to-Unit training for Many-to-Many Multilingual Speech-to-Speech Translation

    M. Kim, J. Choi, D. Kim, and Y . M. Ro, “Many-to-many spoken language translation via unified speech and text repre- sentation learning with unit-to-unit translation,” arXiv preprint arXiv:2308.01831, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  11. [11]

    Soundstream: An end-to-end neural audio codec,

    N. Zeghidour, A. Luebs, A. Omran, J. Skoglund, and M. Tagliasacchi, “Soundstream: An end-to-end neural audio codec,” IEEE/ACM Transactions on Audio, Speech, and Lan- guage Processing, vol. 30, pp. 495–507, 2021

    2021

  12. [12]

    High fidelity neural audio compression,

    A. D ´efossez, J. Copet, G. Synnaeve, and Y . Adi, “High fidelity neural audio compression,” arXiv preprint arXiv:2210.13438 , 2022

    Pith/arXiv arXiv 2022

  13. [13]

    Spectral codecs: Spectrogram-based audio codecs for high quality speech synthesis,

    R. Langman, A. Juki ´c, K. Dhawan, N. R. Koluguri, and B. Gins- burg, “Spectral codecs: Spectrogram-based audio codecs for high quality speech synthesis,” arXiv preprint arXiv:2406.05298, 2024

    Pith/arXiv arXiv 2024

  14. [14]

    High-fidelity audio compression with improved rvqgan,

    R. Kumar, P. Seetharaman, A. Luebs, I. Kumar, and K. Kumar, “High-fidelity audio compression with improved rvqgan,” Ad- vances in Neural Information Processing Systems, vol. 36, 2024

    work page 2024

  15. [15]

    A review of vector quantization tech- niques,

    A. Vasuki and P. Vanathi, “A review of vector quantization tech- niques,” IEEE Potentials, vol. 25, no. 4, pp. 39–47, 2006

    work page 2006

  16. [16]

    Gull: A Generative Multifunctional Audio Codec

    Y . Luo, J. Yu, H. Chen, R. Gu, and C. Weng, “Gull: A generative multifunctional audio codec,” arXiv preprint arXiv:2404.04947 , 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  17. [17]

    Fi- nite scalar quantization: Vq-vae made simple,

    F. Mentzer, D. Minnen, E. Agustsson, and M. Tschannen, “Fi- nite scalar quantization: Vq-vae made simple,” arXiv preprint arXiv:2309.15505, 2023

    Pith/arXiv arXiv 2023

  18. [18]

    An Intra-BRNN and GB-RVQ Based END-TO-END Neural Audio Codec

    L. Xu, J. Jiang, D. Zhang, X. Xia, L. Chen, Y . Xiao, P. Ding, S. Song, S. Yin, and F. Sohel, “An intra-brnn and gb-rvq based end-to-end neural audio codec,” arXiv preprint arXiv:2402.01271, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  19. [19]

    Lightcodec: A high fidelity neural audio codec with low computation complexity,

    L. Xu, J. Wang, J. Zhang, and X. Xie, “Lightcodec: A high fidelity neural audio codec with low computation complexity,” inICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 586–590

    work page 2024

  20. [20]

    Wavtokenizer: an efficient acous- tic discrete codec tokenizer for audio language modeling,

    S. Ji, Z. Jiang, X. Cheng, Y . Chen, M. Fang, J. Zuo, Q. Yang, R. Li, Z. Zhang, X. Yanget al., “Wavtokenizer: an efficient acous- tic discrete codec tokenizer for audio language modeling,” arXiv preprint arXiv:2408.16532, 2024

    Pith/arXiv arXiv 2024

  21. [21]

    Scaling transformers for low-bitrate high-quality speech coding,

    J. D. Parker, A. Smirnov, J. Pons, C. Carr, Z. Zukowski, Z. Evans, and X. Liu, “Scaling transformers for low-bitrate high-quality speech coding,” arXiv preprint arXiv:2411.19842, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    Ts3-codec: Transformer-based simple streaming single codec,

    H. Wu, N. Kanda, S. E. Eskimez, and J. Li, “Ts3-codec: Transformer-based simple streaming single codec,”arXiv preprint arXiv:2411.18803, 2024

    Pith/arXiv arXiv 2024

  23. [23]

    Moshi: a speech-text foundation model for real-time dialogue,

    A. D ´efossez, L. Mazar´e, M. Orsini, A. Royer, P. P´erez, H. J´egou, E. Grave, and N. Zeghidour, “Moshi: a speech-text foundation model for real-time dialogue,” arXiv preprint arXiv:2410.00037, 2024

    Pith/arXiv arXiv 2024

  24. [24]

    Hifi-gan: Generative adversarial net- works for efficient and high fidelity speech synthesis,

    J. Kong, J. Kim, and J. Bae, “Hifi-gan: Generative adversarial net- works for efficient and high fidelity speech synthesis,” Advances in neural information processing systems , vol. 33, pp. 17 022– 17 033, 2020

    work page 2020

  25. [25]

    Bigvgan: A universal neural vocoder with large-scale training,

    S.-g. Lee, W. Ping, B. Ginsburg, B. Catanzaro, and S. Yoon, “Bigvgan: A universal neural vocoder with large-scale training,” arXiv preprint arXiv:2206.04658, 2022

    Pith/arXiv arXiv 2022

  26. [26]

    Neural networks fail to learn periodic functions and how to fix it,

    L. Ziyin, T. Hartwig, and M. Ueda, “Neural networks fail to learn periodic functions and how to fix it,” Advances in Neural Infor- mation Processing Systems, vol. 33, pp. 1583–1594, 2020

    work page 2020

  27. [27]

    Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone,

    E. Casanova, J. Weber, C. D. Shulby, A. C. Junior, E. G ¨olge, and M. A. Ponti, “Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone,” in International Con- ference on Machine Learning. PMLR, 2022, pp. 2709–2720

    work page 2022

  28. [28]

    Clova Baseline System for the VoxCeleb Speaker Recognition Challenge 2020

    H. S. Heo, B.-J. Lee, J. Huh, and J. S. Chung, “Clova baseline sys- tem for the voxceleb speaker recognition challenge 2020,” arXiv preprint arXiv:2009.14153, 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  29. [29]

    Common voice: A massively-multilingual speech corpus,

    R. Ardila, M. Branson, K. Davis, M. Kohler, J. Meyer, M. Hen- retty, R. Morais, L. Saunders, F. Tyers, and G. Weber, “Common voice: A massively-multilingual speech corpus,” in Proceedings of the Twelfth Language Resources and Evaluation Conference , 2020, pp. 4218–4222

    work page 2020

  30. [30]

    Adam: A method for stochastic opti- mization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic opti- mization,” arXiv preprint arXiv:1412.6980, 2014

    Pith/arXiv arXiv 2014

  31. [31]

    Torchaudio-squim: Reference-less speech quality and intelligibility measures in torchaudio,

    A. Kumar, K. Tan, Z. Ni, P. Manocha, X. Zhang, E. Henderson, and B. Xu, “Torchaudio-squim: Reference-less speech quality and intelligibility measures in torchaudio,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    work page 2023

  32. [32]

    Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs,

    A. W. Rix, J. G. Beerends, M. P. Hollier, and A. P. Hekstra, “Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs,” in 2001 IEEE international conference on acoustics, speech, and signal processing. Proceedings (Cat. No. 01CH37221) , vol. 2. IEEE, 2001, pp. 749–752

    2001

  33. [33]

    Scaling speech technology to 1,000+ languages,

    V . Pratap, A. Tjandra, B. Shi, P. Tomasello, A. Babu, S. Kundu, A. Elkahky, Z. Ni, A. Vyas, M. Fazel-Zarandi et al. , “Scaling speech technology to 1,000+ languages,” Journal of Machine Learning Research, vol. 25, no. 97, pp. 1–52, 2024

    work page 2024

  34. [34]

    ECAPA2: A Hybrid Neural Network Architecture and Training Strategy for Robust Speaker Embeddings

    J. Thienpondt and K. Demuynck, “Ecapa2: A hybrid neural net- work architecture and training strategy for robust speaker embed- dings,” arXiv preprint arXiv:2401.08342, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  35. [35]

    Roach, A little encyclopaedia of phonetics

    P. Roach, A little encyclopaedia of phonetics . Verlag nicht er- mittelbar, 2002

    work page 2002

  36. [36]

    Libritts: A corpus derived from librispeech for text- to-speech,

    H. Zen, V . Dang, R. Clark, Y . Zhang, R. J. Weiss, Y . Jia, Z. Chen, and Y . Wu, “Libritts: A corpus derived from librispeech for text- to-speech,” Interspeech 2019, 2019

    work page 2019

  37. [37]

    Hi-Fi Multi-Speaker English TTS Dataset,

    E. Bakhturina, V . Lavrukhin, B. Ginsburg, and Y . Zhang, “Hi-Fi Multi-Speaker English TTS Dataset,” in Proc. Interspeech 2021, 2021, pp. 2776–2780

    work page 2021

  38. [38]

    Mls: A large-scale multilingual dataset for speech research,

    V . Pratap, Q. Xu, A. Sriram, G. Synnaeve, and R. Collobert, “Mls: A large-scale multilingual dataset for speech research,” In- terspeech 2020, 2020

    work page 2020

  39. [39]

    Efficient sequence transduction by jointly predicting tokens and durations,

    H. Xu, F. Jia, S. Majumdar, H. Huang, S. Watanabe, and B. Gins- burg, “Efficient sequence transduction by jointly predicting tokens and durations,” in International Conference on Machine Learn- ing. PMLR, 2023

    work page 2023

  40. [40]

    Titanet: Neural model for speaker representation with 1d depth-wise separable convo- lutions and global context,

    N. R. Koluguri, T. Park, and B. Ginsburg, “Titanet: Neural model for speaker representation with 1d depth-wise separable convo- lutions and global context,” in ICASSP 2022-2022 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 8102–8106

    work page 2022

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.