Pith. sign in

REVIEW 31 cited by

SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models

Not yet reviewed by Pith; the record is open.

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2308.16692 v2 pith:PMTOIQTM submitted 2023-08-31 cs.CL cs.SDeess.AS

SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models

classification cs.CL cs.SDeess.AS
keywords speechspeechtokenizerlanguagetokensmodelsacousticlargesemantic
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Current speech large language models build upon discrete speech representations, which can be categorized into semantic tokens and acoustic tokens. However, existing speech tokens are not specifically designed for speech language modeling. To assess the suitability of speech tokens for building speech language models, we established the first benchmark, SLMTokBench. Our results indicate that neither semantic nor acoustic tokens are ideal for this purpose. Therefore, we propose SpeechTokenizer, a unified speech tokenizer for speech large language models. SpeechTokenizer adopts the Encoder-Decoder architecture with residual vector quantization (RVQ). Unifying semantic and acoustic tokens, SpeechTokenizer disentangles different aspects of speech information hierarchically across different RVQ layers. Furthermore, We construct a Unified Speech Language Model (USLM) leveraging SpeechTokenizer. Experiments show that SpeechTokenizer performs comparably to EnCodec in speech reconstruction and demonstrates strong performance on the SLMTokBench benchmark. Also, USLM outperforms VALL-E in zero-shot Text-to-Speech tasks. Code and models are available at https://github.com/ZhangXInFD/SpeechTokenizer/.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 31 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. FlexiSLM: A Dynamic and Controllable Frame Rate Spoken Language Model

    cs.SD 2026-06 unverdicted novelty 7.0

    FlexiSLM is the first spoken language model supporting dynamic and controllable frame rates on speech input and output, outperforming fixed-rate 7B models at high quality and enabling faster inference at lower rates l...

  2. Codec-Robust Attacks on Audio LLMs

    cs.SD 2026-05 unverdicted novelty 7.0

    CodecAttack perturbs audio in codec latent space with multi-bitrate EoT to achieve 85.5% average ASR on Opus-compressed Audio LLMs versus under 26% for waveform baselines, with transfer to MP3 and AAC.

  3. PairAlign: A Framework for Sequence Tokenization via Self-Alignment with Applications to Audio Tokenization

    cs.LG 2026-05 unverdicted novelty 7.0

    PairAlign learns compact variable-length token sequences for audio via self-alignment on paired content-preserving views, achieving 55% fewer archive tokens than VQ while preserving edit-distance retrieval at 12.71 tokens/s.

  4. PairAlign: A Framework for Sequence Tokenization via Self-Alignment with Applications to Audio Tokenization

    cs.LG 2026-05 unverdicted novelty 7.0

    PairAlign learns compact audio token sequences via self-alignment of paired content views using an autoregressive decoder, achieving strong cross-view consistency and edit-distance preservation while reducing token co...

  5. PolySLGen: Online Multimodal Speaking-Listening Reaction Generation in Polyadic Interaction

    cs.CV 2026-04 unverdicted novelty 7.0

    PolySLGen generates contextually appropriate and temporally coherent multimodal speaking and listening reactions for polyadic interactions by fusing group motion and social cues.

  6. TokenChain: A Discrete Speech Chain via Semantic Token Modeling

    eess.AS 2025-10 unverdicted novelty 7.0

    TokenChain demonstrates that a discrete semantic-token interface can sustain effective chain learning between ASR and TTS, yielding faster convergence and lower error rates on LibriSpeech and TED-LIUM.

  7. OmniVAE: An Audio-Video VAE with Cross-Modal Alignment for Joint Generation

    cs.SD 2026-07 conditional novelty 6.0

    Jointly training audio and video VAEs with segment contrastive loss and semantic distillation yields more learnable, cross-aligned latents that improve downstream joint generation quality and sync.

  8. OmniVAE: An Audio-Video VAE with Cross-Modal Alignment for Joint Generation

    cs.SD 2026-07 conditional novelty 6.0

    A jointly trained audio-video VAE with segment-level contrastive alignment and semantic distillation improves downstream text-to-audio-video synchronization and quality.

  9. Investigating Codec-Internal Latent Audio Watermarking for Neural Codec Robustness

    cs.SD 2026-07 conditional novelty 6.0

    Embedding watermarks inside a codec-like autoencoder's continuous latent space improves EnCodec-24k bit accuracy to ~95–97%, but the gain is in-distribution and does not transfer to EnCodec-16k.

  10. HARP: Harmonic-Aware Residual Partitioning for Neural Audio Codecs

    cs.SD 2026-07 conditional novelty 6.0

    A training-only change to residual vector quantization orders codebook stages from bass to treble, yielding better audio quality and more predictable bitrate scaling with identical inference.

  11. AuEmoChat: Authentic Emotion Understanding and Rendering for Conversational Speech Synthesis

    cs.SD 2026-07 conditional novelty 6.0

    AuEmoChat improves conversational speech synthesis by learning a discrete 1000-code authentic emotion token space, merging redundant dialogue context, and using emotion-guided flow matching to render speech.

  12. AuEmoChat: Authentic Emotion Understanding and Rendering for Conversational Speech Synthesis

    cs.SD 2026-07 conditional novelty 6.0

    AuEmoChat's learned 1,000-code emotion token space, combined with emotion-guided token merging and classifier-guided flow matching, yields higher naturalness and emotion scores than four CSS baselines on NCSSD-EmCap.

  13. SPEARBench: A Benchmark for Naturalness Evaluation in Streaming Speech-to-Speech Language Models

    cs.CL 2026-07 conditional novelty 6.0

    An open-source benchmark for speech-to-speech models shows that current systems produce intelligible audio but diverge from human conversational behavior in latency, dialect consistency, emotional entrainment, and prosody.

  14. Phonemes to the Rescue: Multilingual Tokenization Based on International Phonetic Alphabet

    cs.CL 2026-06 unverdicted novelty 6.0

    IPA-based subword tokenizers trained across 24 languages improve tokenization quality and generalization to unseen languages compared to standard text tokenizers, especially for non-Latin scripts.

  15. Self-Guidance: Enhancing Neural Codecs via Decoder Manifold Alignment

    cs.SD 2026-06 unverdicted novelty 6.0

    Self-guidance adds a lightweight feature-mapping loss to align decoder manifolds in VQ-VAE speech codecs, raising reconstruction metrics and allowing 4x codebook reduction with no fidelity loss.

  16. Codec-Robust Attacks on Audio LLMs

    cs.SD 2026-05 unverdicted novelty 6.0

    CodecAttack optimizes perturbations in neural audio codec latent space to reach 85.5% average target-substring ASR on compressed Opus audio while waveform baselines stay below 26%.

  17. The Equalizer: Introducing Shape-Gain Decomposition in Neural Audio Codecs

    cs.SD 2026-02 conditional novelty 6.0

    Adding classical shape-gain decomposition to a neural audio codec makes it invariant to input gain and improves bitrate-distortion performance.

  18. Two-Dimensional Quantization for Geometry-Aware Audio Coding

    cs.SD 2025-12 unverdicted novelty 6.0

    Q2D2 uses 2D geometric grid projections to quantize feature pairs in neural audio codecs, yielding implicit codebooks that improve efficiency and utilization over RVQ, VQ, and FSQ while maintaining reconstruction quality.

  19. StableToken: A Noise-Robust Semantic Speech Tokenizer for Resilient SpeechLLMs

    cs.CL 2025-09 unverdicted novelty 6.0

    StableToken introduces a multi-branch architecture with bit-wise voting to create noise-robust semantic speech tokens, achieving lower Unit Edit Distance and better SpeechLLM robustness than prior single-path tokenizers.

  20. DeCodec: Rethinking Audio Codecs as Universal Disentangled Representation Learners

    cs.SD 2025-09 conditional novelty 6.0

    DeCodec learns a single neural codec that disentangles speech, background sound, semantic content, and paralinguistic style into orthogonal quantized streams, enabling reconstruction, enhancement, voice conversion, AS...

  21. AudioCodecBench: A Comprehensive Benchmark for Audio Codec Evaluation

    cs.SD 2025-09 conditional novelty 6.0

    A four-part benchmark plus a semantic/acoustic token taxonomy for comparing audio codecs, with correlation analysis across ten models.

  22. Entropy-based Coarse and Compressed Semantic Speech Representation Learning

    cs.CL 2025-08 conditional novelty 6.0

    Predictive entropy from a token-level speech language model finds merge boundaries, producing compressed semantic tokens that keep ASR and translation accuracy at 15 Hz while lowering latency.

  23. Step-Audio 2 Technical Report

    cs.CL 2025-07 unverdicted novelty 6.0

    Step-Audio 2 integrates a latent audio encoder, reasoning-centric reinforcement learning, and discrete audio token generation into language modeling to deliver state-of-the-art performance on audio understanding and c...

  24. Which Speech Representation Better Matches Text-Native Reasoning? A Study of Speech-Text Alignment on Frame Rate and Representation

    eess.AS 2026-06 unverdicted novelty 5.0

    Empirical sweep finds 4.17 Hz frame rate plus intermediate-layer alignment optimal for speech QA under frozen text LLM backbone.

  25. FireRedTTS-2: Towards Long Conversational Speech Generation for Podcast and Chatbot

    cs.SD 2025-09 conditional novelty 5.0

    FireRedTTS-2 generates long multi-speaker conversations in a streaming, sentence-by-sentence way using a new low-rate speech tokenizer and a dual-transformer text-speech model.

  26. CodecBench: A Comprehensive Benchmark for Acoustic and Semantic Evaluation

    eess.AS 2025-08 conditional novelty 5.0

    CodecBench ranks 14 audio codecs on acoustic fidelity and semantic preservation across 19 datasets and four audio domains, revealing a reconstruction-versus-semantics tradeoff.

  27. F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching

    eess.AS 2024-10 unverdicted novelty 5.0

    F5-TTS generates natural speech from text via flow matching on DiT with simple text padding, ConvNeXt refinement, and sway sampling, trained on 100K hours multilingual data.

  28. wav2tok 2.0: Scalable Audio Tokenization Maintaining Explicit Pairwise Token Alignment for Efficient Audio Retrieval

    cs.SD 2026-06 unverdicted novelty 4.0

    wav2tok 2.0 improves audio tokenization for query-by-example spoken term detection via staged training that first learns speaker-invariant representations then enforces pairwise token alignment.

  29. SARA: A Dual-Stream VAE for High-Fidelity Speech Generation via Integrating Semantic and Acoustic Representations

    cs.SD 2026-06 unverdicted novelty 4.0

    SARA is a dual-stream VAE that integrates semantic and acoustic streams to achieve high-fidelity reconstruction and natural zero-shot TTS without complex regularizers.

  30. MOSS-Audio Technical Report

    cs.SD 2026-06 unverdicted novelty 4.0

    MOSS-Audio is an audio-language model using a 12.5 Hz encoder, DeepStack cross-layer injection, time markers, and an event-preserving annotation pipeline for unified audio understanding.

  31. Speech Discrete Tokens or Continuous Features? A Comparative Analysis for Spoken Language Understanding in SpeechLLMs

    cs.CL 2025-08 unverdicted novelty 4.0

    Under matched settings, continuous SSL speech features generally outperform discrete tokens on six spoken language understanding tasks in SpeechLLMs.