Pith. sign in

REVIEW 4 major objections 4 minor 41 references

DiffSoundStream: Efficient Speech Tokenization via Diffusion Decoding

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DiffSoundStream claims half-rate speech tokenization with full-quality waveforms via diffusion decoding.

desk verdict A genuinely useful codec-plus-diffusion system for halving speech token rate, but the abstract overstates the evidence by swapping in a stronger subjective baseline. read the letter →

arxiv 2506.22362 v1 pith:MI4EFQ7A submitted 2025-06-27 eess.AS cs.LG

Yang Yang , Yunpeng Li , George Sung , Shao-Fu Shih , Craig Dooley , Alessio Centazzo , Ramanan Rajeswaran This is my paper
classification eess.AScs.LG
keywords speechtokenizationneuralcodeclatentdiffusionmodelsemantictokensacousticSoundStreamtokenratereductionstepdistillation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a speech tokenizer can halve the token rate without losing perceived quality if the codec is conditioned on semantic tokens and the deterministic decoder is replaced by a latent diffusion decoder. At 50 tokens per second, DiffSoundStream matches a standard SoundStream model operating at 100 tokens per second, and the four-step distilled version keeps that quality nearly intact. These results matter because autoregressive speech models are limited by token rate: halving the rate means each token covers twice as much audio, lowering compute and extending the context an AR model can use. The paper also finds that semantic conditioning alone improves intelligibility, and that the diffusion decoder contributes the most when acoustic tokens are scarce.

What carries the argument

The machinery is a three-part pipeline. SS-SC is a SoundStream autoencoder whose encoder and decoder are conditioned on WavLM semantic tokens via Feature-wise Linear Modulation, so the acoustic tokens capture only complementary information; it runs at 12.5 frames per second with eight residual vector quantizers of 2048 entries each. SS-CL is a SoundStream variant with a continuous latent at 50 Hz, perturbed by Gaussian noise and regularized by range clipping instead of a KL term, which defines the space in which diffusion operates. A WaveNet-based latent diffuser, trained with v-parameterization and a cosine noise schedule, takes the semantic and coarse acoustic token embeddings, pools and upsamples them to the 50 Hz frame rate, and injects them as local conditioning while the diffusion timestep is global conditioning. DDPM sampling generates the latent, and step-size distillation by moment matching reduces sampling from 100 iterations to four with minor quality loss.

What would settle it

A falsifying experiment would compare 50 token/s DiffSoundStream against a 100 token/s SoundStream baseline on a broader battery, such as MUSHRA with more than ten clips spanning noisy and accented speech or WER on LibriSpeech test-other; a statistically reliable quality gap would contradict the on-par claim.

Watch

Extended reading notes

Core claim

The central claim is that hybrid semantic-plus-acoustic tokenization followed by generative decoding reaches a quality that standard codec tokenization needs twice the token rate to match. In DiffSoundStream, WavLM features are quantized into 2048 semantic tokens at 12.5 Hz, and those semantic tokens condition both the encoder and decoder of a SoundStream autoencoder through FiLM layers, so the acoustic tokens carry only information missing from the semantics. A separate continuous-latent SoundStream runs at 50 Hz, and a WaveNet-based latent diffusion model generates that continuous latent conditioned on the semantic and coarse acoustic tokens; the SS-CL decoder then produces the 24 kHz waveform. Evaluated with WER on LibriTTS test-clean, DNSMOS, and a ten-clip MUSHRA, the 50 token/s configuration (one semantic plus three acoustic tokens per frame) is on par with SS-SC at 100 token/s, and the four-step distilled version has near-identical WER with a slightly lower DNSMOS. The authors interpret this as evidence that semantic tokens absorb much of the load that would otherwise be carried by acoustic tokens, and that diffusion sampling supplies the acoustic detail a deterministic GAN decoder would lose.

Load-bearing premise

The load-bearing premise is that the 2048 semantic-token centroids fitted on the held-out data generalize to unseen speech, and that the ten-clip MUSHRA on LibriTTS test-clean is representative enough to establish 'on par' quality.

Editorial extensions

If this is right

  • Autoregressive speech language models can process twice as much audio per token, because 50 token/s is sufficient for quality that previously required 100 token/s.
  • Token-based systems can extend their effective context window for a fixed token budget, which helps long-form generation and dialogue.
  • Diffusion decoding can replace GAN-based decoding at low bitrates, letting generative sampling fill in acoustic details instead of sacrificing fidelity.
  • The four-step distilled version makes the approach practical for deployment, since the number of diffusion sampling iterations drops from 100 to 4 with almost no loss.
  • The token-rate reduction is orthogonal to hybrid autoregressive/non-autoregressive decoding schemes, so it can be combined with those efficiency methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the same conditioning idea could be carried into causal and streaming tokenizers by distilling WavLM semantic information into a causal encoder, a direction the paper lists as future work.
  • Inference: if DNSMOS stays flat across token depths as reported, diffusion decoding might support even lower token rates than 50 token/s; the paper's evaluation stops at 50, so that remains untested.
  • Inference: the on-par claim is established on a single clean-speech subset with a ten-clip listening test, so a broader evaluation across noise, accents, and languages is the natural next check before building on the result.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. DiffSoundStream is a speech tokenization model that pairs WavLM semantic tokens with a semantic-conditioned SoundStream encoder and uses a latent diffusion decoder to synthesize waveforms from the semantic and coarse acoustic tokens. A continuous-latent SoundStream provides the diffusion target, and moment-matching distillation reduces sampling to four steps. The authors evaluate token-rate trade-offs on LibriTTS test-clean via WER, DNSMOS, and a MUSHRA listening test, and claim that at 50 tokens/s their model matches a standard SoundStream at 100 tokens/s.

Significance. If fully supported, the result is practically significant: it implies that token-based speech models can halve the autoregressive token rate without a quality penalty. The paper is clearly structured, reports an ablation without semantic tokens, and directly compares the distilled four-step model with the teacher, which are valuable strengths. Its main weakness is that the subjective evaluation does not include the baseline named in the headline claim, so the central statement currently outruns the evidence.

major comments (4)
  1. [Abstract; Section 3, Figure 6] The abstract's claim that DiffSoundStream at 50 tokens/s is on par with a standard SoundStream model at 100 tokens/s is not directly tested: the MUSHRA reference at 100 tokens/s is SS-SC, not SS-Baseline, and SS-Baseline appears in the MUSHRA only at 50 tokens/s. Since SS-SC is reported in Figures 4 and 5 to be better than SS-Baseline, testing against SS-SC does not test against the baseline named in the headline. Please add SS-Baseline at 100 tokens/s to the listening test or revise the abstract to match the evidence.
  2. [Section 3, Figures 4 and 5] The objective quality conclusions are drawn from DNSMOS values on a 3.90-4.00 scale with no confidence intervals or significance tests. In particular, the statements that the diffusion decoder 'further boosts' perceptual quality and that distillation yields 'slightly degraded DNSMOS' need uncertainty estimates to rule out evaluation noise, especially because the subjective test covers only ten clips.
  3. [Section 2.1, Section 3] The WavLM k-means semantic codebook is fit on an unspecified held-out dataset, while the evaluation is performed on LibriTTS test-clean. The token-rate advantage depends on these semantic tokens transferring beyond the fitting corpus; please specify the dataset used to fit the centroids and, ideally, add a robustness check on at least one additional corpus.
  4. [Section 3, Figure 6] The MUSHRA reporting is incomplete: score differences relative to clean audio are shown, but the number of listeners, anchor conditions, and statistical reliability of the comparisons are not stated. The central claim should not rest on 'it is evident' from ten clips without these details being reported.
minor comments (4)
  1. [Index Terms] The third index term contains a typo: 'nerual' should be 'neural'.
  2. [Section 2.3] The text says 'SS-SL decoder' where it should say 'SS-SC decoder'.
  3. [Figure 6] The legend contains two labels beginning with '(a)'; the 50 tokens/s SS-SC condition should be relabeled so that each row has a distinct identifier.
  4. [Section 2.2.1] Clarify whether the range-clipped latent is normalized to unit standard deviation before being used as the diffusion target, or whether the diffusion operates directly on the clipped representation; the text currently states both conditions without specifying the order.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims are empirical comparisons of trained systems, not derivations from fitted parameters or self-citations.

full rationale

DiffSoundStream's headline result—that 50 tokens/second achieves quality on par with a standard SoundStream at twice the token rate—is supported by measured WER, DNSMOS, and MUSHRA evaluations on LibriTTS test-clean. The paper does not derive this claim from a fitted parameter renamed as a prediction; the quality metrics are external observations of trained models. The k-means centroids used for WavLM semantic tokens are fit on a held-out dataset, and the evaluation set is not used to fit any component. The diffusion model is trained with a standard v-parameterization objective, and conditioning via FiLM or local/global conditioning does not reintroduce the target evaluation outcome by construction. There are no load-bearing self-citations or imported uniqueness theorems; the cited baselines (SoundStream, WavLM, HuBERT, etc.) are external prior work. The mismatch between the abstract's phrase 'standard SoundStream' and the MUSHRA comparison against SS-SC at 100 tokens/second is an evidence/claim gap about which baseline was subjectively tested, but it is not circularity because the comparison is still an empirical measurement rather than a construction that forces the result. Overall, the derivation chain is self-contained: the proposed method is evaluated, not derived from its own claims.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new physical or metaphysically independent entities are introduced. The free parameters are design choices that affect the central claim, while the axioms are unverified assumptions about feature quality, latent suitability, evaluation sufficiency, and data representativeness.

free parameters (4)
  • Semantic token centroids = 2048 centroids from k-means
    Chosen to quantize WavLM features at 12.5 Hz; the held-out dataset used to fit centroids is unspecified.
  • Latent noise std factor = 0.2
    Noise standard deviation as fraction of empirical latent std, set by hand in Section 2.2.1.
  • Latent range = [-1, 1]
    Range clipping for latent regularization, chosen empirically to Gaussianize marginal distributions.
  • Diffusion sampling steps = 100 (teacher), 4 (student)
    Baseline and distilled step counts selected for quality/speed trade-off.
assumptions (4)
  • domain assumption WavLM features, average-pooled with stride 4, retain sufficient semantic content for reconstruction.
    Section 2.1: semantic tokens are derived this way; the paper does not ablate the pooling strategy.
  • ad hoc to paper Range-clipped continuous latents are an appropriate diffusion target.
    Section 2.2.1: range clipping 'works well enough' without KL regularization.
  • domain assumption MUSHRA with 10 clips on LibriTTS test-clean is sufficient for the quality comparison.
    Figure 6 caption: 10 audio clips; no power analysis.
  • domain assumption The private 10k-hour English conversational dataset is representative of general speech.
    Section 3: training data described only as private.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DiffSoundStream: Efficient Speech Tokenization via Diffusion Decoding." pith.science (2026). https://pith.science/paper/MI4EFQ7A

@misc{pith2026250622362,
  author       = {Pith},
  title        = {Pith review of: DiffSoundStream: Efficient Speech Tokenization via Diffusion Decoding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MI4EFQ7A}},
  note         = {Machine review of arXiv:2506.22362}
}
read the original abstract

Token-based language modeling is a prominent approach for speech generation, where tokens are obtained by quantizing features from self-supervised learning (SSL) models and extracting codes from neural speech codecs, generally referred to as semantic tokens and acoustic tokens. These tokens are often modeled autoregressively, with the inference speed being constrained by the token rate. In this work, we propose DiffSoundStream, a solution that improves the efficiency of speech tokenization in non-streaming scenarios through two techniques: (1) conditioning the neural codec on semantic tokens to minimize redundancy between semantic and acoustic tokens, and (2) leveraging latent diffusion models to synthesize high-quality waveforms from semantic and coarse-level acoustic tokens. Experiments show that at 50 tokens per second, DiffSoundStream achieves speech quality on par with a standard SoundStream model operating at twice the token rate. Additionally, we achieve step-size distillation using just four diffusion sampling steps with only a minor quality loss.

Figures

Figures reproduced from arXiv: 2506.22362 by the authors.

Figure 1
Figure 1. Overview of SS-SC (top) and SS-CL (bottom). block. It first goes through a 256-dimensional embedding layer to convert to continuous features, and then followed by a trans￾posed convolution to align with the temporal and depth dimen￾sions of the corresponding activation that FiLM conditioning is applied upon. 2.2. Latent diffusion model Diffusion model has emerged as a strong generative solution across many domains. … view at source ↗
Figure 3
Figure 3. Token to waveform decoding. 8 RVQ acoustic tokens per frame. In total, each frame con￾tains 9 tokens at a frame rate of 12.5Hz, translating to a max￾imum token rate of 112.5 tokens/second (9 tokens/frame × 12.5 frames/sec) with a bitrate of 1.2375 kbps (9 tokens × 11 bits/token × 12.5 frames/sec). At decoding time, Ns semantic tokens (0 or 1) and Na acoustic tokens (1-8) are used in the diffusion decoder. Decoding T… view at source ↗
Figure 4
Figure 4. Word-error-rate on LibriTTS-test-clean comparing the baseline SoundStream (SS-Baseline), semantic-conditioned SoundStream (SS-SC), and DiffSoundStream (Diff-SS) as a function of tokens/frame at 12.5Hz frame rate. 2 3 4 5 6 7 8 Depth (Number of tokens per frame) 3.90 3.92 3.94 3.96 3.98 4.00 DNSMOS (a) SS-SC (b) SS-baseline (c) Diff-SS (d) Diff-SS (Distilled 4 step) [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: DNSMOS study. We evaluate our solution on Libri-TTS [32] test-clean sub￾set by examining the intelligibility and perceptual quality of the reconstructed speech after tokenization and decoding. Specif￾ically, we measure the word-error-rate (WER) of the recon￾structed sp…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 32 canonical work pages

  1. [1]

    Introduction Recent advances in self-supervised learning [1, 2, 3, 4] and neu- ral codecs [5, 6] have allowed the transformation of speech signals into sequences of discrete tokens, effectively refram- ing speech modeling as a token-based language modeling prob- lem. This paradigm shift has led to advances in a wide range of speech modeling applications s...

    work page Pith review arXiv 2025

  2. [2]

    Modeling Our proposed DiffSoundStream model comprises three compo- nents, which we detail in the subsequent subsections. • Semantic-conditioned SoundStream (SS-SC): A modified SoundStream autoencoder [6] where both the encoder and decoder are conditioned on semantic tokens derived from a pretrained WavLM model [3]. • Continuous-latent SoundStream (SS-CL):...

    work page

  3. [3]

    SS-SC and SS-CL are optimized with 1e−4 learn- ing rate for 1 million steps

    Experiments The three components SS-SC, SS-CL, and the diffusion model are all trained with a private dataset containing more than 10k hours of English conversational speech using the Adam opti- mizer [31]. SS-SC and SS-CL are optimized with 1e−4 learn- ing rate for 1 million steps. The diffusion model is trained for 4 million steps with a learning rate s...

    work page

  4. [4]

    There are two major limitations: it only supports non- 2The 10 audio clips are sampled uniformly from LibriTTS test-clean omitting those less than 5s

    Conclusions and limitations We propose DiffSoundStream, a speech tokenization solution that derives acoustic tokens conditioned on WavLM semantic tokens and utilizes latent diffusion for efficient speech decod- ing. There are two major limitations: it only supports non- 2The 10 audio clips are sampled uniformly from LibriTTS test-clean omitting those less...

    work page

  5. [5]

    High Fidelity Neural Audio Compression,

    A. D ´efossez, J. Copet, G. Synnaeve, and Y . Adi, “High Fidelity Neural Audio Compression,” 2022. [Online]. Available: https://arxiv.org/abs/2210.13438

    arXiv 2022

  6. [6]

    variant that replaces the discrete residual vector quantiza- tion (RVQ) bottleneck with a continuous latent space, trained using noise augmentation and regularization following the V AE formulation [23]. • SoundStream latent diffuser: A diffusion model trained to model the continuous latent distribution from SS-CL, while conditioned on both the semantic t...

    work page

  7. [7]

    HuBERT: Self-Supervised Speech Rep- resentation Learning by Masked Prediction of Hidden Units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdi- nov, and A. Mohamed, “HuBERT: Self-Supervised Speech Rep- resentation Learning by Masked Prediction of Hidden Units,” IEEE/ACM Transactions on Audio, Speech, and Language Pro- cessing, vol. 29, pp. 3451–3460, 2021

    work page 2021

  8. [8]

    wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Represen- tations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Represen- tations,” in NeurIPS 2020

    work page 2020

Show all 41 references
  1. [9]

    Wavlm: Large-scale self-supervised pre-training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L. Zhou, S. Ren, Y . Qian, Y . Qian, J. Wu, M. Zeng, X. Yu, and F. Wei, “Wavlm: Large-scale self-supervised pre-training for full stack speech processing,” IEEE Journal of Selecte...

    2022

  2. [10]

    w2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre- Training,

    Y .-A. Chung, Y . Zhang, W. Han, C.-C. Chiu, J. Qin, R. Pang, and Y . Wu, “w2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre- Training,” in ASRU 2021

    2021

  3. [11]

    PolyV oice: Language Models for Speech to Speech Translation,

    Q. qian Dong, Z. Huang, Q. Tian, C. Xu, T. Ko, yunlong zhao, S. Feng, T. Li, K. Wang, X. Cheng, F. Yue, Y . Bai, X. Chen, L. Lu, Z. MA, Y . Wang, M. Wang, and Y . Wang, “PolyV oice: Language Models for Speech to Speech Translation,” in ICLR 2024

    2024

  4. [12]

    Soundstream: An end-to-end neural audio codec,

    N. Zeghidour, A. Luebs, A. Omran, J. Skoglund, and M. Tagliasacchi, “Soundstream: An end-to-end neural audio codec,” IEEE/ACM Transactions on Audio, Speech, and Lan- guage Processing, vol. 30, pp. 495–507, 2022

    2022

  5. [13]

    AudioLM: A Language Modeling Approach to Audio Generation,

    Z. Borsos, R. Marinier, D. Vincent, E. Kharitonov, O. Pietquin, M. Sharifi, D. Roblek, O. Teboul, D. Grangier, M. Tagliasacchi, and N. Zeghidour, “AudioLM: A Language Modeling Approach to Audio Generation,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 3...

    2023

  6. [14]

    Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervision,

    E. Kharitonov, D. Vincent, Z. Borsos, R. Marinier, S. Girgin, O. Pietquin, M. Sharifi, M. Tagliasacchi, and N. Zeghidour, “Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervision,” 2023. [Online]. Available: https: //arxiv.org/abs/2302.03540

    2023 arXiv

  7. [15]

    Neural codec language models are zero-shot text to speech synthesizers,

    C. Wang, S. Chen, Y . Wu, Z. Zhang, L. Zhou, S. Liu, Z. Chen, Y . Liu, H. Wang, J. Li, L. He, S. Zhao, and F. Wei, “Neural codec language models are zero-shot text to speech synthesizers,” 2023. [Online]. Available: https://arxiv.org/abs/2301.02111

    2023 arXiv

  8. [16]

    TokenSplit: Using Discrete Speech Representations for Direct, Refined, and Transcript- Conditioned Speech Separation and Recognition,

    H. Erdogan, S. Wisdom, X. Chang, Z. Borsos, M. Tagliasacchi, N. Zeghidour, and J. Hershey, “TokenSplit: Using Discrete Speech Representations for Direct, Refined, and Transcript- Conditioned Speech Separation and Recognition,” in Interspeech 2023, 2023. [Online]. Available: ht...

    2023

  9. [17]

    Denoising Diffusion Probabilistic Models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising Diffusion Probabilistic Models,” in NeurIPS 2020

    2020

  10. [18]

    High-Fidelity Simultaneous Speech-To-Speech Translation,

    T. Labiausse, L. Mazar ´e, E. Grave, P. P ´erez, A. D ´efossez, and N. Zeghidour, “High-Fidelity Simultaneous Speech-To-Speech Translation,” 2025. [Online]. Available: https://arxiv.org/abs/ 2502.03382

    2025 arXiv

  11. [19]

    Moshi: a speech- text foundation model for real-time dialogue,

    A. D ´efossez, L. Mazar ´e, M. Orsini, A. Royer, P. P ´erez, H. J ´egou, E. Grave, and N. Zeghidour, “Moshi: a speech- text foundation model for real-time dialogue,” 2024. [Online]. Available: https://arxiv.org/abs/2410.00037

    2024 arXiv

  12. [20]

    Autoregressive Image Generation Using Residual Quantization,

    D. Lee, C. Kim, S. Kim, M. Cho, and W.-S. Han, “Autoregressive Image Generation Using Residual Quantization,” in CVPR 2022

    2022

  13. [21]

    HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis,

    J. Kong, J. Kim, and J. Bae, “HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis,” in NeurIPS 2020

    2020

  14. [22]

    Mel- GAN: Generative Adversarial Networks for Conditional Wave- form Synthesis,

    K. Kumar, R. Kumar, T. de Boissiere, L. Gestin, W. Z. Teoh, J. Sotelo, A. de Br´ebisson, Y . Bengio, and A. C. Courville, “Mel- GAN: Generative Adversarial Networks for Conditional Wave- form Synthesis,” in NeurIPS 2019

    2019

  15. [23]

    Auto-Encoding Variational Bayes,

    D. P. Kingma and M. Welling, “Auto-Encoding Variational Bayes,” in ICLR 2014

    2014

  16. [24]

    Deep Unsupervised Learning using Nonequilibrium Thermody- namics,

    J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep Unsupervised Learning using Nonequilibrium Thermody- namics,” in ICML 2015

    2015

  17. [25]

    to replace the GAN-trained deterministic SoundStream de- coder to synthesize high-quality waveforms with reduced to- ken rate. Specifically, we adopt the latent diffusion framework and rely on a continuous-latent SoundStream (SS-CL) model, shown in the lower half of Figure 1, ...

  18. [26]

    Multi- step Distillation of Diffusion Models via Moment Matching,

    T. Salimans, T. Mensink, J. Heek, and E. Hoogeboom, “Multi- step Distillation of Diffusion Models via Moment Matching,” in NeurIPS 2024

    2024

  19. [27]

    SpeechTok- enizer: Unified Speech Tokenizer for Speech Language Models,

    X. Zhang, D. Zhang, S. Li, Y . Zhou, and X. Qiu, “SpeechTok- enizer: Unified Speech Tokenizer for Speech Language Models,” in ICLR 2024

    2024

  20. [28]

    Simple and Controllable Music Gen- eration,

    J. Copet, F. Kreuk, I. Gat, T. Remez, D. Kant, G. Synnaeve, Y . Adi, and A. D´efossez, “Simple and Controllable Music Gen- eration,” in NeurIPS 2023

    2023

  21. [29]

    SoundStorm: Efficient Parallel Audio Generation,

    Z. Borsos, M. Sharifi, D. Vincent, E. Kharitonov, N. Zeghidour, and M. Tagliasacchi, “SoundStorm: Efficient Parallel Audio Generation,” 2023. [Online]. Available: https://arxiv.org/abs/ 2305.09636

    2023 arXiv

  22. [30]

    FiLM: Visual Reasoning with a General Conditioning Layer,

    E. Perez, F. Strub, H. de Vries, V . Dumoulin, and A. Courville, “FiLM: Visual Reasoning with a General Conditioning Layer,” AAAI 2018 , vol. 32, no. 1. [Online]. Available: https: //ojs.aaai.org/index.php/AAAI/article/view/11671

    2018

  23. [31]

    High-Resolution Image Synthesis With Latent Diffusion Mod- els,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-Resolution Image Synthesis With Latent Diffusion Mod- els,” in CVPR 2022

    2022

  24. [32]

    Improved Denoising Diffusion Probabilistic Models,

    A. Q. Nichol and P. Dhariwal, “Improved Denoising Diffusion Probabilistic Models,” in ICLR 2021

    2021

  25. [33]

    Progressive Distillation for Fast Sampling of Diffusion Models,

    T. Salimans and J. Ho, “Progressive Distillation for Fast Sampling of Diffusion Models,” in ICLR 2022

    2022

  26. [34]

    Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation,

    D. Kingma and R. Gao, “Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation,” in NeurIPS 2023

    2023

  27. [35]

    WaveNet: A Generative Model for Raw Audio,

    A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, “WaveNet: A Generative Model for Raw Audio,” in Arxiv, 2016. [Online]. Available: https://arxiv.org/abs/1609.03499

    2016 arXiv

  28. [36]

    Improved Distribution Matching Distillation for Fast Image Synthesis,

    T. Yin, M. Gharbi, T. Park, R. Zhang, E. Shechtman, F. Durand, and W. T. Freeman, “Improved Distribution Matching Distillation for Fast Image Synthesis,” in NeurIPS 2024

    2024

  29. [37]

    Adam: A Method for Stochastic Opti- mization,

    D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Opti- mization,” in ICLR 2015

    2015

  30. [38]

    LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech,

    H. Zen, V . Dang, R. Clark, Y . Zhang, R. J. Weiss, Y . Jia, Z. Chen, and Y . Wu, “LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech,”INTERSPEECH 2019

    2019

  31. [39]

    DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors,

    C. K. A. Reddy, V . Gopal, and R. Cutler, “DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors,” in ICASSP 2022

    2022

  32. [40]

    Method for the subjective assessment of intermediate quality level of audio systems,

    B. Series, “Method for the subjective assessment of intermediate quality level of audio systems,” International Telecommunication Union Radiocommunication Assembly, vol. 2, 2014

    2014

  33. [41]

    From Slow Bidirectional to Fast Autoregressive Video Diffusion Models,

    T. Yin, Q. Zhang, R. Zhang, W. T. Freeman, F. Durand, E. Shechtman, and X. Huang, “From Slow Bidirectional to Fast Autoregressive Video Diffusion Models,” 2025. [Online]. Available: https://arxiv.org/abs/2412.07772

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.