Pith. sign in

REVIEW 3 major objections 4 minor 85 references

Speech enhancement can be recast as generating clean audio-code tokens from noisy ones with absorbing discrete diffusion.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 20:42 UTC pith:GUDH3T27

load-bearing objection Competent absorbing-discrete-diffusion SE paper with a useful RQDiT architecture, but the competitive claim depends on test-set-selected steps and non-intrusive metrics; worth reviewing with a demand for a fixed protocol. the 3 major comments →

arxiv 2602.22417 v2 pith:GUDH3T27 submitted 2026-02-25 cs.SD eess.AS

Absorbing Discrete Diffusion for Speech Enhancement

Philippe Gonzalez This is my paper
classification cs.SD eess.AS
keywords speech enhancementabsorbing discrete diffusionneural audio codecresidual vector quantizationnon-autoregressive generationdiffusion transformerlow-SNR speechmasked diffusion
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

ADDSE treats speech enhancement as a conditional generation problem in the discrete token space of a neural audio codec: given the codec codes of a noisy waveform, it models the distribution of the clean waveform's codes using absorbing discrete diffusion, in which tokens are progressively unmasked from a fully masked state. The authors propose a non-autoregressive Transformer architecture, RQDiT, that processes the hierarchical multi-codebook structure of residual vector quantization along both time and codebook-depth dimensions. They report that, measured by non-intrusive speech-quality metrics (DNSMOS, NISQA, UTMOS, SCOREQ), the approach is competitive with state-of-the-art waveform-domain systems on two held-out datasets, particularly at low signal-to-noise ratios and with as few as 8–16 sampling steps. A practical payoff is that network predictions can be reused across sampling steps, cutting the effective number of function evaluations by roughly half at 1024 steps. If the approach holds, convolutional and STFT-domain diffusion for enhancement can be replaced by cheap, parallel token-level generation that leverages advances in neural codecs and discrete diffusion.

Core claim

On the paper's own terms, the central discovery is that speech enhancement can be posed as a conditional discrete-diffusion problem over neural-audio-codec tokens: the model learns q_theta(c | c_lambda, c_tilde), the distribution of clean RVQ codes given partially masked clean codes and the noisy codes, optimized with the denoising cross-entropy (Eq. 10). The authors show that, with the non-autoregressive RQDiT architecture and a time-independent reparametrization of the score, sampling from the fully absorbed state produces enhanced speech that scores competitively with established waveform-domain baselines on non-intrusive metrics on two cross-corpus test sets, and that the advantage is la

What carries the argument

Two ingredients carry the argument. First, the absorbing state M — a mask token appended to each codebook — defines the forward process that gradually overwrites clean codes with M; the reverse process then unmask tokens by sampling from the learned conditional distribution, under a log-linear noise schedule that makes the transition probabilities simple and allows the model's predictions to be reused when no token was unmasked in the previous step. Second, RQDiT, a non-autoregressive architecture with a frame-DiT and a depth-DiT: the frame-DiT processes the sum of codebook embeddings across depths, the depth-DiT refines each frame independently along the codebook-depth dimension, and adapti

Load-bearing premise

The competitive claim rests on trusting non-intrusive metrics (DNSMOS, NISQA, UTMOS, SCOREQ) as faithful measures of enhancement quality, because the paper's intrusive metrics are far worse for ADDSE than for waveform-domain baselines, and on the assumption that selecting each system's best number of sampling steps on the test set does not inflate the comparison.

What would settle it

A MUSHRA listening test comparing ADDSE outputs against the noisy input and the codec-reconstructed clean signal would settle the central claim; if listeners do not prefer ADDSE over the noisy input, or if the reported non-intrusive gains disappear when Nsteps is chosen on a validation set rather than the test set, the competitive claim is falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Token-level non-autoregressive diffusion is a viable alternative to STFT-domain diffusion for enhancement, with near-peak quality at 8–16 sampling steps.
  • Because the codec is frozen and shared, the enhanced output is already a 2 kbps code sequence, so enhancement and compression are obtained in one pass.
  • The prediction-reuse trick cuts the number of network evaluations roughly in half at 1024 nominal steps, making high-step-count sampling cheaper than time-dependent models.
  • A 4 M-parameter ADDSE-XS already beats two established baseline systems on DNSMOS and NISQA on both test sets, indicating the framework is not compute-hungry at the low end.
  • Scaling model size from 4M to 580M parameters monotonically lowers the denoising cross-entropy and improves non-intrusive metrics, a clean scaling signal for further capacity increases.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the poor intrusive-metric scores (PESQ, ESTOI, SDR) likely reflect codec phase and reconstruction limitations rather than a failure of the diffusion sampler; a direct comparison of ADDSE against codec-reconstructed clean speech on those metrics would isolate the codec's contribution to the gap.
  • Editorial inference: the same masked-diffusion machinery should transfer to other token-sequence restoration tasks — bandwidth extension, declipping, or packet-loss concealment — since the conditional distribution is over generic RVQ tokens.
  • Editorial inference: selecting the best Nsteps on the test set may inflate the reported numbers; re-running the comparison with Nsteps chosen on a held-out validation set would test whether the favorable ranking persists, especially against the strongest waveform-domain baselines.
  • Editorial inference: the relationship between DCE and model size suggests DCE on a small validation set could serve as a cheap, sampling-free proxy for selecting model hyperparameters before spending compute on full reverse sampling.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes ADDSE, a speech enhancement method that models the conditional distribution of clean speech codec tokens given noisy speech codec tokens using absorbing discrete diffusion in the latent space of a neural audio codec (NAC). The authors introduce RQDiT, a non-autoregressive hierarchical architecture that processes RVQ codebooks along frame and depth dimensions, and train it with a denoising cross-entropy objective. Inference starts from a fully masked state and unmasks clean codes with a few sampling steps. The method is evaluated on two cross-corpus test sets (Libri-TUT and Clarity-FSD50K) against waveform-domain, STFT-domain, and latent-domain baselines. The paper reports that ADDSE achieves competitive non-intrusive metrics, especially at low SNRs and with few sampling steps, while its intrusive metrics (PESQ, ESTOI, SDR) are substantially lower than those of waveform-domain baselines.

Significance. If the empirical claims hold, the paper makes a useful contribution by adapting absorbing discrete diffusion to speech enhancement, a formulation that has not been applied to this task before. The RQDiT architecture is a sensible adaption of RQ-Transformer and diffusion Transformers to hierarchical RVQ codes, and the use of the NAC codebook entries as token embeddings is elegant. The authors provide reproducible code and audio examples, and the training objective is well grounded in the published discrete-diffusion literature. The central novelty is the combination of a NAC latent space with a time-independent absorbing discrete diffusion model, with the potential for efficient low-step inference. However, the evidence supporting the headline 'competitive performance' claim is weakened by methodological choices in the evaluation, specifically the test-set-based selection of the number of sampling steps and the reliance on non-intrusive metrics as the primary arbiters.

major comments (3)
  1. [5, Table 1 caption] The caption states 'For the ADDSE systems, the best Nsteps is selected.' Selecting Nsteps on the test set for each row means the reported numbers do not correspond to a single, fixed operating point of any ADDSE system. Figure 3 shows that the optimal Nsteps is metric-dependent: NISQA peaks at Nsteps=8, while DNSMOS, UTMOS, and SCOREQ generally improve or plateau through 16 and, for some systems, keep improving at 1024. Thus the DNSMOS and NISQA entries in Table 1 may come from different Nsteps values, so the table does not describe a runnable system and the 'few sampling steps' conclusion is not what was actually measured. The comparison should either fix Nsteps for all systems (or at least per dataset) based on a validation split, report the full Nsteps curves as primary evidence, or explicitly disclose the selected Nsteps and justify why test-set selection does not inflate the reporte
  2. [5, Table 1] All results are single-run averages with no confidence intervals, significance tests, or variability estimates. Statements such as 'all ADDSE systems, including the small 4 M-parameter ADDSE-XS, outperform Conv-TasNet and SGMSE+ on DNSMOS and NISQA' cannot be assessed for reliability without a measure of uncertainty. Given that the differences between some systems are small (e.g., DNSMOS differences of 0.02–0.05), the ranking could easily change across runs or test subsets. The authors should report confidence intervals (e.g., bootstrap over utterances) and, where claims of superiority are made, paired significance tests.
  3. [5, Table 1; Figure 3; Section 5] The central claim of competitive performance rests on non-intrusive metrics (DNSMOS, NISQA, UTMOS, SCOREQ), while the paper's own intrusive metrics (PESQ, ESTOI, SDR) are consistently much poorer for ADDSE than for waveform-domain baselines. The paper explains this by the NAC not reconstructing phase, but this does not by itself establish that the non-intrusive metrics are appropriate arbiters. In fact, ADDSE-XL exceeds the Clean-NAC upper bound on DNSMOS and NISQA on Libri-TUT (DNSMOS 3.76 vs 3.75; NISQA 3.43 vs 3.33), which suggests that these predictors can reward codec-like output regardless of enhancement fidelity. The manuscript should either provide evidence that the non-intrusive differences correspond to perceptually meaningful improvements (e.g., a listening test, or at least a correlation analysis with a reference-aware metric), or reframe the contribution as a codec-based gen
minor comments (4)
  1. [Eq. (7)] The notation '1/λ δ_{cλ M}' is ambiguous. The indicator should be written as a function of the observed masked code, e.g., 1_{cλ,i = M}, and the expected value should be over the forward process transition. Please clarify.
  2. [Sec. 2.2, Eq. (9)] The variable t is used both for the continuous time in Eq. (4) and for the scaled time in Eq. (9). This can confuse readers. Use a distinct symbol, e.g., τ, for the rescaled time.
  3. [Sec. 3.2] The description of RQDiT is clear overall, but it would help to state explicitly that the depth-DiT processes all depths for a given frame in parallel, and to specify the number of layers and heads for each DiT in the architecture figure or caption (the values 12 layers and 12 heads are only given in Sec. 4.3).
  4. [Sec. 5, Figure 3 caption] The caption says 'best NISQA is achieved at Nsteps = 8, while the remaining metrics plateau at Nsteps = 16', but the curves for DNSMOS and SCOREQ continue to rise beyond 16 for several ADDSE sizes. The text understates this, and the claim of 'few sampling steps' should be qualified accordingly.

Circularity Check

0 steps flagged

No circularity found; the ADDSE derivation is a conditional generative model trained with DCE against clean codes, with no fitted parameter renamed as a prediction.

full rationale

The paper's derivation chain is not circular. ADDSE models the conditional distribution of clean NAC codes given noisy codes via absorbing discrete diffusion, and the training objective in Eq. (10) is the denoising cross-entropy, which directly asks q_theta to predict clean codes from noisy codes and partially absorbed clean codes. The evaluation metrics (PESQ, DNSMOS, NISQA, etc.) are external to the training objective; no parameter is fitted to these metrics and then reported as a prediction. The ADD sampling rule in Eq. (9) is imported from prior independent work [39], and the architecture RQDiT combines existing RQ-Transformer and DiT components without defining its target in terms of its inputs. The few self-citations ([3, 69, 70]) are used only to support the EDM-SE baseline's design choices and are not load-bearing for the central claim. The main caveat in the paper is experimental rather than derivational: Table 1 states 'For the ADDSE systems, the best Nsteps is selected' on the test set, and Figure 3 shows metric-dependent optimal Nsteps, so the reported operating point is not unique. This weakens the empirical claim's fairness and reproducibility, but it is not a case of a prediction reducing to its inputs by construction. Similarly, ADDSE-XL exceeding the Clean-NAC upper bound on DNSMOS/NISQA suggests those non-intrusive predictors can reward codec-like output, but that is a measurement-interpretation issue, not circular reasoning. No specific circular step can be exhibited from the paper's equations or argumentation.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

No new physical entities are invented. Main uncharged inputs are hand-chosen codec and model hyperparameters plus the test-set-selected Nsteps. The ADD theory is imported from prior work.

free parameters (4)
  • Nsteps (number of sampling steps) = best selected per model/dataset on test set; chosen values not reported
    Table 1 reports each ADDSE system at its best Nsteps, a post-hoc fit to test metrics that inflates reported performance.
  • NAC design: D=4 RVQ codebooks, K=1024, bitrate 2 kbps, downsampling 320 = specified in Sec. 4.2
    The codec capacity is hand-chosen; all downstream ADDSE results depend on the reconstructed latent space.
  • RQDiT hidden dimension/layers/heads = dims 96/192/384/768/1152; 12 layers; 12 heads
    Model capacity sweep is chosen by hand; DCE and metrics improve with size, so this affects the reported scaling and best result.
  • NAC loss weights = 15, 1, 1, 1, 0.25
    Hand-set weights in Sec. 4.2; codec quality is an input to ADDSE.
axioms (5)
  • standard math Forward/reverse CTMC equations and the reverse transition formula (Eq. 9) from [34,36,39,42]
    The paper imports the absorbing-diffusion machinery without re-deriving it; the central sampling procedure rests on these results.
  • domain assumption Frozen NAC provides a faithful enough discrete representation of clean and noisy speech
    Sec. 3.1 assumes clean/noisy codes from one codec; the 'strong speech prior' claim in Sec. 5 depends on this.
  • domain assumption Non-intrusive metrics are the correct benchmark and intrusive metrics are deemphasized
    Sec. 5 reports poor PESQ/ESTOI/SDR for ADDSE, so the competitive claim rests entirely on DNSMOS/NISQA/UTMOS/SCOREQ.
  • ad hoc to paper Mask code embedding e(M;j)=0 is a valid input representation
    Introduced in Sec. 3.2; necessary for RQDiT to process masked tokens.
  • domain assumption Each (frame, depth) token is an independent diffusion dimension with a shared rate matrix
    Standard factorization in ADD; if RVQ depths are strongly dependent, the independent-process assumption may underfit, partially compensated by the depth-DiT.

pith-pipeline@v1.3.0-alltime-deepseek · 11474 in / 13222 out tokens · 123264 ms · 2026-08-02T20:42:07.613989+00:00 · methodology

0 comments
read the original abstract

Inspired by recent developments in neural speech coding and diffusion-based language modeling, we tackle speech enhancement by modeling the conditional distribution of clean speech codes given noisy speech codes using absorbing discrete diffusion. The proposed approach, which we call ADDSE, leverages both the expressive latent space of neural audio codecs and the non-autoregressive sampling procedure of diffusion models. To efficiently model the hierarchical structure of residual vector quantization codes, we propose RQDiT, which combines techniques from RQ-Transformer and diffusion Transformers for non-autoregressive modeling. Results show competitive performance in terms of non-intrusive objective metrics on two datasets, especially at low signal-to-noise ratios and with few sampling steps. Code and audio examples are available online.

Figures

Figures reproduced from arXiv: 2602.22417 by Philippe Gonzalez.

Figure 1
Figure 1. Figure 1: Proposed ADDSE framework. for SE. While some studies have applied MaskGIT [40], which resembles ADD, for SE [21–24], MaskGIT lacks a theoretically grounded formulation, and thus may not approximate a princi￾pled likelihood [38]. Moreover, these studies did not investigate the time-independent modeling for efficient sampling [39]. 2. Background 2.1. Neural audio codecs and residual vector quantization A typ… view at source ↗
Figure 1
Figure 1. Figure 1: During training, a frozen NAC transforms the clean and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: RQDiT architecture. Two DiTs are applied along the NAC frame and depth dimensions, respectively. each frame and depth pair. Conditioning on the noisy codes c˜ is done using adaptive layer normalization [7, 44] in both DiTs. Each DiT comprises multi-head self-attention (MHSA) blocks and MLPs with residual connections in multiple layers. We use rotary position embedding (RoPE) [45] for encoding the frame and… view at source ↗
Figure 4
Figure 4. Figure 4: shows the non-intrusive metrics averaged across both datasets as a function of the input SNR for all systems. 3.2 3.4 3.6 3.8 DNSMOS 2.50 2.75 3.00 3.25 3.50 3.75 NISQA (-5, 0] (0, 5] (5, 10] (10, 15] Input SNR (dB) 2.4 2.6 2.8 3.0 3.2 UTMOS (-5, 0] (0, 5] (5, 10] (10, 15] Input SNR (dB) 3.0 3.5 4.0 SCOREQ Conv-TasNet BSRNN NAC-SE SGMSE+ EDM-SE EDM-NAC-SE ADDSE-XS ADDSE-S ADDSE-M ADDSE-L ADDSE-XL [PITH_FU… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

85 extracted references · 5 linked inside Pith

  1. [1]

    They typically consider a conditional probability distribution over clean speech given noisy speech in the continuous short-time Fourier transform (STFT) domain

    Introduction Diffusion models have recently gained attention as powerful generative approaches for speech enhancement (SE) [1–4]. They typically consider a conditional probability distribution over clean speech given noisy speech in the continuous short-time Fourier transform (STFT) domain. However, the high dimen- sionality of the STFT representation, co...

  2. [2]

    Neural audio codecs and residual vector quantization A typical NAC [13–15] consists of an encoder, a quantizer, and a decoder

    Background 2.1. Neural audio codecs and residual vector quantization A typical NAC [13–15] consists of an encoder, a quantizer, and a decoder. The encoder transforms the input waveform into a low-frame-rate latent representation h∈R L×H, where L is the number of time frames, and H is the dimension of the latent space. The quantizer then maps the latent re...

    Pith/arXiv arXiv 2026

  3. [3]

    Noisy-speech-conditioned absorbing discrete diffusion We propose to perform SE by modeling the conditional distri- bution of clean speech codes given noisy speech codes using ADD

    Absorbing discrete diffusion for speech enhancement 3.1. Noisy-speech-conditioned absorbing discrete diffusion We propose to perform SE by modeling the conditional distri- bution of clean speech codes given noisy speech codes using ADD. The framework, which we callADDSE, is illustrated in Fig. 1. During training, a frozen NAC transforms the clean and nois...

  4. [4]

    Datasets Noisy speech is simulated dynamically during training [46, 47] by mixing clean speech utterances with noise segments at 16 kHz

    Experimental setup 4.1. Datasets Noisy speech is simulated dynamically during training [46, 47] by mixing clean speech utterances with noise segments at 16 kHz. Clean speech utterances are selected from DNS5 [48], LibriSpeech [49], MLS [50], VCTK [51], and EARS [52]. Noise segments are selected from DNS5 [48], WHAM! [53], FSD50K [54], FMA [55], and DEMAND...

    2025

  5. [5]

    Average results for each model and each dataset are shown in Table 1.Clean-NACdenotes the clean speech encoded and de- coded by the NAC

    Results Systems are compared using 4 intrusive metrics PESQ [71], ESTOI [72], SDR, and MCD [73], 4 non-intrusive metrics DNS- MOS [74], NISQA [75], UTMOS [76], and SCOREQ [77], and 2 downstream-task-independent metrics LPS [78] and SBS [79]. Average results for each model and each dataset are shown in Table 1.Clean-NACdenotes the clean speech encoded and ...

  6. [6]

    The discrete nature of the speech codes allows reusing the network predictions for multiple sampling steps, leading to more efficient sampling compared to continuous diffusion

    Conclusion We proposed ADDSE, a framework for SE based on ADD in the latent space of a NAC. The discrete nature of the speech codes allows reusing the network predictions for multiple sampling steps, leading to more efficient sampling compared to continuous diffusion. We also proposed RQDiT, a non-autoregressive ar- chitecture for modeling the hierarchica...

  7. [7]

    Conditional diffusion probabilistic model for speech enhancement,

    Y .-J. Lu, Z.-Q. Wang, S. Watanabe, A. Richard, C. Yu, and Y . Tsao, “Conditional diffusion probabilistic model for speech enhancement,” inProc. ICASSP, 2022

    2022

  8. [8]

    Speech enhancement and dereverberation with diffusion-based generative models,

    J. Richter, S. Welker, J.-M. Lemercier, B. Lay, and T. Gerkmann, “Speech enhancement and dereverberation with diffusion-based generative models,”IEEE/ACM Trans. Audio, Speech, Lang. Pro- cess., 2023

    2023

  9. [9]

    Investigating the design space of diffusion models for speech enhancement,

    P. Gonzalez, Z.-H. Tan, J. Østergaard, J. Jensen, T. S. Alstrøm, and T. May, “Investigating the design space of diffusion models for speech enhancement,”IEEE/ACM Trans. Audio, Speech, Lang. Process., 2024

    2024

  10. [10]

    Diffusion models for audio restoration: A review,

    J.-M. Lemercier, J. Richter, S. Welker, E. Moliner, V . V¨alim¨aki, and T. Gerkmann, “Diffusion models for audio restoration: A review,”IEEE Signal Process. Mag., 2025

    2025

  11. [11]

    Auto-encoding variational bayes,

    D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” inProc. ICLR, 2014

    2014

  12. [12]

    High-resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in Proc. CVPR, 2022

    2022

  13. [13]

    Scalable diffusion models with transform- ers,

    W. Peebles and S. Xie, “Scalable diffusion models with transform- ers,” inProc. ICCV, 2023

    2023

  14. [14]

    PixArt- α: Fast training of diffusion trans- former for photorealistic text-to-image synthesis,

    J. Chen, J. Yu, C. Ge, L. Yao, E. Xie, Y . Wu, Z. Wang, J. Kwok, P. Luo, H. Luet al., “PixArt- α: Fast training of diffusion trans- former for photorealistic text-to-image synthesis,” inProc. ICLR, 2024

    2024

  15. [15]

    Analyzing and improving the training dynamics of diffusion mod- els,

    T. Karras, M. Aittala, J. Lehtinen, J. Hellsten, T. Aila, and S. Laine, “Analyzing and improving the training dynamics of diffusion mod- els,” inProc. CVPR, 2024

    2024

  16. [16]

    High-resolution speech restoration with latent diffusion model,

    T. Dhyani, F. Lux, M. Mancusi, G. Fabbro, F. Hohl, and N. T. Vu, “High-resolution speech restoration with latent diffusion model,” in Proc. ICASSP, 2025

    2025

  17. [17]

    ProSE: Diffusion priors for speech enhancement,

    S. Kumar, S. Ghosh, U. Tyagi, A. J. Ratnarajah, C. K. R. Evuru, R. Duraiswami, and D. Manocha, “ProSE: Diffusion priors for speech enhancement,” inProc. NAACL, 2025

    2025

  18. [18]

    DiTSE: High-fidelity generative speech enhancement via latent diffusion transformers,

    H. R. Guimar˜aes, J. Su, R. Kumar, T. H. Falk, and Z. Jin, “DiTSE: High-fidelity generative speech enhancement via latent diffusion transformers,”arXiv preprint arXiv:2504.09381, 2025

    arXiv 2025

  19. [19]

    Soundstream: An end-to-end neural audio codec,

    N. Zeghidour, A. Luebs, A. Omran, J. Skoglund, and M. Tagliasac- chi, “Soundstream: An end-to-end neural audio codec,”IEEE/ACM Trans. Audio, Speech, Lang. Process., 2021

    2021

  20. [20]

    High fidelity neural audio compression,

    A. D´efossez, J. Copet, G. Synnaeve, and Y . Adi, “High fidelity neural audio compression,”Trans. Mach. Learn. Res., 2023

    2023

  21. [21]

    High-fidelity audio compression with improved RVQGAN,

    R. Kumar, P. Seetharaman, A. Luebs, I. Kumar, and K. Kumar, “High-fidelity audio compression with improved RVQGAN,” in Proc. NeurIPS, 2023

    2023

  22. [22]

    BigCodec: Push- ing the limits of low-bitrate neural speech codec,

    D. Xin, X. Tan, S. Takamichi, and H. Saruwatari, “BigCodec: Push- ing the limits of low-bitrate neural speech codec,”arXiv preprint arXiv:2409.05377, 2024

    Pith/arXiv arXiv 2024

  23. [23]

    Moshi: A speech-text foundation model for real-time dialogue,

    A. D´efossez, L. Mazar´e, M. Orsini, A. Royer, P. P´erez, H. J´egou, E. Grave, and N. Zeghidour, “Moshi: A speech-text foundation model for real-time dialogue,”arXiv preprint arXiv:2410.00037, 2024

    Pith/arXiv arXiv 2024

  24. [24]

    Focal- Codec: Low-bitrate speech coding via focal modulation networks,

    L. Della Libera, F. Paissan, C. Subakan, and M. Ravanelli, “Focal- Codec: Low-bitrate speech coding via focal modulation networks,” inProc. NeurIPS, 2025

    2025

  25. [25]

    SELM: Speech enhancement using discrete tokens and language models,

    Z. Wang, X. Zhu, Z. Zhang, Y . Lv, N. Jiang, G. Zhao, and L. Xie, “SELM: Speech enhancement using discrete tokens and language models,” inProc. ICASSP, 2024

    2024

  26. [26]

    Low-latency speech enhancement via speech token generation,

    H. Xue, X. Peng, and Y . Lu, “Low-latency speech enhancement via speech token generation,” inProc. ICASSP, 2024

    2024

  27. [27]

    Genhancer: High-fidelity speech enhancement via generative modeling on discrete codec tokens

    H. Yang, J. Su, M. Kim, and Z. Jin, “Genhancer: High-fidelity speech enhancement via generative modeling on discrete codec tokens.” inProc. Interspeech, 2024

    2024

  28. [28]

    MaskSR: Masked language model for full-band speech restoration,

    X. Li, Q. Wang, and X. Liu, “MaskSR: Masked language model for full-band speech restoration,” inProc. Interspeech, 2024

    2024

  29. [29]

    Joint semantic knowledge distillation and masked acoustic modeling for full-band speech restoration with improved intelligibility,

    X. Liu, X. Li, J. Serr`a, and S. Pascual, “Joint semantic knowledge distillation and masked acoustic modeling for full-band speech restoration with improved intelligibility,” inProc. ICASSP, 2025

    2025

  30. [30]

    DisContSE: Single-step diffusion speech enhancement based on joint discrete and continuous embeddings,

    Y . Fu and T. Fingscheidt, “DisContSE: Single-step diffusion speech enhancement based on joint discrete and continuous embeddings,” inProc. ICASSP, 2026

    2026

  31. [31]

    Speech enhancement using continuous embeddings of neural audio codec,

    H. Li, J. Q. Yip, T. Fan, and E. S. Chng, “Speech enhancement using continuous embeddings of neural audio codec,” inProc. ICASSP, 2025

    2025

  32. [32]

    GenSE: Generative speech enhancement via language models using hierar- chical modeling,

    J. Yao, H. Liu, C. Chen, Y . Hu, E. Chng, and L. Xie, “GenSE: Generative speech enhancement via language models using hierar- chical modeling,” inProc. ICLR, 2025

    2025

  33. [33]

    LLaSE-G1: Incentivizing generalization capability for LLaMA-based speech enhancement,

    B. Kang, X. Zhu, Z. Zhang, Z. Ye, M. Liu, Z. Wang, Y . Zhu, G. Ma, J. Chen, L. Xiaoet al., “LLaSE-G1: Incentivizing generalization capability for LLaMA-based speech enhancement,” inProc. ACL, 2025

    2025

  34. [34]

    Universal discrete-domain speech enhancement,

    F. Liu, Y . Ai, Y .-X. Lu, R.-C. Zheng, H.-P. Du, and Z.-H. Ling, “Universal discrete-domain speech enhancement,”IEEE Trans. Au- dio, Speech, Lang. Process., 2025

    2025

  35. [35]

    Modeling strategies for speech enhancement in the latent space of a neural audio codec,

    S. Kammoun, X. Alameda-Pineda, and S. Leglaive, “Modeling strategies for speech enhancement in the latent space of a neural audio codec,” inProc. ICASSP, 2026

    2026

  36. [36]

    High-fidelity speech enhancement via discrete audio tokens,

    L. A. Lanzend¨orfer, F. Berdoz, A. Asonitis, and R. Wattenhofer, “High-fidelity speech enhancement via discrete audio tokens,” in Proc. ICASSP, 2026

    2026

  37. [37]

    Autoregres- sive speech enhancement via acoustic tokens,

    L. Della Libera, C. Subakan, and M. Ravanelli, “Autoregres- sive speech enhancement via acoustic tokens,”arXiv preprint arXiv:2507.12825, 2025

    Pith/arXiv arXiv 2025

  38. [38]

    UniSE: A uni- fied framework for decoder-only autoregressive LM-based speech enhancement,

    H. Yan, C. Liu, S. Xue, X. Liang, and Z. Xue, “UniSE: A uni- fied framework for decoder-only autoregressive LM-based speech enhancement,”arXiv preprint arXiv:2510.20441, 2025

    Pith/arXiv arXiv 2025

  39. [39]

    Structured denoising diffusion models in discrete state-spaces,

    J. Austin, D. D. Johnson, J. Ho, D. Tarlow, and R. Van Den Berg, “Structured denoising diffusion models in discrete state-spaces,” in Proc. NeurIPS, 2021

    2021

  40. [40]

    A continuous time framework for discrete denoising models,

    A. Campbell, J. Benton, V . De Bortoli, T. Rainforth, G. Deligian- nidis, and A. Doucet, “A continuous time framework for discrete denoising models,” inProc. NeurIPS, 2022

    2022

  41. [41]

    Concrete score matching: Generalized score matching for discrete data,

    C. Meng, K. Choi, J. Song, and S. Ermon, “Concrete score matching: Generalized score matching for discrete data,” inProc. NeurIPS, 2022

    2022

  42. [42]

    Score-based continuous-time discrete diffusion models,

    H. Sun, L. Yu, B. Dai, D. Schuurmans, and H. Dai, “Score-based continuous-time discrete diffusion models,” inProc. ICLR, 2023

    2023

  43. [43]

    Discrete diffusion modeling by estimating the ratios of the data distribution,

    A. Lou, C. Meng, and S. Ermon, “Discrete diffusion modeling by estimating the ratios of the data distribution,” inProc. ICML, 2024

    2024

  44. [44]

    Simplified and generalized masked diffusion for discrete data,

    J. Shi, K. Han, Z. Wang, A. Doucet, and M. Titsias, “Simplified and generalized masked diffusion for discrete data,” inProc. NeurIPS, 2024

    2024

  45. [45]

    Your absorb- ing discrete diffusion secretly models the conditional distributions of clean data,

    J. Ou, S. Nie, K. Xue, F. Zhu, J. Sun, Z. Li, and C. Li, “Your absorb- ing discrete diffusion secretly models the conditional distributions of clean data,” inProc. ICLR, 2025

    2025

  46. [46]

    MaskGIT: Masked generative image transformer,

    H. Chang, H. Zhang, L. Jiang, C. Liu, and W. T. Freeman, “MaskGIT: Masked generative image transformer,” inProc. CVPR, 2022

    2022

  47. [47]

    W. J. Anderson,Continuous-time Markov chains: An applications- oriented approach. Springer Science & Business Media, 2012

    2012

  48. [48]

    F. P. Kelly,Reversibility and stochastic networks. Cambridge University Press, 1979

    1979

  49. [49]

    Autoregressive image generation using residual quantization,

    D. Lee, C. Kim, S. Kim, M. Cho, and W.-S. Han, “Autoregressive image generation using residual quantization,” inProc. CVPR, 2022

    2022

  50. [50]

    FiLM: Visual reasoning with a general conditioning layer,

    E. Perez, F. Strub, H. De Vries, V . Dumoulin, and A. Courville, “FiLM: Visual reasoning with a general conditioning layer,” inProc. AAAI, 2018

    2018

  51. [51]

    RoFormer: Enhanced transformer with rotary position embedding,

    J. Su, M. Ahmed, Y . Lu, S. Pan, W. Bo, and Y . Liu, “RoFormer: Enhanced transformer with rotary position embedding,”Neurocom- puting, 2024

    2024

  52. [52]

    Wavesplit: End-to-end speech separation by speaker clustering,

    N. Zeghidour and D. Grangier, “Wavesplit: End-to-end speech separation by speaker clustering,”IEEE/ACM Trans. Audio, Speech, Lang. Process., 2021

    2021

  53. [53]

    An empirical study of training mixture generation strategies on speech separation: Dynamic mixing and augmenta- tion,

    S. Choi, Y . Lee, J. Park, H. Y . Kim, B.-Y . Kim, Z.-Q. Wang, and S. Watanabe, “An empirical study of training mixture generation strategies on speech separation: Dynamic mixing and augmenta- tion,” inProc. APSIPA ASC, 2022

    2022

  54. [54]

    ICASSP 2023 Deep Noise Suppression Challenge,

    H. Dubey, A. Aazami, V . Gopal, B. Naderi, S. Braun, R. Cutler, H. Gamper, M. Golestaneh, and R. Aichner, “ICASSP 2023 Deep Noise Suppression Challenge,”IEEE Open J. Signal Process., 2024

    2023

  55. [55]

    LibriSpeech: An ASR corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “LibriSpeech: An ASR corpus based on public domain audio books,” inProc. ICASSP, 2015

    2015

  56. [56]

    MLS: A large-scale multilingual dataset for speech research,

    V . Pratap, Q. Xu, A. Sriram, G. Synnaeve, and R. Collobert, “MLS: A large-scale multilingual dataset for speech research,” inProc. Interspeech, 2020

    2020

  57. [57]

    The V oice Bank corpus: Design, collection and data analysis of a large regional accent speech database,

    C. Veaux, J. Yamagishi, and S. King, “The V oice Bank corpus: Design, collection and data analysis of a large regional accent speech database,” inProc. O-COCOSDA/CASLRE, 2013

    2013

  58. [58]

    EARS: An anechoic fullband speech dataset benchmarked for speech enhancement and derever- beration,

    J. Richter, Y .-C. Wu, S. Krenn, S. Welker, B. Lay, S. Watanabe, A. Richard, and T. Gerkmann, “EARS: An anechoic fullband speech dataset benchmarked for speech enhancement and derever- beration,” inProc. Interspeech, 2024

    2024

  59. [59]

    WHAM!: Extending speech separation to noisy environments,

    G. Wichern, J. Antognini, M. Flynn, L. R. Zhu, E. McQuinn, D. Crow, E. Manilow, and J. L. Roux, “WHAM!: Extending speech separation to noisy environments,” inProc. Interspeech, 2019

    2019

  60. [60]

    FSD50K: An open dataset of human-labeled sound events,

    E. Fonseca, X. Favory, J. Pons, F. Font, and X. Serra, “FSD50K: An open dataset of human-labeled sound events,”IEEE/ACM Trans. Audio, Speech, Lang. Process., 2022

    2022

  61. [61]

    FMA: A dataset for music analysis,

    M. Defferrard, K. Benzi, P. Vandergheynst, and X. Bresson, “FMA: A dataset for music analysis,” inProc. ISMIR, 2017

    2017

  62. [62]

    The diverse environments multi-channel acoustic noise database (DEMAND): A database of multichannel environmental noise recordings,

    J. Thiemann, N. Ito, and E. Vincent, “The diverse environments multi-channel acoustic noise database (DEMAND): A database of multichannel environmental noise recordings,” inProc. Mtgs. Acoust., 2013

    2013

  63. [63]

    TUT database for acous- tic scene classification and sound event detection,

    A. Mesaros, T. Heittola, and T. Virtanen, “TUT database for acous- tic scene classification and sound event detection,” inProc. EU- SIPCO, 2016

    2016

  64. [64]

    Dataset of British English speech recordings for psychoacoustics and speech process- ing research: The Clarity speech corpus,

    S. Graetzer, M. A. Akeroyd, J. Barker, T. J. Cox, J. F. Culling, G. Naylor, E. Porter, and R. Viveros-Mu˜noz, “Dataset of British English speech recordings for psychoacoustics and speech process- ing research: The Clarity speech corpus,”Data Br ., 2022

    2022

  65. [65]

    Neural networks fail to learn periodic functions and how to fix it,

    L. Ziyin, T. Hartwig, and M. Ueda, “Neural networks fail to learn periodic functions and how to fix it,” inProc. NeurIPS, 2020

    2020

  66. [66]

    Vector-quantized image modeling with improved VQGAN,

    J. Yu, X. Li, J. Y . Koh, H. Zhang, R. Pang, J. Qin, A. Ku, Y . Xu, J. Baldridge, and Y . Wu, “Vector-quantized image modeling with improved VQGAN,” inProc. ICLR, 2022

    2022

  67. [67]

    HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis,

    J. Kong, J. Kim, and J. Bae, “HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis,” inProc. NeurIPS, 2025

    2025

  68. [68]

    Adam: A method for stochastic optimiza- tion,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimiza- tion,” inProc. ICLR, 2015

    2015

  69. [69]

    Decoupled weight decay regulariza- tion,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regulariza- tion,” inProc. ICLR, 2019

    2019

  70. [70]

    Conv-TasNet: Surpassing ideal time- frequency magnitude masking for speech separation,

    Y . Luo and N. Mesgarani, “Conv-TasNet: Surpassing ideal time- frequency magnitude masking for speech separation,”IEEE/ACM Trans. Audio, Speech, Lang. Process., 2019

    2019

  71. [71]

    Music source separation with band-split RNN,

    Y . Luo and J. Yu, “Music source separation with band-split RNN,” IEEE/ACM Trans. Audio, Speech, Lang. Process., 2023

    2023

  72. [72]

    Scaling beyond denoising: Submitted system and findings in URGENT Challenge 2025,

    Z. Sun, A. Li, T. Lei, R. Chen, M. Yu, C. Zheng, Y . Zhou, and D. Yu, “Scaling beyond denoising: Submitted system and findings in URGENT Challenge 2025,” inProc. Interspeech, 2025

    2025

  73. [73]

    Elucidating the design space of diffusion-based generative models,

    T. Karras, M. Aittala, T. Aila, and S. Laine, “Elucidating the design space of diffusion-based generative models,” inProc. NeurIPS, 2022

    2022

  74. [74]

    Diffusion models beat GANs on image synthesis,

    P. Dhariwal and A. Nichol, “Diffusion models beat GANs on image synthesis,” inProc. NeurIPS, 2021

    2021

  75. [75]

    Diffusion-based speech enhancement in matched and mismatched conditions using a Heun-based sampler,

    P. Gonzalez, Z.-H. Tan, J. Østergaard, J. Jensen, T. S. Alstrøm, and T. May, “Diffusion-based speech enhancement in matched and mismatched conditions using a Heun-based sampler,” inProc. ICASSP, 2024

    2024

  76. [76]

    The effect of training dataset size on discriminative and diffusion-based speech enhancement systems,

    ——, “The effect of training dataset size on discriminative and diffusion-based speech enhancement systems,”IEEE Signal Pro- cess. Lett., 2024

    2024

  77. [77]

    Per- ceptual evaluation of speech quality (PESQ)–A new method for speech quality assessment of telephone networks and codecs,

    A. W. Rix, J. G. Beerends, M. P. Hollier, and A. P. Hekstra, “Per- ceptual evaluation of speech quality (PESQ)–A new method for speech quality assessment of telephone networks and codecs,” in Proc. ICASSP, 2001

    2001

  78. [78]

    An algorithm for predicting the intelligi- bility of speech masked by modulated noise maskers,

    J. Jensen and C. H. Taal, “An algorithm for predicting the intelligi- bility of speech masked by modulated noise maskers,”IEEE/ACM Trans. Audio, Speech, Lang. Process., 2016

    2016

  79. [79]

    Mel-cepstral distance measure for objective speech quality assessment,

    R. Kubichek, “Mel-cepstral distance measure for objective speech quality assessment,” inProc. PACRIM, 1993

    1993

  80. [80]

    DNSMOS p.835: A non- intrusive perceptual objective speech quality metric to evaluate noise suppressors,

    C. K. Reddy, V . Gopal, and R. Cutler, “DNSMOS p.835: A non- intrusive perceptual objective speech quality metric to evaluate noise suppressors,” inProc. ICASSP, 2022

    2022

Showing first 80 references.