REVIEW 1 major objections 5 minor 34 references
Posterior Transition Modeling for Unsupervised Diffusion-Based Speech Enhancement
T0 review · 1 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The paper proposes two unsupervised speech-enhancement algorithms that model the conditional reverse diffusion transition directly; the second, which diffuses noisy speech itself, yields an exact likelihood score and consistently beats…
desk verdict Worth a careful revision: the DEPSE-TL exact-likelihood claim does not hold as written, but DEPSE-IL is a solid contribution and the experiments are honest. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the conditional reverse transition distribution of diffusion states, $p_\varphi(s_{i-1}|s_i,x_{i-1})$, expressed as the Gaussian product of the prior transition $p(s_{i-1}|s_i)$ (computed from the Euler-Maruyama discretization plus Langevin MCMC) and the observation likelihood $p_\varphi(x_{i-1}|s_{i-1})$. For DEPSE-TL the likelihood is made tractable by running the same forward SDE on the noisy signal $x_0=x$, giving $x_{i-1}=e^{-\gamma\tau_{i-1}}x+\sigma_{\tau_{i-1}}\zeta$; under the shared-noise observation $x=s+n$, the covariance reduces to $e^{-2\gamma\tau_{i-1}}\mathrm{diag}(v_\varphi)$, so the posterior transition is exactly a Gaussian with closed-form mean and covariance. This replaces posterior-score guidance with direct transition sampling.
What would settle it
Run DEPSE-TL twice on the same input, once with shared Gaussian noise draws for the forward diffusions of $s$ and $x$ and once with independent draws; if enhancement metrics change measurably, or if the empirical covariance of $x_{i-1}-s_{i-1}$ shows the extra $2\sigma_{\tau_{i-1}}^2 I$ term, the claimed exactness fails. A simpler check: compute the Monte Carlo variance of $x_{i-1}-s_{i-1}$ on held-out data and compare it to $e^{-2\gamma\tau_{i-1}}\mathrm{diag}(v_\varphi)$.
Extended reading notes
Core claim
The central claim is that posterior sampling for unsupervised diffusion-based speech enhancement is better done by explicitly modeling the distribution $p_\varphi(s_{i-1}|s_i,x)$ (or with $x_{i-1}$) at each reverse step, rather than by modifying only the mean of the unconditional reverse transition with a posterior score. DEPSE-IL derives this conditional Gaussian from the prior transition and an approximate likelihood, removing the balancing hyperparameter. DEPSE-TL goes further: it defines a diffusion process over the noisy speech $x$, so the likelihood $p_\varphi(x_{i-1}|s_{i-1})$ becomes a closed-form Gaussian with covariance $e^{-2\gamma\tau_{i-1}}\mathrm{diag}(v_\varphi)$. The product of two Gaussians yields the conditional transition (mean and variance), which the algorithm samples from while updating the noise parameters $\varphi$ via a Tweedie-based estimate inside the EM loop. The reported result is that DEPSE-TL outperforms prior unsupervised baselines on SI-SDR, PESQ and ESTOI in almost all matched and mismatched settings, and on one cross-corpus setup it beats the supervised baselines as well.
Load-bearing premise
The exact likelihood formula in equation (19) assumes the diffusion noises driving $x_t$ and $s_t$ are the same Wiener realization at every time $t$; if the two forward processes use independent noises, the covariance of $x_{i-1}-s_{i-1}$ gains an extra $2\sigma_{\tau_{i-1}}^2 I$ term and the 'exact' likelihood is an optimistic approximation.
Editorial extensions
If this is right
- Unsupervised speech enhancement no longer needs a tuned likelihood-guidance hyperparameter; the posterior is sampled from a closed-form Gaussian at each reverse step.
- DEPSE-TL can be dropped into existing EM-based noise estimation loops, since the M-step and the Tweedie-based $\varphi$ update are unchanged.
- The exact likelihood avoids the uninformative-prior approximation used in previous methods, so the posterior reflects the actual noise model more faithfully.
- Because the prior is trained only on clean speech, the method retains more SI-SDR under mismatched training and test corpora than supervised baselines, often matching or exceeding them.
Reading between the lines
- The shared-noise assumption in DEPSE-TL may be the price of 'exactness'; if it is violated in practice, the formulation could be corrected by replacing $\mathrm{diag}(v_\varphi)$ with $\mathrm{diag}(v_\varphi)+2\sigma_{\tau_{i-1}}^2 I$, which remains tractable and would test the bias empirically.
- The same conditional-transition construction should transfer to other linear inverse problems with structured noise, such as dereverberation or blind deconvolution, where the observation model is also factorized Gaussian.
- The authors' observation that DEPSE-TL trades SI-SIR for SI-SAR suggests a possible per-step control of the relative weight between likelihood and prior, which the paper does not explore.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes two unsupervised, diffusion-based speech enhancement algorithms operating in the STFT domain with a Gaussian noise model whose variance is parameterized by NMF factors and estimated online by an EM/Tweedie procedure inherited from prior work. DEPSE-IL derives the reverse transition of the posterior by forming the Gaussian product of the prior reverse transition (11) and a likelihood (16) that uses the uninformative prior approximation, thereby avoiding the balancing hyperparameter (lambda) of UDiffSE+. DEPSE-TL instead runs a forward diffusion over the noisy observation itself and claims a closed-form, exact likelihood (19), from which exact posterior transitions (20)-(22) are obtained. Experiments on WSJ0-QUT and VoiceBank-DEMAND, with matched and mismatched training/test conditions, report SI-SDR, SI-SIR, SI-SAR, PESQ, and ESTOI; DEPSE-TL is consistently better than the unsupervised baselines on most metrics and exhibits smaller degradation than the supervised baselines under domain mismatch.
Significance. Strengths of the manuscript: the algebra in DEPSE-IL (Eqs. (16)-(18)) is correct under the uninformative-prior approximation, and removing the trade-off hyperparameter is a genuine simplification. The experimental protocol is careful: matched and mismatched corpus evaluations, a standard metric suite, and paired t-tests, with an honest caveat that the baselines are not state of the art. The domain-mismatch robustness of DEPSE-TL (31.6% SI-SDR drop versus 53.8-64.6% for the supervised baselines) is a substantive, falsifiable finding. However, the headline property of DEPSE-TL, the 'fully tractable and exact likelihood score', is where the correctness burden lies, and the manuscript does not discharge it: Eq. (19) requires a pathwise coupling that is neither stated nor implemented (see the major comment). The contribution is therefore potentially sound but overstated as written; the empirical results suggest the method works as an approximation even if the exactness claim fails.
major comments (1)
- [§III-C, Eq. (19); Algorithm 2, line 5] Eq. (19) is presented as a closed-form consequence of the observation model x = s + n, but the identity holds only when the speech and measurement diffusions are driven by the same Wiener path, so that x_t = s_t + e^{-γt}n for every t. This coupling is never stated, and Algorithm 2 does not implement it: line 5 draws x_{i-1} = e^{-γτ_{i-1}}x + σ_{τ_{i-1}}ζ with a fresh Gaussian independent of the s-chain draws in lines 6-7, and the x-chain is re-drawn from scratch at every step with x_i never used elsewhere. The marginal law of x_{i-1} is correct, so the problem is invisible unless one inspects the joint distribution: under the distribution actually sampled, the conditional p_φ(x_{i-1}|s_{i-1}) is not the Gaussian of Eq. (19). With independent forward noises the covariance contains extra terms (in the simplest model, a 2σ²_{τ_{i-1}}I contribution) and the conditional mean is not s_{i-1} in general. Consequently the posterior transitions (20)-(22) are not exact, and the abstract's claim of a 'fully tractable and exact likelihood score' is unsupported. This is load-bearing because exactness is the stated advantage of DEPSE-TL over DEPSE-IL. Two fixes are available: realize the coupling in the sampler (e.g., maintain a single n ~ NC(0, diag(v_φ)) and set x_{i-1} = s_{i-1} + e^{-γτ_{i-1}}n after drawing s_{i-1}), or derive the true conditional under the marginal-draw scheme and recharacterize DEPSE-TL as an approximate-likelihood method with an explicit bias. The empirical improvements may survive the correction, but the exactness narrative must be revised.
minor comments (5)
- [§III-B, Eqs. (16)-(18)] The algebraic steps are correct, but the notation divides by covariance terms such as Σ_back_i and e^{-γτ_{i-1}}Σ_IL_{x,i}; this is only meaningful because all quantities are diagonal or scalar multiples of the identity. Writing the precision-weighted mean and covariance with explicit inverses (or stating the diagonal structure once) would improve readability and remove the apparent dimension mismatch from e^{-γτ} appearing in a denominator.
- [Author list and Abstract] There are spacing artifacts in the author line ('Xav ier Alameda-Pineda') and in the abstract ('V oiceBank'); these should be corrected against the LaTeX source.
- [Tables I-II] Several SI-SIR entries appear broken across lines in the rendering (e.g., the UDiffSE+ row of Table I lists '12.34 4.34'); the table formatting should be fixed so that each metric is a single number.
- [§IV, Results] The sentence 'DEPSE-TL consistently outperforms DEPSE-IL and UDiffSE+ in terms of SI-SDR, PESQ, and ESTOI, across almost all evaluation setups' should name the one visible exception in Table I (matched), where DEPSE-TL's PESQ (2.17) is slightly below UDiffSE+'s (2.19); the 'almost all' hedge covers this, but the text should be explicit.
- [Abstract and §I] Once the assumption behind Eq. (19) is stated or the method is recharacterized, the phrase 'exact likelihood score' should be scoped, since the posterior still relies on the discretized reverse transition (11) with an approximate learned score and on the per-step NMF re-estimation of v_φ.
Circularity Check
No significant circularity: the derivations are self-contained and the empirical claims are not forced by fitted inputs; heavy self-citation is contextual, not load-bearing.
full rationale
The paper's derivation chain is largely self-contained. DEPSE-IL derives the conditional transition by Gaussian conditioning from the approximated likelihood (16) and prior transition (10)-(11); no parameter is fitted to the target metrics. DEPSE-TL defines a measurement-space diffusion and obtains the likelihood (19) from the stated observation model x=s+n; the posterior (22) follows algebraically from (19) and (10). The central exactness claim does not reduce to a fitted parameter or to an equation that is equivalent to its input by construction. The heavy reliance on the authors' prior work [13],[14] is contextual: those works supply the diffusion prior, the score network, the EM/NMF noise-parameter update, and the baseline comparisons, but they are not invoked as an external uniqueness theorem and the new conditional-transition derivation is shown in the text. The strongest caveat is a correctness gap rather than circularity: Eq. (19) holds only if the clean-speech and noisy-speech diffusion SDEs share the same Wiener realization, whereas Algorithm 2 samples x_{i-1} and s_{i-1} with independent Gaussian draws; this makes the 'exact' posterior (22) an approximation under the implemented sampler. That is an unstated coupling assumption, not a self-referential reduction, so it does not raise the circularity score. Score 2 reflects the presence of minor, non-load-bearing self-citation only.
Assumptions & free parameters
free parameters (2)
- Langevin step size r
- NMF rank of W and H
assumptions (4)
- domain assumption Observation model x = s + n with n ~ NC(0, diag(vφ)), vφ = vec(WH) low-rank NMF
- ad hoc to paper Uninformative prior p(s0|s_{i-1}) ∝ p(s_{i-1}|s0) in DEPSE-IL
- ad hoc to paper Signal and measurement diffusion share the same Wiener noise (implicit in eq. (19))
- standard math Reverse-time diffusion holds (Anderson 1982)
Cite this review
Pith. "Pith review of Posterior Transition Modeling for Unsupervised Diffusion-Based Speech Enhancement." pith.science (2026). https://pith.science/paper/JFZLRPLJ
@misc{pith2026250702391,
author = {Pith},
title = {Pith review of: Posterior Transition Modeling for Unsupervised Diffusion-Based Speech Enhancement},
year = {2026},
howpublished = {\url{https://pith.science/paper/JFZLRPLJ}},
note = {Machine review of arXiv:2507.02391}
}
read the original abstract
We explore unsupervised speech enhancement using diffusion models as expressive generative priors for clean speech. Existing approaches guide the reverse diffusion process using noisy speech through an approximate, noise-perturbed likelihood score, combined with the unconditional score via a trade-off hyperparameter. In this work, we propose two alternative algorithms that directly model the conditional reverse transition distribution of diffusion states. The first method integrates the diffusion prior with the observation model in a principled way, removing the need for hyperparameter tuning. The second defines a diffusion process over the noisy speech itself, yielding a fully tractable and exact likelihood score. Experiments on the WSJ0-QUT and VoiceBank-DEMAND datasets demonstrate improved enhancement metrics and greater robustness to domain shifts compared to both supervised and unsupervised baselines.
Reference graph
Works this paper leans on
-
[24]
Solving inverse problems in medical imaging with score-based generative models,
Y . Song, L. Shen, L. Xing, and S. Ermon, “Solving inverse problems in medical imaging with score-based generative models,” in International Conference on Learning Representations , 2022
work page 2022
-
[26]
Diffusion posterior sampling for li near inverse problem solving: A filtering perspective,
Z. Dou and Y . Song, “Diffusion posterior sampling for li near inverse problem solving: A filtering perspective,” in The Twelfth International Conference on Learning Representations , 2024
work page 2024
-
[13]
Unsupervised s peech enhance- ment with diffusion-based generative models,
B. Nortier, M. Sadeghi, and R. Serizel, “Unsupervised s peech enhance- ment with diffusion-based generative models,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICA SSP), 2024
work page 2024
-
[14]
Diffusion- based unsupervised audio-visual speech enhancement,
J.-E. Ayilo, M. Sadeghi, R. Serizel, and X. Alameda-Pin eda, “Diffusion- based unsupervised audio-visual speech enhancement,” in IEEE In- ternational Conference on Acoustics, Speech and Signal Pro cessing (ICASSP), 2025
work page 2025
-
[1]
Score-based generative modeling through stocha stic differen- tial equations,
Y . Song, J. Sohl-Dickstein, D. P . Kingma, A. Kumar, S. Erm on, and B. Poole, “Score-based generative modeling through stocha stic differen- tial equations,” in International Conference on Learning Representations (ICLR), 2021
work page 2021
-
[2]
Di ffusion models in vision: A survey,
F.-A. Croitoru, V . Hondru, R. T. Ionescu, and M. Shah, “Di ffusion models in vision: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 45, no. 9, pp. 10 850–10 869, 2023
work page 2023
-
[3]
A survey on diffusion models for inverse problems,
G. Daras, H. Chung, C.-H. Lai, Y . Mitsufuji, J. C. Y e, P . Mi lanfar, A. G. Dimakis, and M. Delbracio, “A survey on diffusion models for inverse problems,” arXiv preprint arXiv:2410.00083 , 2024
arXiv 2024
-
[4]
Speech enhancement and dereverberation with diffusion-b ased genera- tive models,
J. Richter, S. Welker, J.-M. Lemercier, B. Lay, and T. Ger kmann, “Speech enhancement and dereverberation with diffusion-b ased genera- tive models,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023
work page 2023
Show all 34 references
-
[5]
StoRM: A diffusion-based stochastic regeneration model for speech enhancement and dereverberation,
J.-M. Lemercier, J. Richter, S. Welker, and T. Gerkmann, “StoRM: A diffusion-based stochastic regeneration model for speech enhancement and dereverberation,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 2724–2737, 2023
2023
-
[6]
Diffusion-based speech enhancement in matched and mis- matched conditions using a Heun-based sampler,
P . Gonzalez, Z.-H. Tan, J. Østergaard, J. Jensen, T. S. Al strøm, and T. May, “Diffusion-based speech enhancement in matched and mis- matched conditions using a Heun-based sampler,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICA SSP), 2024, p...
2024
-
[7]
Univer- sal speech enhancement with score-based diffusion,
J. Serr` a, S. Pascual, J. Pons, R. O. Araz, and D. Scaini, “ Univer- sal speech enhancement with score-based diffusion,” arXiv preprint arXiv:2206.03065, 2022
2022 arXiv
-
[8]
Conditional diffusion probabilistic model for speech enh ancement,
Y .-J. Lu, Z.-Q. Wang, S. Watanabe, A. Richard, C. Y u, and Y . Tsao, “Conditional diffusion probabilistic model for speech enh ancement,” in IEEE International Conference on Acoustics, Speech and Sig nal Processing (ICASSP) , 2022, pp. 7402–7406
2022
-
[9]
Cold dif fusion for speech enhancement,
H. Y en, F. G. Germain, G. Wichern, and J. Le Roux, “Cold dif fusion for speech enhancement,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2023, pp. 1–5
2023
-
[10]
Supervised speech separation base d on deep learning: An overview,
D. Wang and J. Chen, “Supervised speech separation base d on deep learning: An overview,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 26, no. 10, pp. 1702–1726, 2018
2018
-
[11]
A regression appr oach to speech enhancement based on deep neural networks,
Y . Xu, J. Du, L.-R. Dai, and C.-H. Lee, “A regression appr oach to speech enhancement based on deep neural networks,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 23, no. 1, pp. 7–19, 2014
2014
-
[12]
Conv-TasNet: Surpassing idea l time– frequency magnitude masking for speech separation,
Y . Luo and N. Mesgarani, “Conv-TasNet: Surpassing idea l time– frequency magnitude masking for speech separation,” IEEE/ACM trans- actions on audio, speech, and language processing , vol. 27, no. 8, pp. 1256–1266, 2019
2019
-
[15]
UnDiff: Unsupervised V oice Restoration with Uncondition al Diffusion Model,
A. Iashchenko, P . Andreev, I. Shchekotov, N. Babaev, an d D. V etrov, “UnDiff: Unsupervised V oice Restoration with Uncondition al Diffusion Model,” in Proc. INTERSPEECH 2023 , 2023, pp. 4294–4298
2023
-
[16]
BUDDy: Single-channel blind unsupervised dereverberati on with dif- fusion models,
E. Moliner, J.-M. Lemercier, S. Welker, T. Gerkmann, an d V . V¨ alim¨ aki, “BUDDy: Single-channel blind unsupervised dereverberati on with dif- fusion models,” in 18th International W orkshop on Acoustic Signal Enhancement (IWAENC), 2024, pp. 120–124
2024
-
[17]
Diffusion models for audio restoration: A rev iew,
J.-M. Lemercier, J. Richter, S. Welker, E. Moliner, V . V ¨ alim¨ aki, and T. Gerkmann, “Diffusion models for audio restoration: A rev iew,” IEEE Signal Processing Magazine , vol. 41, no. 6, pp. 72–84, 2025
2025
-
[18]
Diffusion posterior sampling for general noisy inverse pr oblems,
H. Chung, J. Kim, M. T. Mccann, M. L. Klasky, and J. C. Y e, “Diffusion posterior sampling for general noisy inverse pr oblems,” in International Conference on Learning Representations , 2023. [Online]. Available: https://openreview.net/forum?id=OnD9zGAGT0k
2023
-
[19]
Diffusion model based poster ior sampling for noisy linear inverse problems,
X. Meng and Y . Kabashima, “Diffusion model based poster ior sampling for noisy linear inverse problems,” in The 16th Asian Conference on Machine Learning (Conference Track) , 2024. [Online]. Available: https://openreview.net/forum?id=A07d0PMRqc
2024
-
[20]
Reverse-time diffusion equation mod- els,
B. D. Anderson, “Reverse-time diffusion equation mod- els,” Stochastic Processes and their Applications , vol. 12, no. 3, pp. 313–326, 1982. [Online]. Available: https://www.sciencedirect.com/science/article/pii/0304414982900515
1982
-
[21]
Understanding diffusion objec tives as the ELBO with simple data augmentation,
D. P . Kingma and R. Gao, “Understanding diffusion objec tives as the ELBO with simple data augmentation,” in Thirty-seventh Conference on Neural Information Processing Systems , 2023
2023
-
[22]
C. M. Bishop and N. M. Nasrabadi, Pattern recognition and machine learning. Springer, 2006, vol. 4, no. 4
2006
-
[23]
Tweedie’s formula and selection bias,
B. Efron, “Tweedie’s formula and selection bias,” Journal of the Amer- ican Statistical Association , vol. 106, no. 496, pp. 1602–1614, 2011
2011
-
[25]
UDiffSE Supplementary Material,
B. Nortier, M. Sadeghi, and R. Ser- izel, “UDiffSE Supplementary Material,” https://github.com/joanne-b-nortier/UDiffSE/blob/main/docs/UDiffSE Supplementary.pdf 2024
2024
-
[27]
A recurrent variational autoencoder for speech enhancement,
S. Leglaive, X. Alameda-Pineda, L. Girin, and R. Horaud , “A recurrent variational autoencoder for speech enhancement,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICA SSP), 2020
2020
-
[28]
In vestigating rnn-based speech enhancement methods for noise-robust tex t-to-speech,
C. V . Botinhao, X. Wang, S. Takaki, and J. Y amagishi, “In vestigating rnn-based speech enhancement methods for noise-robust tex t-to-speech,” in 9th ISCA speech synthesis workshop , 2016, pp. 159–165
2016
-
[29]
CSR-I (W SJ0) complete LDC93S6B,
J. Garofolo, D. Graff, D. Paul, and D. Pallett, “CSR-I (W SJ0) complete LDC93S6B,” W eb Download. Philadelphia: Linguistic Data Consor- tium, vol. 83, 1993
1993
-
[30]
The QUT-NOISE-SRE protocol for the evalu ation of noisy speaker recognition,
D. Dean, A. Kanagasundaram, H. Ghaemmaghami, M. H. Rahm an, and S. Sridharan, “The QUT-NOISE-SRE protocol for the evalu ation of noisy speaker recognition,” in Proceedings of Interspeech , 2015, pp. 3456–3460
2015
-
[31]
The diverse enviro nments multi- channel acoustic noise database (DEMAND): A database of mul tichan- nel environmental noise recordings,
J. Thiemann, N. Ito, and E. Vincent, “The diverse enviro nments multi- channel acoustic noise database (DEMAND): A database of mul tichan- nel environmental noise recordings,” in Proceedings of Meetings on Acoustics, vol. 19, no. 1. AIP Publishing, 2013
2013
-
[32]
SDR –half- baked or well done?
J. Le Roux, S. Wisdom, H. Erdogan, and J. R. Hershey, “SDR –half- baked or well done?” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2019
2019
-
[33]
An algorithm for predicting th e intelligibility of speech masked by modulated noise maskers,
J. Jensen and C. H. Taal, “An algorithm for predicting th e intelligibility of speech masked by modulated noise maskers,” IEEE/ACM Transac- tions on Audio, Speech, and Language Processing , vol. 24, no. 11, pp. 2009–2022, 2016
2009
-
[34]
Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs,
A. W. Rix, J. G. Beerends, M. P . Hollier, and A. P . Hekstra , “Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs,” in IEEE international conference on acoustics, speech, and signal processing. Pr oceedi...
2001
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.