Diffusion Buffer: Online Diffusion-based Speech Enhancement with Sub-Second Latency

Bunlong Lay; Rostislav Makarov; Timo Gerkmann

REVIEW 4 major objections 5 minor 34 references

Diffusion Buffer: Online Diffusion-based Speech Enhancement with Sub-Second Latency

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A sliding buffer that assigns more noise to newer frames turns diffusion-based speech enhancement into an online, sub-second-latency process, at the price of a per-frame output delay.

desk verdict Solid, practical adaptation of rolling diffusion to online speech enhancement; the buffer-size-as-steps equivalence is plausible but not rigorously established. read the letter →

arxiv 2506.02908 v2 pith:IWAZO5T4 submitted 2025-06-03 eess.AS cs.LG

Bunlong Lay , Rostislav Makarov , Timo Gerkmann This is my paper

classification eess.AScs.LG

keywords speechenhancementdiffusionmodelsonlineprocessingstreamingaudioscore-basedgenerativelowlatencyBufferstochasticdifferentialequations

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Diffusion models clean speech superbly but, because each denoising step needs a neural-network call, they have been too slow for live audio. This paper proposes a Diffusion Buffer: the most recent B frames of a noisy stream sit in a buffer, the newest frame carrying the most noise and older frames progressively cleaner, and each incoming frame triggers exactly one network call that advances every buffered frame one reverse-diffusion step. The buffer size B sets the algorithmic latency as B times the 16 ms hop length, so the user can trade quality against delay. On an EARS-WHAM test set, the method runs online on a laptop GPU with 320–960 ms latency and, at 60 steps, matches or slightly beats offline diffusion baselines in PESQ while staying within 0.01 ESTOI of a real-time discriminative system. If correct, this is the first diffusion-based speech enhancer that can follow a live stream.

What carries the argument

The central object is the Diffusion Buffer, a length-B sliding window in which frame position is identified with a diffusion time step: the newest frame enters at $t_B = T_{\max}$ and each older frame sits one step earlier, down to $t_1 = \epsilon$. The load-bearing move is that a single forward pass of the score network on the whole buffer is treated as one reverse step for every buffered frame simultaneously, so that the oldest frame reaches $t=0$ and is emitted. This turns the per-step cost of reverse diffusion into a per-frame cost, and makes the number of reverse steps equal to the buffer length, so latency is $h_s \cdot B$ with hop length $h_s$. Training uses the per-frame denoising score matching loss of Equation (7), with the network output cropped to the last B frames; inference is Algorithm 1.

What would settle it

Take a fixed two-second noisy utterance, run Algorithm 1 with B=60, then run the same buffer through 60 independent per-frame score-model calls so each frame follows its own reverse trajectory; if the two enhanced signals differ by more than the reported 0.05 PESQ margin, the paper's key equivalence between the joint update and per-frame reverse steps is false.

Watch

Extended reading notes

Core claim

The central claim is that spatial position inside a sliding buffer can stand in for diffusion time. The paper writes the buffer as $V_{\vec{t}}$, where the last B frames of the current chunk are assigned ascending diffusion times $t_1 < \cdots < t_B = T_{\max}$; the newest frame is deliberately corrupted to look like $t_B$, and each older frame sits at an earlier time. A single score-model call $u_\theta$ applied to the whole buffer is taken to move every frame one step back in the reverse SDE, so the oldest frame leaves the buffer clean. Because this one call replaces the B calls that a vanilla score-based enhancer needs, the method fits inside one 16 ms hop; the price is that output is delayed by B hops. The paper reports that DB-BBED and DB-OUVE, trained with the same SDE parameterizations as the BBED and OUVE baselines, slightly trail those baselines at small B but overtake them by 0.05 PESQ at B=60, and that WVMOS is on par or better in most configurations.

Load-bearing premise

The inference procedure assumes that one network call can correctly denoise all buffer frames at different diffusion times at once, and that this joint update is exactly equivalent to advancing each frame one step in the reverse SDE, an equivalence the paper does not derive.

Editorial extensions

If this is right

With B=60 reverse steps, both DB-OUVE and DB-BBED exceed their offline counterparts by 0.05 PESQ, so the online constraint does not cost quality at that buffer size.
The method's latency is exactly $h_s \cdot B$; choosing B between 20 and 60 puts algorithmic latency in the 320–960 ms range while keeping one score call under the 16 ms hop time.
Because the score model runs once per frame rather than B times, the method achieves RTF below 1 on a laptop GPU, where original OUVE and BBED would need RTF equal to B times 14/16.
On WVMOS, DB versions are at least on par with vanilla baselines for most step counts, and DB-BBED outperforms BBED for 20, 30, and 60 steps.
Compared with Demucs, the DB methods yield higher PESQ when N>30 and comparable intelligibility, with ESTOI values differing by only 0.01.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The time-to-diffusion alignment inside the buffer is not speech-specific, so the same one-call-per-frame mechanism could plausibly be ported to other sequential generative tasks such as streaming vocoding, live captioning, or online dereverberation.
Because latency is proportional to B while quality saturates, an adaptive controller could dynamically change the buffer size based on estimated noise level or available compute, a schedule the paper does not explore.
The joint single-call update is an approximation; a direct training loss that simulates the buffer's mixed-noise inputs, rather than the per-frame matching loss of Equation (7), might make the equivalence more robust under distribution shift.
A shorter buffer with every frame updated on alternating hops could cut latency below $h_s \cdot B$ while keeping the same number of reverse steps per frame, though the paper does not test such interleaved schedules.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a solid, honest engineering paper. The new thing is taking the rolling/FIFO diffusion idea from video generation, adapting it to speech enhancement with a buffer of frames assigned increasing diffusion noise, and showing that one score-network call per hop runs on a consumer GPU with 320–960 ms latency. Code is released and the RTF measurements are concrete. That "first practical diffusion-based online SE" claim is plausible within the cited literature, though I would not treat it as established beyond these experiments.

The paper does well: the training loss matches the inference input distribution, the latency/performance trade-off is demonstrated across buffer sizes, and the baselines use the same architecture with the same reduced capacity. The comparison against Demucs is a nice sanity check.

Soft spots, in order of size. First, the equivalence between one joint buffer update and B separate reverse steps is asserted, not derived. The paper trains the network on buffer inputs with mixed diffusion times, so the joint update is at least learned, but there is no ablation showing that buffer-based inference behaves like the baseline at the same N, and no analysis of how the score estimate depends on the other frames' noise levels. I would like to see a per-frame reverse-step counter or an ablation where the buffer is frozen. This is the main technical concern, but it is addressable. Second, WVMOS values are clamped to 1.0 when the network predicts negative MOS; that should be reported and possibly handled differently. Third, there are no error bars or significance tests; given the 0.05 PESQ difference at N=60, that is worth noting. Fourth, the "first practical" label could use a broader literature check, but within the paper's own scope it is fine.

Overall, the core latency claim is credible, the code release is a real asset, and the weaknesses are not fatal. This deserves a serious referee; I would encourage revision rather than rejection. If you are working on streaming SE, cite it. I would bring it to our reading group as a practical example of diffusion adaptation, but with the caveat that the theory is thin.

Referee Report

4 major / 5 minor

Summary. The paper proposes Diffusion Buffer (DB), an adaptation of score-based diffusion models to online speech enhancement. The method maintains a buffer of the most recent B STFT frames, assigns a different diffusion time step to each frame (noisier toward the present), and at each hop performs a single score-model call that jointly updates all frames in the buffer. One fully denoised past frame is then emitted, giving an algorithmic latency of h_s times B (320-960 ms for the tested settings). The authors train the score model with a denoising score matching loss on filtered EARS-WHAM, compare DB-OUVE and DB-BBED against their offline counterparts and Demucs, and report RTF below 1 on an RTX 4080. The paper claims this is the first practical diffusion-based online speech enhancement system.

Significance. If the central mechanism is valid, the paper would be a useful engineering contribution: it demonstrates that a diffusion-based enhancer can run online on consumer hardware with sub-second latency, with quality close to offline diffusion baselines. The release of source code and the precise RTF and latency measurements are strengths that support reproducibility. However, the significance depends on an unverified equivalence between one joint buffer update and B independent reverse steps per frame. The paper currently provides no derivation or ablation for this equivalence, and the algorithm has a dimension mismatch and an undefined update operator. These are not merely presentation issues; they affect whether the latency/quality trade-off claimed in Figure 2 is actually established. I therefore view the contribution as promising but not yet fully supported.

major comments (4)

[Section 3.2, Algorithm 1 line 7] The reverse update operator u_theta is never defined. The algorithm states only that V_t is assigned u_theta(V_t, Y_c, vec t), but no discretization of the reverse SDE (4) is given. Moreover, Section 4.2 states that the network output is cropped to the last B frames, i.e., its shape is F x B, while V_t is a buffer of K frames. Assigning an F x B tensor to an F x K buffer is dimensionally inconsistent. Please specify the exact reverse update (e.g., an Euler-Maruyama step) and state explicitly which frames of the buffer are replaced and which are left unchanged.
[Section 3.2 and Eq. (7)] The claim that one joint buffer update is equivalent to advancing each frame by one reverse step is not established. In training, Eq. (7) is a per-frame denoising score matching objective evaluated at a single diffusion time per frame. At inference, the network is called once on B frames that are at different diffusion times simultaneously. Because the network is convolutional, the score estimate for a given frame can depend on the noise levels and contents of the other frames in the buffer; it is therefore a context-dependent posterior estimate, not generally the marginal score required by Eq. (4). The comparison in Figure 2 between DB-B and N-step baselines at the same N is consequently not yet justified. Please provide a derivation or, failing that, an ablation that isolates the effect of joint mixed-time updates (e.g., comparing against a per-frame sequential update with the same number of calls).
[Section 3.2, Algorithm 1 line 6] Line 6 adds noise to the incoming frame R as R + sigma_tB Z and states that this makes R follow the perturbation kernel (2). However, Eq. (2) defines the perturbation kernel with a mean mu_t(X0,Y), and for both OUVE and BBED the mean evolution is not equal to the noisy frame Y. For BBED, for example, the mean at a finite time t contains a contribution from the clean signal X0. Since X0 is unavailable at inference, the initialization is an approximation that should be stated explicitly and validated; otherwise the reverse process starts from a distribution that is not the intended prior.
[Section 5, Figure 2] The reported performance margins are small - the claimed advantage at 60 reverse steps is 0.05 PESQ, and the WVMOS differences are described as 'on par' - but no error bars, confidence intervals, or significance tests are provided. Without a measure of variance across the test set, a 0.05 PESQ difference is within typical run-to-run or test-item variability and does not support the conclusion that DB methods 'marginally outperform' their baselines. Please report per-item statistics or a significance test.

minor comments (5)

[Abstract and Introduction] There are typographical errors: 'ouput' in the abstract and 'beginng' in Section 3; these should be corrected.
[Section 4.3] The WVMOS metric is modified by clamping negative values to 1.0. Please justify this procedure and also report the unclamped WVMOS values, since clamping can change the ordering of methods when many utterances are affected.
[Section 4.2] The description of the time-embedding adaptation is brief: 'we use a Conv2D layer with stride to match the channel and frame dimension to the feature.' Please specify the kernel size, stride, and how the per-frame time steps are arranged as an input tensor.
[Algorithm 1] The initialization 'Y_c <- last K received frames' and the popping/appending in lines 5-6 should be described more precisely: after popping and appending, does Y_c contain the current frame, and is the zero-padding at stream onset handled identically during training and inference?
[Section 5] The ESTOI comparison with Demucs is mentioned in the text ('differ only by 0.01') but no ESTOI values are plotted or given in a table; please report the actual numbers.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central latency/quality claims are empirical measurements, and the self-citations are parameter choices and baseline comparisons, not load-bearing reductions.

full rationale

The paper's core claims are empirical rather than derived from fitted constants. The Diffusion Buffer construction (Eq. 5, Algorithm 1) is a definition, not a prediction: latency h_s*B is algorithmic by design, and the reported PESQ/WVMOS gains at B=60 are measured on a held-out EARS-WHAM test partition with the code released. The SDE drift/diffusion parameters are taken from the authors' prior analysis [24] rather than re-fitted to the test set, and the same parameterizations are applied identically to the DB variants and the vanilla baselines; citing one's own earlier parameter analysis is therefore a normal comparison choice, not a self-citation that carries the argument. The training loss (Eq. 7) is standard denoising score matching, with no target metric encoded in the loss. The one methodological risk is that Algorithm 1's joint update treats one network call as advancing every buffer frame by one reverse step despite mixed diffusion times; the paper gives no proof that the jointly conditioned score equals the per-frame marginal score. That is an unverified modeling approximation, and a potential correctness weakness, but it is not circular: the paper does not define the network output as the reverse step, and the performance numbers are not forced by the definition of the buffer. No step in the claimed derivation reduces to its own input.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical entities. It does introduce a new algorithmic structure (the Diffusion Buffer) and relies on a training-time noise schedule plus the hardware measurement for the real-time claim.

free parameters (3)

Buffer size B = 10, 20, 30, 60
Chosen by hand as the key latency/quality knob; directly sets algorithmic latency h_s*B and the number of reverse steps per frame. Not fitted to the test set but is the main tunable in the paper.
WVMOS clamp threshold = 1.0
Negative WVMOS predictions are reset to 1.0 (Section 4.3), an arbitrary post-processing choice that can inflate the reported WVMOS for poor-quality outputs.
Smallest diffusion time-step epsilon = not specified
Mentioned as epsilon > 0 in Section 3.1 but no value is given; a hyperparameter that must be set at training time.

assumptions (4)

standard math Anderson's reverse-time SDE theorem relating forward and reverse diffusions
Invoked in Section 2.1 (Equation 4) to justify the reverse process used for inference.
domain assumption The score model trained on random time-step sequences generalizes to the fixed ascending schedule used at inference
Training (Section 3.1) samples t uniformly at random; inference (Algorithm 1) uses a fixed schedule. The paper does not analyze the distribution shift.
domain assumption A single forward pass over a 128-frame context on an NVIDIA RTX 4080 takes about 14 ms, below the 16 ms hop
Reported in Section 5 as a hardware measurement; the online claim depends on this specific hardware and model size.
ad hoc to paper Jointly denoising buffer frames at different diffusion times in one network call is a valid approximation of the reverse SDE
This is the core approximation of the method, asserted in Section 3.2 and Algorithm 1, with no error analysis.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Diffusion Buffer: Online Diffusion-based Speech Enhancement with Sub-Second Latency." pith.science (2026). https://pith.science/paper/IWAZO5T4

@misc{pith2026250602908,
  author       = {Pith},
  title        = {Pith review of: Diffusion Buffer: Online Diffusion-based Speech Enhancement with Sub-Second Latency},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IWAZO5T4}},
  note         = {Machine review of arXiv:2506.02908}
}

read the original abstract

Diffusion models are a class of generative models that have been recently used for speech enhancement with remarkable success but are computationally expensive at inference time. Therefore, these models are impractical for processing streaming data in real-time. In this work, we adapt a sliding window diffusion framework to the speech enhancement task. Our approach progressively corrupts speech signals through time, assigning more noise to frames close to the present in a buffer. This approach outputs denoised frames with a delay proportional to the chosen buffer size, enabling a trade-off between performance and latency. Empirical results demonstrate that our method outperforms standard diffusion models and runs efficiently on a GPU, achieving an input-output latency in the order of 0.3 to 1 seconds. This marks the first practical diffusion-based solution for online speech enhancement.

Figures

Figures reproduced from arXiv: 2506.02908 by the authors.

**Figure 1.** The DB scheme for streamed data. The top part shows the noisy stream Ys. The bottom presents the enhanced signal. In V −→t , we see exponentially increasing noise in the DB. The output frame is indicated in red at the beginning of the DB. such as [12, 7] on streaming data requires solving the reverse process within the time constraints defined above. Since the reverse SDE calls the score model several times, current… view at source ↗

**Figure 1.** Let Ys be an infinite stream of data in the STFT domain and assume we receive the frame R in the for-loop of Algorithm 1. We then add in line 6 an amount of Gaussian noise so that the random variable R′ = R + σtB Z follows the perturbation kernel (2), meaning R′ is at diffusion time-step tB. As usual, we then run one reverse step for R′ where the score function is approximated by the trained score model. Consequ… view at source ↗

**Figure 2.** Comparing OUVE, BBED against their DB versions on the filtered EARS-WHAM test set. We experimented with different number of reverse steps. This is also the buffer size B for the DB versions with algorithmic latency equal to hs · B. 4.3. Datasets and metrics We use the publicly available dataset EARS-WHAM [21], originally recorded in 48 kHz. We downsampled the data to 16 kHz and filter it by removing files of type … view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

34 extracted references · 34 canonical work pages

[1]

Online SE, which refers to enhancing the speech signal with a limited latency as it is being received, holds significant importance in a variety of applications

Introduction The objective of speech enhancement (SE) is to retrieve the original clean speech signal from a noisy mixture that is af- fected by additive environmental noise [1]. Online SE, which refers to enhancing the speech signal with a limited latency as it is being received, holds significant importance in a variety of applications. The ability to p...

work page
[2]

Background We consider a speech enhancement task for speech signals cor- rupted by additive noise. The input is a noisy mixtureY= S+Ein the complex short-time Fourier transform (STFT) do- main, consisting of a clean speech signalS∈C F×K and envi- ronmental noiseE∈C F×K , whereKandFare the number arXiv:2506.02908v2 [eess.AS] 12 Sep 2025 of frames and numbe...

work page Pith review arXiv 2025
[3]

Proposed method: Diffusion Buffer Inspired by [10, 11], we propose to align the diffusion time- steps with the time axis of the noisy mixture. To this end, we introduce a Diffusion Buffer containing the lastBframes, whereas the current frameR∈C F×1 is placed at the end of this buffer and past frames are closer to the beginning of the buffer. Within this b...

work page
[4]

whisper”, “highpitch

Experimental Setup 4.1. Data representation Each audio input, sampled at 16 kHz, is converted to a complex- valued STFT. As in [7], we use a window size of 510 samples (32 ms), a hop length of 256 samples (h s = 16ms), and a pe- riodic Hann window. The input to the score model is cropped randomly toK= 128time frames, resulting in approximately 2 seconds o...

work page
[5]

First, we discuss, as in Section 2.2 why the vanilla diffu- sion models OUVE and BBED do not operate in real-time on streamable data as their RTFs≫1

Results We train the proposed DB-OUVE and DB-BBED on the fil- tered 16 kHz EARS-WHAM dataset and experimented with the buffer lengthB= 5,10,20,30,60. First, we discuss, as in Section 2.2 why the vanilla diffu- sion models OUVE and BBED do not operate in real-time on streamable data as their RTFs≫1. Compared to their original implementation in [7, 12] wher...

work page
[6]

Inspired by [10, 11], we denoise noisy frames through physical time

Conclusion In this work, we successfully adapted score-based diffusion models to process streamed audio data online. Inspired by [10, 11], we denoise noisy frames through physical time. Unlike standard diffusion models that do not take the physical time axis into consideration, we introduce the Diffusion Buffer, where the position of each frame is taken i...

work page
[7]

Acknowledgments Funded by the Deutsche Forschungsgemeinschaft (DFG, Ger- man Research Foundation) – 545210893, 498394658. Funded by the Federal Ministry for Economic Affairs and Climate Action (Bundesministerium f ¨ur Wirtschaft und Kli- maschutz), Zentrales Innovationsprogramm Mittelstand (ZIM), Germany, within the project FKZ KK5528802VW4. The authors g...

work page
[8]

Universal score-based speech enhancement with high content preservation,

R. Scheibler, Y . Fujita, Y . Shirahata, and T. Komatsu, “Universal score-based speech enhancement with high content preservation,” IEEE Int. Conf. on Acoustics, Speech and Signal Proc. (ICASSP), 2024

work page 2024

Show all 34 references

[9]

R. C. Hendriks, T. Gerkmann, and J. Jensen,DFT-domain based single-microphone noise reduction for speech enhancement: A survey of the state-of-the-art. Morgan & Claypool, 2013

2013
[10]

Spectral masking and filtering,

T. Gerkmann and E. Vincent, “Spectral masking and filtering,” in Audio Source Separation and Speech Enhancement, E. Vincent, T. Virtanen, and S. Gannot, Eds. John Wiley & Sons, 2018

2018
[11]

Supervised speech separation based on deep learning: An overview,

D. Wang and J. Chen, “Supervised speech separation based on deep learning: An overview,”IEEE Trans. on Audio, Speech, and Language Proc. (TASLP), vol. 26, no. 10, pp. 1702–1726, 2018

2018
[12]

Conv-TasNet: Surpassing ideal time–frequency magnitude masking for speech separation,

Y . Luo and N. Mesgarani, “Conv-TasNet: Surpassing ideal time–frequency magnitude masking for speech separation,”IEEE Trans. on Audio, Speech, and Language Proc. (TASLP), vol. 27, no. 8, pp. 1256–1266, 2019

2019
[13]

An investigation of incorporating mamba for speech enhancement,

R. Chao, W.-H. Cheng, M. La Quatra, S. M. Siniscalchi, C.-H. H. Yang, S.-W. Fu, and Y . Tsao, “An investigation of incorporating mamba for speech enhancement,”IEEE Spoken Language Tech- nology Workshop, 2024

2024
[14]

Denoising diffusion probabilis- tic models,

J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilis- tic models,”Advances in Neural Inf. Proc. Systems (NeurIPS), vol. 33, pp. 6840–6851, 2020

2020
[15]

Speech enhancement and dereverberation with diffusion-based generative models,

J. Richter, S. Welker, J.-M. Lemercier, B. Lay, and T. Gerkmann, “Speech enhancement and dereverberation with diffusion-based generative models,”IEEE Trans. on Audio, Speech, and Language Proc. (TASLP), 2023

2023
[16]

Reverse-time diffusion equation models,

B. D. Anderson, “Reverse-time diffusion equation models,” Stochastic Processes and their Applications, vol. 12, no. 3, pp. 313–326, 1982

1982
[17]

Unsupervised speech enhancement with diffusion-based generative models,

B. Nortier, M. Sadeghi, and R. Serizel, “Unsupervised speech enhancement with diffusion-based generative models,”IEEE Int. Conf. on Acoustics, Speech and Signal Proc. (ICASSP), 2023

2023
[18]

Rolling dif- fusion models,

D. Ruhe, J. Heek, T. Salimans, and E. Hoogeboom, “Rolling dif- fusion models,”Int. Conf. on Machine Learning (ICML), 2024

2024
[19]

Fifo-diffusion: Generating infinite videos from text without training,

J. Kim, J. Kang, J. Choi, and B. Han, “Fifo-diffusion: Generating infinite videos from text without training,”Advances in Neural Inf. Proc. Systems (NeurIPS), 2024

2024
[20]

Reducing the prior mismatch of stochastic differential equations for diffusion- based speech enhancement,

B. Lay, S. Welker, J. Richter, and T. Gerkamnn, “Reducing the prior mismatch of stochastic differential equations for diffusion- based speech enhancement,”Interspeech, 2023

2023
[21]

Karatzas and S

I. Karatzas and S. E. Shreve,Brownian Motion and Stochastic Calculus, 2nd ed. Springer, 1996

1996
[22]

Rudin,Real and Complex Analysis, 3rd ed

W. Rudin,Real and Complex Analysis, 3rd ed. McGraw-Hill, Inc., 1987

1987
[23]

S ¨arkk¨a and A

S. S ¨arkk¨a and A. Solin,Applied Stochastic Differential Equations. Cambridge University Press, 2019, no. 10

2019
[24]

An analysis of the variance of diffusion-based speech enhancement,

B. Lay and T. Gerkmann, “An analysis of the variance of diffusion-based speech enhancement,”Interspeech, 2024

2024
[25]

Speech enhancement with score-based generative models in the complex STFT do- main,

S. Welker, J. Richter, and T. Gerkmann, “Speech enhancement with score-based generative models in the complex STFT do- main,”Interspeech, 2022

2022
[26]

The SDE parameterization of BBED has a larger variance schedule compared to OUVE, as shown in [24]

values differ only by 0.01. The SDE parameterization of BBED has a larger variance schedule compared to OUVE, as shown in [24]. A higher vari- ance schedule reduces the number of reverse steps needed for enhancement, which is why BBED outperforms OUVE in terms of PESQ and WVMO...
[27]

Empirical distributions of DFT- domain speech coefficients based on estimated speech variances,

T. Gerkmann and R. Martin, “Empirical distributions of DFT- domain speech coefficients based on estimated speech variances,” Int. Workshop on Acoustic Echo and Noise Control, 2010

2010
[28]

Score-based generative modeling through stochas- tic differential equations,

Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochas- tic differential equations,”Int. Conf. on Learning Representations (ICLR), 2021

2021
[29]

Adam: A method for stochastic opti- mization,

D. P. Kingma and J. Ba, “Adam: A method for stochastic opti- mization,”Int. Conf. on Learning Representations (ICLR), 2015

2015
[30]

EARS: An anechoic fullband speech dataset benchmarked for speech enhancement and derever- beration,

J. Richter, Y .-C. Wu, S. Krenn, S. Welker, B. Lay, S. Watan- abe, A. Richard, and T. Gerkmann, “EARS: An anechoic fullband speech dataset benchmarked for speech enhancement and derever- beration,”Interspeech, 2024

2024
[31]

Perceptual eval- uation of speech quality (PESQ) - a new method for speech qual- ity assessment of telephone networks and codecs,

A. Rix, J. Beerends, M. Hollier, and A. Hekstra, “Perceptual eval- uation of speech quality (PESQ) - a new method for speech qual- ity assessment of telephone networks and codecs,”IEEE Int. Conf. on Acoustics, Speech and Signal Proc. (ICASSP), vol. 2, pp. 749– 752, 2001

2001
[32]

Hifi++: A unified framework for bandwidth extension and speech enhance- ment,

P. Andreev, A. Alanov, O. Ivanov, and D. Vetrov, “Hifi++: A unified framework for bandwidth extension and speech enhance- ment,”IEEE Int. Conf. on Acoustics, Speech and Signal Proc. (ICASSP), 2023

2023
[33]

Real time speech en- hancement in the waveform domain,

A. Defossez, G. Synnaeve, and Y . Adi, “Real time speech en- hancement in the waveform domain,” 2020

2020
[34]

An algorithm for predicting the intel- ligibility of speech masked by modulated noise maskers,

J. Jensen and C. H. Taal, “An algorithm for predicting the intel- ligibility of speech masked by modulated noise maskers,”IEEE Trans. on Audio, Speech, and Language Proc. (TASLP), vol. 24, no. 11, pp. 2009–2022, 2016

2009

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Online SE, which refers to enhancing the speech signal with a limited latency as it is being received, holds significant importance in a variety of applications

Introduction The objective of speech enhancement (SE) is to retrieve the original clean speech signal from a noisy mixture that is af- fected by additive environmental noise [1]. Online SE, which refers to enhancing the speech signal with a limited latency as it is being received, holds significant importance in a variety of applications. The ability to p...

work page

[2] [2]

Background We consider a speech enhancement task for speech signals cor- rupted by additive noise. The input is a noisy mixtureY= S+Ein the complex short-time Fourier transform (STFT) do- main, consisting of a clean speech signalS∈C F×K and envi- ronmental noiseE∈C F×K , whereKandFare the number arXiv:2506.02908v2 [eess.AS] 12 Sep 2025 of frames and numbe...

work page Pith review arXiv 2025

[3] [3]

Proposed method: Diffusion Buffer Inspired by [10, 11], we propose to align the diffusion time- steps with the time axis of the noisy mixture. To this end, we introduce a Diffusion Buffer containing the lastBframes, whereas the current frameR∈C F×1 is placed at the end of this buffer and past frames are closer to the beginning of the buffer. Within this b...

work page

[4] [4]

whisper”, “highpitch

Experimental Setup 4.1. Data representation Each audio input, sampled at 16 kHz, is converted to a complex- valued STFT. As in [7], we use a window size of 510 samples (32 ms), a hop length of 256 samples (h s = 16ms), and a pe- riodic Hann window. The input to the score model is cropped randomly toK= 128time frames, resulting in approximately 2 seconds o...

work page

[5] [5]

First, we discuss, as in Section 2.2 why the vanilla diffu- sion models OUVE and BBED do not operate in real-time on streamable data as their RTFs≫1

Results We train the proposed DB-OUVE and DB-BBED on the fil- tered 16 kHz EARS-WHAM dataset and experimented with the buffer lengthB= 5,10,20,30,60. First, we discuss, as in Section 2.2 why the vanilla diffu- sion models OUVE and BBED do not operate in real-time on streamable data as their RTFs≫1. Compared to their original implementation in [7, 12] wher...

work page

[6] [6]

Inspired by [10, 11], we denoise noisy frames through physical time

Conclusion In this work, we successfully adapted score-based diffusion models to process streamed audio data online. Inspired by [10, 11], we denoise noisy frames through physical time. Unlike standard diffusion models that do not take the physical time axis into consideration, we introduce the Diffusion Buffer, where the position of each frame is taken i...

work page

[7] [7]

Acknowledgments Funded by the Deutsche Forschungsgemeinschaft (DFG, Ger- man Research Foundation) – 545210893, 498394658. Funded by the Federal Ministry for Economic Affairs and Climate Action (Bundesministerium f ¨ur Wirtschaft und Kli- maschutz), Zentrales Innovationsprogramm Mittelstand (ZIM), Germany, within the project FKZ KK5528802VW4. The authors g...

work page

[8] [8]

Universal score-based speech enhancement with high content preservation,

R. Scheibler, Y . Fujita, Y . Shirahata, and T. Komatsu, “Universal score-based speech enhancement with high content preservation,” IEEE Int. Conf. on Acoustics, Speech and Signal Proc. (ICASSP), 2024

work page 2024