Pith. sign in

REVIEW 4 major objections 5 minor 71 references

Exploring Length Generalization For Transformer-based Speech Enhancement

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Transformer speech enhancement models can be trained on 1- or 2-second clips and still generalize to 20-second noisy utterances, provided attention scores are biased by a learned linear function of frame distance.

desk verdict LearnLin itself is already in the authors' INTERSPEECH 2024 paper; this version's value is the systematic PE comparison, which is solid but lacks error bars, code, and the learned beta values. read the letter →

arxiv 2506.06697 v1 pith:53ZW3QTU submitted 2025-06-07 eess.AS

Qiquan Zhang , Hongxu Zhu , Xinyuan Qian , Eliathamby Ambikairajah , Haizhou Li This is my paper
classification eess.AS
keywords lengthgeneralizationspeechenhancementTransformerrelativepositionencodingpositionalself-attentionLearnLinutterance-lengthextrapolation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether Transformer-based speech enhancement models trained on short 1- or 2-second clips can still perform well on noisy speech that is 5 to 20 seconds long, and argues the answer is yes when positions are encoded as relative distances. Across four training objectives and five quality metrics, absolute position embeddings fail to generalize while relative position encodings mostly succeed. The paper's proposed scheme, LearnLin, adds to each attention head a bias equal to one learned scalar times the frame distance, and reports that it matches or beats every other positional encoding while using the fewest parameters. If this holds, the length gap is less a computational obstacle to engineer around than a position-encoding choice that a single learned slope per head can resolve.

What carries the argument

The central object is LearnLin, a relative position bias $P^h_{i,j} = \beta_h \cdot |i - j|$ indexed by attention head $h$, with one learnable scalar $\beta_h$ per head shared across all Transformer layers. It injects position information by addition before softmax in the scaled dot-product attention, so the bias directly shapes which time frames attend to which. This single linear distance penalty carries the argument: it is simpler than the sigmoidal DA-Bias, the kernelized KERPLE, the radial-basis TISA, the bucketed T5-Bias, or the Gaussian Gauss-Bias, yet the paper finds it damps the effect of utterance length more effectively across all four training objectives and both causal and non-causal configurations.

What would settle it

Train LearnLin on 1-second clips, record the learned $\beta_h$ values for all heads, and inspect attention maps on 20- and 30-second inputs: if any head's attention saturates on distant frames, becomes nearly uniform, or if the model's PESQ and ESTOI scores on 30-second utterances fall below the no-position-encoding baseline, the claimed length generalization fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that a head-wise learnable linear bias on relative frame distance enables Transformer speech enhancement models to extrapolate from short training utterances to much longer test utterances. Specifically, for each attention head $h$, the position bias is $P^h_{i,j} = \beta_h \cdot |i - j|$, where $\beta_h$ is one trainable scalar, shared across layers, and the bias is added to the raw attention scores before softmax. A positive $\beta_h$ makes the head favor long-range dependencies and a negative one favors short-range dependencies, so a few scalars let different heads specialize. In experiments on four training targets (magnitude spectrum, ideal ratio mask, phase-sensitive mask, and complex ideal ratio mask) and five metrics (PESQ, ESTOI, CSIG, CBAK, COVL), the paper reports that LearnLin gives the best or tied-best scores at 5s, 10s, 15s, and 20s test lengths in nearly all settings, and that its margin over other methods grows as the test utterance gets longer.

Load-bearing premise

The load-bearing premise is that a single scalar $\beta_h$ learned only from 1- or 2-second clips, and shared across all layers, continues to produce sensible attention weights at frame distances up to roughly 1,250 that were never seen during training, rather than over-amplifying or suppressing long-range attention.

Editorial extensions

If this is right

  • A model trained only on 1-second or 2-second noisy-clean pairs can enhance 20-second utterances with quality scores that match or exceed all seven compared positional encodings, across all four training objectives.
  • Absolute position embeddings, both sinusoidal and learned, collapse on long inputs and in some cases score below unprocessed noisy speech, while every relative position encoding generalizes better.
  • For LearnLin, longer inputs help rather than hurt: scores at 10s to 20s are generally higher than at the training length, because more context is available to each frame.
  • Chunk-based processing with 1-second segments loses context and underperforms full-length extrapolation, even with 50% overlap, so the length-generalization ability itself is what delivers the gains.
  • A single scalar per attention head is sufficient; the extra parameters and nonlinearities of TISA, KERPLE, and DA-Bias do not translate into better speech enhancement, and LearnLin's margin over them grows with test length.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the linear bias is an extrapolation gamble, because the paper does not report the learned $\beta_h$ values; a natural next check is whether heads trained on 1-second clips keep their intended long-range or short-range profiles when evaluated at roughly 1,250 frames, since a large positive slope could make distant frames dominate attention in unintended ways.
  • Editorial inference: because speech spectrograms have strong local temporal structure, LearnLin's success suggests a monotonic distance penalty captures most of the position information needed for enhancement, a hypothesis that could be tested by ablating content-based attention and keeping only the distance bias.
  • Editorial inference: the same one-scalar-per-head recipe may transfer to other frame-based audio tasks such as speaker separation or speech recognition, where long-context extrapolation is also practically valuable, but the present study only demonstrates results for speech enhancement.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies whether Transformer-based speech enhancement models trained on short utterances (1s or 2s) can generalize to longer inputs (5s, 10s, 15s, 20s) at test time. It compares seven existing positional encoding methods, including absolute (Sinusoidal, BERT-Pos) and relative (Gauss-Bias, T5-Bias, TISA, DA-Bias, KERPLE, RoPE) schemes, and proposes a new scheme called LearnLin, in which the attention bias for head h is P^h_{i,j} = β_h |i − j| with one learnable scalar per head shared across layers (Eq. 15). The evaluation covers four training objectives (MS, IRM, PSM, cIRM), five metrics (PESQ, ESTOI, CSIG, CBAK, COVL), causal and non-causal configurations, and chunk-based processing. The headline findings are that absolute positional encoding methods collapse on long utterances, relative positional encoding methods are more robust, and LearnLin is usually the best or tied with the best method.

Significance. If the empirical claims are taken at face value, this is a practically useful result: a single scalar per attention head added as a linear relative-distance bias is a very low-cost way to keep Transformer speech enhancement models functional on inputs longer than those seen in training. The paper's strengths are the breadth of the comparison (seven baseline positional encodings, four training objectives, five metrics, causal and chunk settings) and the clear demonstration that absolute positional encodings fail in the length-generalization regime. However, the absence of variance estimates or significance testing, the incomplete cIRM coverage, and the unverified mechanism attributed to LearnLin substantially temper the strength of the 'excellent length generalization' and 'superior' claims. The qualitative ranking visible in the tables is credible, but the quantitative comparative claims need additional support before the paper can be accepted as is.

major comments (4)
  1. [§VII-B, Tables II–X] The paper's comparative claims that LearnLin 'always performs best' and is 'superior' are supported only by point estimates. No number of independent runs, standard deviations, confidence intervals, or significance tests are reported. Many of the decisive gaps are within 0.01–0.03 in PESQ, CSIG, or COVL; for example, Table VI (5s row) shows LearnLin PESQ 2.74 vs. DA-Bias 2.73, and Table VIII (5s row) shows TISA PESQ 2.76 vs. LearnLin 2.75 and TISA CSIG 3.41 vs. LearnLin 3.40. With no variance measure, these margins cannot support the stated ranking. Please state the number of independent runs, report variance or confidence intervals, or explicitly discuss effect sizes relative to the uncertainty.
  2. [§V-B, Eq. (15)] The LearnLin mechanism is asserted but not verified. The paper never reports the learned β_h values, their distribution across heads, their variability across runs, or any diagnostic of attention as a function of relative distance. This matters because the bias is an unbounded linear function of |i − j|: at a 20s input (roughly 1,250 frames), a head with |β_h| ≥ 0.01 would add a bias above 12 to the content logits, potentially dominating the softmax. That the model performs well on 5–20s inputs is indirect evidence that the learned β_h values are benign, but the claimed 'preferences on short- or long-term dependencies' (Section V-B) and the safety of extrapolation are not established. Please report the learned β_h values and include a sensitivity/attention-distance analysis.
  3. [§VII-B, Tables II–V and VI–VIII] The 'four training objectives' coverage is incomplete. For 1s training, Table V reports cIRM only for test lengths 1s, 10s, and 20s, omitting 5s and 15s; for 2s training, no cIRM table is given at all, since Tables VI–VIII contain only MS, IRM, and PSM. The conclusion that LearnLin is superior across four training objectives therefore goes beyond the reported evidence. The authors should supply the missing cIRM results or qualify the claim to the objectives and lengths actually evaluated.
  4. [Table VIII, 5s row, and surrounding text] The text states that aside from TISA showing 0.02 CSIG and 0.01 COVL gains over LearnLin at the 5s test length, LearnLin 'always performs best in all other test cases'. The table actually shows TISA also with higher PESQ (2.76 vs. 2.75) and DA-Bias with equal/higher CSIG (3.41 vs. 3.40), and the TISA CSIG gain is 0.01, not 0.02. Please correct this description and re-examine all similar 'always best' statements across Tables VI–VIII.
minor comments (5)
  1. [§VII-B] The sentence 'Tables VI–IV report the comparison results' should read 'Tables VI–VIII'; the typo appears in the paragraph introducing the 2s training results.
  2. [Figs. 5–7 and Table VII] Notation is inconsistent: the figure legends use 'TIAS' instead of 'TISA', and Table VII uses 'Gauss. Bias' and 'T5 Bias' while other tables use 'Gauss-Bias' and 'T5-Bias'. Please unify the naming.
  3. [§III-B, Eq. (6)] The non-causal variant of KERPLE is not described. Equation (6) gives the position bias, but the paper does not state how the original causal formulation is adapted to the non-causal speech enhancement setting. Please specify the exact modification.
  4. [§VI-B, Table I] Table I reports the number of trainable parameters for BERT-Pos as L'·d_model, but the value of L' used in the experiments is never stated. Please report it, since the parameter count of that baseline depends on it.
  5. [General] No code or trained checkpoint release is mentioned. Given the density of the tables and the small performance margins, releasing code would substantially improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: LearnLin is an empirically evaluated proposal whose length-generalization claim is tested on held-out utterances longer than the training ones.

full rationale

The paper's central claim is that the LearnLin position bias P^h_{i,j} = β_h · |i − j| (Eq. 15) helps Transformer speech enhancement models generalize from 1s/2s training utterances to 5s–20s test utterances. This claim is supported by direct experiments on held-out noisy mixtures scored with PESQ, ESTOI, CSIG, CBAK, and COVL (Tables II–X). No fitted β_h value is folded into a 'prediction'; the β_h parameters are trained only on short clips, and the test results are external measurements, not algebraic consequences of the definition. The observation that LearnLin contains the term w|i−j| from DA-Bias is an algebraic remark, not a derivation of the empirical result. The only self-citation, [17], is used to say 'we extend our previous work' and does not carry any load-bearing assumption. The absence of reported β_h values or attention-distance diagnostics is a verification gap, not circularity. The derivation chain therefore does not reduce to its own inputs.

Assumptions & free parameters 1 free parameters · 2 assumptions · 1 invented entities

The ledger is small because the paper is an empirical comparison rather than a derivation. The main added assumption is the adequacy of a one-scalar-per-head linear bias, and the paper does not analyze the fitted beta_h values or their stability across lengths.

free parameters (1)
  • LearnLin head-wise scale beta_h = 8 learned scalars, values not reported
    One trainable scalar per attention head multiplies the relative distance in Eq. 15. The sign and magnitude of these scalars are the claimed mechanism for short- versus long-range preference, but the learned values and their extrapolation behavior are not analyzed.
assumptions (2)
  • domain assumption Adding a head-wise linear relative-position bias to raw attention scores before softmax is sufficient for length generalization in Transformer speech enhancement.
    Section V-B, Eq. 14-15. This is tested empirically, not proven; there is no derivation or bound showing why the bias extrapolates to unseen distances.
  • domain assumption Attention heads have stable short- or long-range preferences that can be captured by a single signed scalar per head, shared across all layers.
    Section V-B, paragraph following Eq. 15. The paper does not report learned beta_h values or verify that the preferences learned on 1s/2s clips persist at 20s.
invented entities (1)
  • LearnLin position bias P^h_{i,j} = beta_h * |i - j|
    purpose: New relative-position encoding term for self-attention, using one learned scalar per head to scale distance.
    The bias is introduced as a design choice with no external theoretical or empirical support beyond the paper's own experiments. Learned beta_h values are not reported, so the falsifiable handle is limited to the aggregate metrics in the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploring Length Generalization For Transformer-based Speech Enhancement." pith.science (2026). https://pith.science/paper/53ZW3QTU

@misc{pith2026250606697,
  author       = {Pith},
  title        = {Pith review of: Exploring Length Generalization For Transformer-based Speech Enhancement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/53ZW3QTU}},
  note         = {Machine review of arXiv:2506.06697}
}
read the original abstract

Transformer network architecture has proven effective in speech enhancement. However, as its core module, self-attention suffers from quadratic complexity, making it infeasible for training on long speech utterances. In practical scenarios, speech enhancement models are often required to perform on noisy speech at run-time that is substantially longer than the training utterances. It remains a challenge how a Transformer-based speech enhancement model can generalize to long speech utterances. In this paper, extensive empirical studies are conducted to explore the model's length generalization ability. In particular, we conduct speech enhancement experiments on four training objectives and evaluate with five metrics. Our studies establish that positional encoding is an effective instrument to dampen the effect of utterance length on speech enhancement. We first explore several existing positional encoding methods, and the results show that relative positional encoding methods exhibit a better length generalization property than absolute positional encoding methods. Additionally, we also explore a simpler and more effective positional encoding scheme, i.e. LearnLin, that uses only one trainable parameter for each attention head to scale the real relative position between time frames, which learns the different preferences on short- or long-term dependencies of these heads. The results demonstrate that our proposal exhibits excellent length generalization ability with comparable or superior performance than other state-of-the-art positional encoding strategies.

Figures

Figures reproduced from arXiv: 2506.06697 by the authors.

Figure 1
Figure 1. Length generalization in speech enhancement: The ability to learn from short speech samples (training set) to generalize to longer speech samples (test set). Monaural speech enhancement has been an active research direction in speech signal processing for decades, leading to the development of a variety of methods. Conventional speech enhancement techniques [1] mainly involve spectral subtraction [2], statistical mo… view at source ↗
Figure 2
Figure 2. Illustration of (a) the position-aware Transformer backbone network and (b) the position-aware self-attention. ⊕ denotes the element-wise summation operation. V. SPEECH ENHANCEMENT WITH POSITION-AWARE TRANSFORMER A. Network Architecture [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Raw dot-product attention scores are biased by adding a learnable position-aware bias. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The length distribution over the 1000 speech utterances in the validation dataset. perform validation experiments, we randomly selected 1 000 clean speech utterances (over 2 seconds) and noise clips from the clean speech and noise data to create a validation set of 1 0…
Figure 5
Figure 5. Figure 5: The (a) training loss and (b) validation loss of the [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: The (a) training loss and (b) validation loss of the [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: The (a) training loss and (b) validation loss of the [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

71 extracted references · 62 canonical work pages

  1. [17]

    An exploration of length generalization in transformer-based speech enhancement,

    Q. Zhang, H. Zhu, X. Qian, E. Ambikairajah, and H. Li, “An exploration of length generalization in transformer-based speech enhancement,” in Proc. INTERSPEECH, 2024, pp. 1725–1729

    work page 2024

  2. [1]

    P. C. Loizou, Speech Enhancement: Theory and Practice , 2nd ed. Boca Raton, FL, USA: CRC Press, Inc., 2013

    work page 2013

  3. [2]

    Suppression of acoustic noise in speech using spectral subtrac- tion,

    S. Boll, “Suppression of acoustic noise in speech using spectral subtrac- tion,” IEEE Transactions on acoustics, speech, and signal processing , vol. 27, no. 2, pp. 113–120, 1979

    work page 1979

  4. [3]

    Speech Enhancement Using a Minimum Mean-Square Error Short-Time Spectral Amplitude Estimator,

    Y . Ephraim and D. Malah, “Speech Enhancement Using a Minimum Mean-Square Error Short-Time Spectral Amplitude Estimator,” IEEE Trans. Acoust., Speech, Signal Process. , vol. ASSP-32, no. 6, pp. 1109– 1121, Dec. 1984

    work page 1984

  5. [4]

    On MMSE-based estimation of amplitude and complex speech spectral coefficients under phase- uncertainty,

    M. Krawczyk-Becker and T. Gerkmann, “On MMSE-based estimation of amplitude and complex speech spectral coefficients under phase- uncertainty,” IEEE/ACM Trans. Audio, Speech, and Lang. Proc. , vol. 24, no. 12, pp. 2251–2262, 2016

    work page 2016

  6. [5]

    A novel fast nonstationary noise tracking approach based on mmse spectral power estimator,

    Q. Zhang, M. Wang, Y . Lu, L. Zhang, and M. Idrees, “A novel fast nonstationary noise tracking approach based on mmse spectral power estimator,” Digital Signal Processing , vol. 88, pp. 41–52, 2019

    work page 2019

  7. [6]

    Speech enhancement based on a priori signal to noise estimation,

    P. Scalart and J. V . Filho, “Speech enhancement based on a priori signal to noise estimation,” in Proc. ICASSP, vol. 2, 1996, pp. 629–632

    work page 1996

  8. [7]

    Supervised speech separation based on deep learning: An overview,

    D. Wang and J. Chen, “Supervised speech separation based on deep learning: An overview,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 26, no. 10, pp. 1702–1726, 2018

    2018

Show all 71 references
  1. [8]

    Deep neural network techniques for monaural speech enhancement: state of the art analysis,

    P. Ochieng, “Deep neural network techniques for monaural speech enhancement: state of the art analysis,”arXiv preprint arXiv:2212.00369, 2022

    2022 arXiv

  2. [9]

    Speech denoising in the waveform domain with self-attention,

    Z. Kong, W. Ping, A. Dantrey, and B. Catanzaro, “Speech denoising in the waveform domain with self-attention,” in Proc. ICASSP , 2022, pp. 7867–7871

    2022

  3. [10]

    Masked multi-head self-attention for causal speech enhancement,

    A. Nicolson and K. K. Paliwal, “Masked multi-head self-attention for causal speech enhancement,” Speech Communication, vol. 125, pp. 80– 96, 2020

    2020

  4. [11]

    Monaural speech derever- beration using temporal convolutional networks with self attention,

    Y . Zhao, D. Wang, B. Xu, and T. Zhang, “Monaural speech derever- beration using temporal convolutional networks with self attention,” IEEE/ACM Trans. Audio, speech, Lang. Process. , vol. 28, pp. 1598– 1607, 2020

    2020

  5. [12]

    T-GSA: Transformer with gaussian- weighted self-attention for speech enhancement,

    J. Kim, M. El-Khamy, and J. Lee, “T-GSA: Transformer with gaussian- weighted self-attention for speech enhancement,” in Proc. ICASSP , 2020, pp. 6649–6653

    2020

  6. [13]

    Noisy-reverberant speech enhancement using denseunet with time-frequency attention

    Y . Zhao and D. Wang, “Noisy-reverberant speech enhancement using denseunet with time-frequency attention.” in Proc. INTERSPEECH , 2020, pp. 3261–3265

    2020

  7. [14]

    Ripple sparse self-attention for monaural speech enhancement,

    Q. Zhang, H. Zhu, Q. Song, X. Qian, Z. Ni, and H. Li, “Ripple sparse self-attention for monaural speech enhancement,” in Proc. ICASSP , 2023, pp. 1–5

    2023

  8. [15]

    Conformer: Convolution-augmented Transformer for Speech Recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu et al. , “Conformer: Convolution-augmented Transformer for Speech Recognition,” 2020, pp. 5036–5040

    2020

  9. [16]

    Exploring length generalization in large language models,

    C. Anil, Y . Wu, A. J. Andreassen, A. Lewkowycz, V . Misra, V . V . Ramasesh, A. Slone, G. Gur-Ari, E. Dyer, and B. Neyshabur, “Exploring length generalization in large language models,” in Proc. NeurIPS, 2022

    2022

  10. [18]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proc. NeurIPS, 2017, pp. 5998–6008

    2017

  11. [19]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, P. J. Liuet al., “Exploring the limits of transfer learning with a unified text-to-text transformer.” J. Mach. Learn. Res. , vol. 21, no. 140, pp. 1–67, 2020

    2020

  12. [20]

    The case for translation-invariant self- attention in transformer-based language models,

    U. Wennberg and G. E. Henter, “The case for translation-invariant self- attention in transformer-based language models,” in Proc. ACL-IJCNLP, 2021, pp. 130–140

    2021

  13. [21]

    Kerple: Kernelized relative positional embedding for length extrapolation,

    T.-C. Chi, T.-H. Fan, P. J. Ramadge, and A. Rudnicky, “Kerple: Kernelized relative positional embedding for length extrapolation,” Proc. NeurIPS, vol. 35, pp. 8386–8399, 2022

    2022

  14. [22]

    SEGAN: Speech enhancement generative adversarial network,

    S. Pascual, A. Bonafonte, and J. Serr `a, “SEGAN: Speech enhancement generative adversarial network,” Proc. INTERSPEECH, pp. 3642–3646, 2017

    2017

  15. [23]

    Raw waveform-based speech enhancement by fully convolutional networks,

    S.-W. Fu, Y . Tsao, X. Lu, and H. Kawai, “Raw waveform-based speech enhancement by fully convolutional networks,” in Proc. APSIPA ASC , 2017, pp. 006–012

    2017

  16. [24]

    On loss functions for supervised monaural time-domain speech enhancement,

    M. Kolbæk, Z.-H. Tan, S. H. Jensen, and J. Jensen, “On loss functions for supervised monaural time-domain speech enhancement,” IEEE/ACM Trans. Audio, speech, Lang. Process. , vol. 28, pp. 825–838, 2020

    2020

  17. [25]

    End-to-end waveform utterance enhancement for direct evaluation metrics optimiza- tion by fully convolutional neural networks,

    S.-W. Fu, T.-W. Wang, Y . Tsao, X. Lu, and H. Kawai, “End-to-end waveform utterance enhancement for direct evaluation metrics optimiza- tion by fully convolutional neural networks,” IEEE/ACM Trans. Audio, Speech, and Lang. Proc. , vol. 26, no. 9, pp. 1570–1584, 2018

    2018

  18. [26]

    Real time speech enhancement in the waveform domain,

    A. Defossez, G. Synnaeve, and Y . Adi, “Real time speech enhancement in the waveform domain,” in Proc. INTERSPEECH, 2020

    2020

  19. [27]

    A regression approach to speech enhancement based on deep neural networks,

    Y . Xu, J. Du, L.-R. Dai, and C.-H. Lee, “A regression approach to speech enhancement based on deep neural networks,” IEEE/ACM Trans. Audio, Speech, and Lang. Proc. , vol. 23, no. 1, pp. 7–19, 2014

    2014

  20. [28]

    Convolutional-recurrent neural networks for speech enhancement,

    H. Zhao, S. Zarar, I. Tashev, and C.-H. Lee, “Convolutional-recurrent neural networks for speech enhancement,” in Proc. ICASSP , 2018, pp. 2401–2405

    2018

  21. [29]

    Gated residual networks with dilated convolutions for monaural speech enhancement,

    K. Tan, J. Chen, and D. Wang, “Gated residual networks with dilated convolutions for monaural speech enhancement,” IEEE/ACM transac- tions on audio, speech, and language processing , vol. 27, no. 1, pp. 189–198, 2018

    2018

  22. [30]

    Speech enhancement using multi-stage self-attentive temporal con- volutional networks,

    J. Lin, A. J. d. L. van Wijngaarden, K.-C. Wang, and M. C. Smith, “Speech enhancement using multi-stage self-attentive temporal con- volutional networks,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 3440–3450, 2021

    2021

  23. [31]

    Exploring tradeoffs in models for low- latency speech enhancement,

    K. Wilson, M. Chinen, J. Thorpe, B. Patton, J. Hershey, R. A. Saurous, J. Skoglund, and R. F. Lyon, “Exploring tradeoffs in models for low- latency speech enhancement,” in Proc. IWAENC, 2018, pp. 366–370

    2018

  24. [32]

    Learning complex spectral mapping with gated convolutional recurrent networks for monaural speech enhancement,

    K. Tan and D. Wang, “Learning complex spectral mapping with gated convolutional recurrent networks for monaural speech enhancement,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 28, pp. 380–390, 2019

    2019

  25. [33]

    Towards scaling up classification-based speech separation,

    Y . Wang and D. Wang, “Towards scaling up classification-based speech separation,” IEEE Trans. Audio, Speech, and Lang. Proc. , vol. 21, no. 7, pp. 1381–1390, 2013

    2013

  26. [34]

    On training targets for super- vised speech separation,

    Y . Wang, A. Narayanan, and D. Wang, “On training targets for super- vised speech separation,” IEEE/ACM Trans. Audio, speech, and Lang. Proc., vol. 22, no. 12, pp. 1849–1858, 2014

    2014

  27. [35]

    Complex ratio masking for monaural speech separation,

    D. S. Williamson, Y . Wang, and D. Wang, “Complex ratio masking for monaural speech separation,” IEEE/ACM Trans. Audio, Speech, and Lang. Proc., vol. 24, no. 3, pp. 483–492, 2015

    2015

  28. [36]

    Phase- sensitive and recognition-boosted speech separation using deep recurrent neural networks,

    H. Erdogan, J. R. Hershey, S. Watanabe, and J. Le Roux, “Phase- sensitive and recognition-boosted speech separation using deep recurrent neural networks,” in Proc. ICASSP, 2015, pp. 708–712

    2015

  29. [37]

    Single-channel speech sep- aration with memory-enhanced recurrent neural networks,

    F. Weninger, F. Eyben, and B. Schuller, “Single-channel speech sep- aration with memory-enhanced recurrent neural networks,” in Proc. ICASSP, 2014, pp. 3709–3713

    2014

  30. [38]

    Speech enhancement with lstm recurrent neural networks and its application to noise-robust asr,

    F. Weninger, H. Erdogan, S. Watanabe, E. Vincent, J. Le Roux, J. R. Hershey, and B. Schuller, “Speech enhancement with lstm recurrent neural networks and its application to noise-robust asr,” in International Conference on Latent V ariable Analysis and Signal separation , 2015...

    2015

  31. [39]

    Long short-term memory for speaker general- ization in supervised speech separation,

    J. Chen and D. Wang, “Long short-term memory for speaker general- ization in supervised speech separation,” The Journal of the Acoustical Society of America , vol. 141, no. 6, pp. 4705–4714, 2017

    2017

  32. [40]

    An empirical evaluation of generic convolutional and recurrent networks for sequence modeling,

    S. Bai, J. Z. Kolter, and V . Koltun, “An empirical evaluation of generic convolutional and recurrent networks for sequence modeling,” arXiv preprint arXiv:1803.01271, 2018

    2018 arXiv

  33. [41]

    DeepMMSE: A deep learning approach to mmse-based noise power spectral density estimation,

    Q. Zhang, A. Nicolson, M. Wang, K. K. Paliwal, and C. Wang, “DeepMMSE: A deep learning approach to mmse-based noise power spectral density estimation,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 28, pp. 1404–1415, 2020

    2020

  34. [42]

    Time-frequency attention for monaural speech enhancement,

    Q. Zhang, Q. Song, Z. Ni, A. Nicolson, and H. Li, “Time-frequency attention for monaural speech enhancement,” in Proc. ICASSP , 2022, pp. 7852–7856

    2022

  35. [43]

    Deep attention gated dilated temporal convolutional networks with intra-parallel con- volutional modules for end-to-end monaural speech separation

    Z. Shi, H. Lin, L. Liu, R. Liu, J. Han, and A. Shi, “Deep attention gated dilated temporal convolutional networks with intra-parallel con- volutional modules for end-to-end monaural speech separation.” in Proc. INTERSPEECH, 2019, pp. 3183–3187

    2019

  36. [44]

    Conv-TasNet: Surpassing ideal time– frequency magnitude masking for speech separation,

    Y . Luo and N. Mesgarani, “Conv-TasNet: Surpassing ideal time– frequency magnitude masking for speech separation,” IEEE/ACM trans- actions on audio, speech, and language processing , vol. 27, no. 8, pp. 1256–1266, 2019

    2019

  37. [45]

    A time-frequency attention module for neural speech enhancement,

    Q. Zhang, X. Qian, Z. Ni, A. Nicolson, E. Ambikairajah, and H. Li, “A time-frequency attention module for neural speech enhancement,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 31, pp. 462–475, 2023

    2023

  38. [46]

    SE-Conformer: Time-Domain Speech Enhance- ment Using Conformer,

    E. Kim and H. Seo, “SE-Conformer: Time-Domain Speech Enhance- ment Using Conformer,” in Proc. INTERSPEECH, 2021, pp. 2736–2740

    2021

  39. [47]

    DPT-FSNet: Dual-path transformer based full-band and sub-band fusion network for speech enhancement,

    F. Dang, H. Chen, and P. Zhang, “DPT-FSNet: Dual-path transformer based full-band and sub-band fusion network for speech enhancement,” in Proc. ICASSP, 2022, pp. 6857–6861. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2019 14

    2022

  40. [48]

    Dual-branch attention-in-attention transformer for single-channel speech enhance- ment,

    G. Yu, A. Li, C. Zheng, Y . Guo, Y . Wang, and H. Wang, “Dual-branch attention-in-attention transformer for single-channel speech enhance- ment,” in Proc. ICASSP, 2022, pp. 7847–7851

    2022

  41. [49]

    Train short, test long: Attention with linear biases enables input length extrapolation,

    O. Press, N. Smith, and M. Lewis, “Train short, test long: Attention with linear biases enables input length extrapolation,” in Proc. ICLR, 2021

    2021

  42. [50]

    CAPE: Encoding relative positions with continuous augmented positional embeddings,

    T. Likhomanenko, Q. Xu, G. Synnaeve, R. Collobert, and A. Rogozh- nikov, “CAPE: Encoding relative positions with continuous augmented positional embeddings,” in Proc. NeurIPS, 2021

    2021

  43. [51]

    Location attention for extrapolation to longer sequences,

    Y . Dubois, G. Dagan, D. Hupkes, and E. Bruni, “Location attention for extrapolation to longer sequences,” in Proc. ACL, 2020, pp. 403–413

    2020

  44. [52]

    The EOS decision and length extrapolation,

    B. Newman, J. Hewitt, P. Liang, and C. D. Manning, “The EOS decision and length extrapolation,” in BlackBoxNLP@EMNLP, 2020

    2020

  45. [53]

    From local structures to size generalization in graph neural networks,

    G. Yehudai, E. Fetaya, E. Meirom, G. Chechik, and H. Maron, “From local structures to size generalization in graph neural networks,” in Proc. ICML. PMLR, 2021, pp. 11 975–11 986

    2021

  46. [54]

    BERT: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  47. [55]

    Efficient transformer-based speech enhancement using long frames and STFT magnitudes,

    D. de Oliveira, T. Peer, and T. Gerkmann, “Efficient transformer-based speech enhancement using long frames and STFT magnitudes,” in Proc. INTERSPEECH, 2022, pp. 2948–2952

    2022

  48. [56]

    Attention is all you need in speech separation,

    C. Subakan, M. Ravanelli, S. Cornell, M. Bronzi, and J. Zhong, “Attention is all you need in speech separation,” in Proc. ICASSP, 2021, pp. 21–25

    2021

  49. [57]

    Self- attentional acoustic models,

    M. Sperber, J. Niehues, G. Neubig, S. St ¨uker, and A. Waibel, “Self- attentional acoustic models,” in Proc. INTERSPEECH, 2018, pp. 3723– 3727

    2018

  50. [58]

    Da-transformer: Distance-aware trans- former,

    C. Wu, F. Wu, and Y . Huang, “Da-transformer: Distance-aware trans- former,” in Proc. of Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolo- gies, 2021, pp. 2059–2068

    2021

  51. [59]

    Roformer: Enhanced transformer with rotary position embedding,

    J. Su, Y . Lu, S. Pan, A. Murtadha, B. Wen, and Y . Liu, “Roformer: Enhanced transformer with rotary position embedding,” arXiv preprint arXiv:2104.09864, 2021

    2021 arXiv

  52. [60]

    Librispeech: an asr corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: an asr corpus based on public domain audio books,” in Proc. ICASSP, 2015, pp. 5206–5210

    2015

  53. [61]

    100 nonspeech environmental sounds,

    G. Hu, “100 nonspeech environmental sounds,” The Ohio State Univer- sity, Department of Computer Science and Engineering , 2004

    2004

  54. [62]

    Description of the RSG-10 noise database,

    H. J. Steeneken and F. W. Geurtsen, “Description of the RSG-10 noise database,” Report IZF 1988-3, TNO Institute for Perception, Soesterberg, The Netherlands , 1988

    1988

  55. [63]

    Smartphone-based real-time classification of noise signals using subband features and random forest classifier,

    F. Saki, A. Sehgal, I. Panahi, and N. Kehtarnavaz, “Smartphone-based real-time classification of noise signals using subband features and random forest classifier,” in Proc. ICASSP, 2016, pp. 2204–2208

    2016

  56. [64]

    Automatic switching between noise classi- fication and speech enhancement for hearing aid devices,

    F. Saki and N. Kehtarnavaz, “Automatic switching between noise classi- fication and speech enhancement for hearing aid devices,” in Proc. IEEE EMBC, 2016, pp. 736–739

    2016

  57. [65]

    A dataset and taxonomy for urban sound research,

    J. Salamon, C. Jacoby, and J. P. Bello, “A dataset and taxonomy for urban sound research,” in Proc. ACM MM , 2014, pp. 1041–1044

    2014

  58. [66]

    MUSAN: A music, speech, and noise corpus,

    D. Snyder, G. Chen, and D. Povey, “MUSAN: A music, speech, and noise corpus,” CoRR, vol. abs/1510.08484, 2015

    2015 arXiv

  59. [67]

    The QUT- NOISE-TIMIT corpus for the evaluation of voice activity detection algorithms,

    D. B. Dean, S. Sridharan, R. J. V ogt, and M. W. Mason, “The QUT- NOISE-TIMIT corpus for the evaluation of voice activity detection algorithms,” in Proc. INTERSPEECH, 2010, pp. 3110–3113

    2010

  60. [68]

    Interactive speech and noise modeling for speech enhancement,

    C. Zheng, X. Peng, Y . Zhang, S. Srinivasan, and Y . Lu, “Interactive speech and noise modeling for speech enhancement,” arXiv preprint arXiv:2012.09408, 2020

    2012 arXiv

  61. [69]

    862.2: Wideband extension to recommendation P. 862 for the assessment of wideband telephone networks and speech codecs. ITU-Telecommunication standardization sector, 2007

    R. I.-T. P. ITU, “862.2: Wideband extension to recommendation P. 862 for the assessment of wideband telephone networks and speech codecs. ITU-Telecommunication standardization sector, 2007.”

    2007

  62. [70]

    An algorithm for predicting the intelligibility of speech masked by modulated noise maskers,

    J. Jensen and C. H. Taal, “An algorithm for predicting the intelligibility of speech masked by modulated noise maskers,” IEEE/ACM Trans. Audio, Speech, and Lang. Proc. , vol. 24, no. 11, pp. 2009–2022, 2016

    2009

  63. [71]

    Evaluation of objective quality measures for speech enhancement,

    Y . Hu and P. C. Loizou, “Evaluation of objective quality measures for speech enhancement,” IEEE Trans. Audio, Speech, and Lang. proc. , vol. 16, no. 1, pp. 229–238, 2007

    2007

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.