Pith. sign in

REVIEW 3 major objections 5 minor 39 references

FLASepformer: Efficient Speech Separation with Gated Focused Linear Attention Transformer

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Gated Focused Linear Attention replaces quadratic self-attention in speech separation, matching quality while making inference and memory scale linearly with audio length.

desk verdict A credible incremental application of focused linear attention to speech separation with usable efficiency gains, though the speedup numbers need tighter baseline measurements before I'd trust them fully. read the letter →

arxiv 2508.19528 v1 pith:V2X4GBUD submitted 2025-08-27 eess.AS cs.SD

Haoxu Wang , Yiheng Jiang , Gang Qiao , Pengteng Shi , Biao Tian This is my paper · ORCID
classification eess.AScs.SD
keywords speechseparationfocusedlinearattentiongatedmodulecomplexitylongsequencemodelingmonauralefficientinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Speech separation models built on transformers pay a quadratic price in time and memory as audio gets longer, because self-attention compares every pair of positions. This paper claims that replacing that attention with Gated Focused Linear Attention keeps separation quality essentially intact while making inference scale linearly with audio length. It demonstrates this by rebuilding two recent separators, the time-domain SepReformer and the time-frequency TF-Locoformer, with the new module, and reports speedups of up to 2.29x on 30-second mixtures with roughly a fifth to a third of the GPU memory, while staying within about 0.4 dB of the original models' SI-SNRi on the WSJ0-2Mix benchmark. If true, the result matters because it removes a practical barrier to separating minute-long or longer audio in applications such as meeting transcription and hearing assistance.

What carries the argument

The central object is the Gated Focused Linear Attention module. Its core is the focused kernel $\phi_p(\cdot)$, a ReLU followed by a power-$p$ normalization that pulls similar query-key pairs together and pushes dissimilar ones apart, plus a fixed-cost 1D depthwise convolution that restores feature diversity by raising the rank of the attention matrix, and a gated MLP that reweights the output. This combination computes attention as $\phi(Q)(\phi(K)^\top V)$, preserving the associativity that makes the cost linear in sequence length, while counteracting the over-smoothing and low-rank failure modes of vanilla linear attention.

What would settle it

Measure inference time and GPU memory for SepReformer-T/B/L and TF-Locoformer-S/M/L on 30-second mixtures on the same GPU hardware, batch size, and software stack as the FLA variants, and check whether the speedup and memory ratios reproduce the reported 2.29x/1.91x/1.49x and 15.8%/20.9%/31.9% figures; a second check would be to determine whether the focus factor $p=3$ and kernel size $k=7$ were chosen on the validation split rather than the test split.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the quadratic-complexity self-attention used in strong speech separators can be swapped for a linear-complexity Gated Focused Linear Attention (Gated FLA) module without giving up separation quality. FLA rewrites attention as $\phi(Q)(\phi(K)^\top V)$ so complexity drops from $\mathcal{O}(N^2)$ to $\mathcal{O}(N)$, while a focused kernel $\phi_p(x)=f_p(\mathrm{ReLU}(x))$ with $f_p(x)=\frac{\|x\|}{\|x^{*p}\|}x^{*p}$ keeps attention weights sharp, and a 1D depthwise convolution restores the rank that vanilla linear attention loses. A gated MLP then modulates the output. The result is two models, FLA-SepReformer and FLA-TFLocoformer, that stay within a few tenths of a decibel of the SI-SNRi of SepReformer and TF-Locoformer across the studied model sizes on clean, noisy, and reverberant two-speaker mixtures, while inference time and memory grow linearly with sequence length rather than quadratically.

Load-bearing premise

The reported speedups assume the published SepReformer and TF-Locoformer baselines were measured under comparable, reasonably optimized inference conditions; if the baseline implementations were slower or more memory-hungry than necessary, the speedup and memory ratios would overstate the real gain.

Editorial extensions

If this is right

  • All six FLA variants hold SI-SNRi within about 0.4 dB of their corresponding SepReformer or TF-Locoformer baselines on WSJ0-2Mix, so the efficiency gain does not come with a large quality penalty.
  • Because inference time and memory scale linearly, a 30-second mixture is processed 1.5–2.3x faster using a fraction of the GPU memory, with the relative advantage growing for longer segments where quadratic baselines diverge.
  • The design transfers to noisy and reverberant conditions: on WHAMR! the FLA-TFLocoformer-M model reaches 18.7 dB SI-SNRi, matching or exceeding the TF-Locoformer-M baseline, and on Libri2Mix-360 it reaches 22.2 dB.
  • The reproduced SepReformer-B baseline (23.6 dB vs. the published 23.8 dB) gives a same-setup comparison point, making the speed and memory ratios more trustworthy than a table-only comparison would be.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the same swap works broadly, Gated FLA could be dropped into other quadratic attention modules in speech enhancement, speaker diarization, or audio captioning, which share the same regime of long sequences and small channel dimensions.
  • Editorial inference: the paper does not report whether the focus factor $p=3$ and kernel size $k=7$ were selected on a validation split or the test split; if they were tuned on test data, the reported quality numbers are slightly optimistic.
  • Editorial inference: the rank argument implies the DWC1d module should matter most when the channel dimension $d$ is much smaller than the sequence length $N$, so varying $d$ while holding $N$ fixed would be a direct test of the mechanism beyond the exact settings used here.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes FLASepformer, a monaural speech separation architecture that replaces quadratic-complexity softmax attention with Gated Focused Linear Attention. It builds two variants by substituting the global attention modules in SepReformer and TF-Locoformer, adding a 1D depthwise convolution and a gated MLP to the FLA module. The authors evaluate the variants on WSJ0-2Mix, WHAM!, WHAMR!, and Libri2Mix, reporting SI-SNRi/SDRi within about 0.4 dB of the corresponding baselines and claiming 1.49-2.29x inference speedups with large memory reductions on 30-second inputs. They also include ablations over the gate, the focused factor p, and the DWC kernel size k.

Significance. If the efficiency comparison is fair and the quality differences are statistically meaningful, the paper provides a useful result: linear-complexity attention can be substituted into strong speech separation models with only a small quality drop and substantially better long-sequence inference. The paper's strengths include a reproduced SepReformer-B baseline, multi-dataset evaluation, and a clear ablation of the gating module and FLA hyperparameters. The main uncertainties are baseline timing provenance, run-to-run variance, and hyperparameter selection protocol, so the headline claims currently need strengthening.

major comments (3)
  1. [Section 4.2, Table 1, Figure 3] The headline speedup and memory claims rest on comparisons to SepReformer-T/L and TF-Locoformer-S/M/L, but only SepReformer-B is reproduced in Table 1. The manuscript does not state whether the other baselines' timing and memory measurements in Figure 3 were obtained on the same hardware, batch size, sequence length, and software stack as the FLA models, nor whether the baseline checkpoints and inference code paths were equally optimized. If those numbers come from the original papers or from a less optimized setup, the reported 1.49-2.29x speedups and memory ratios would be inflated. Please report the exact measurement protocol and either measure all baselines in-house or clearly attribute each baseline number.
  2. [Tables 1-3] The central quality claim is that FLA models match SepReformer and TF-Locoformer, yet the reported SI-SNRi gaps are only 0.2-0.4 dB and no standard deviations or multiple seeds are given. Given the typical run-to-run variance in speech separation training, these differences are not enough to establish comparable performance as a statistical statement. Please provide repeated-run intervals or another explicit variance measure for at least the main WSJ0-2Mix results.
  3. [Section 4.3, Table 4] The paper does not say whether the focused factor p=3 and DWC kernel size k=7 were chosen on a validation split or on the test set. Because Table 4 reports test-set SI-SNRi for several p and k values, any selection based on those test numbers would be a form of test-set leakage. Please specify the selection rule and, if necessary, re-select the hyperparameters on a held-out split.
minor comments (5)
  1. [Eq. (4)] The displayed formula for the focused function appears to be missing the division by the norm of x**p; in the original FLA formulation the function is ||x|| / ||x**p|| times x**p. Please correct the equation and any resulting ambiguity.
  2. [Section 2.3, References] The text cites SepReformer as [5], but reference [5] is the SepFormer paper; SepReformer is reference [23]. This citation inconsistency appears in Section 2.3 and should be corrected throughout.
  3. [Table 1, Section 4.2] Table 1 lists MACs for all methods, but Section 4.2 states that PyTorch-OpCounter MACs are inaccurate and do not align with inference time. Please either correct the MAC values or remove the column, since it presently displays numbers the paper itself discredits.
  4. [Table 1] The parameter counts for FLA-SepReformer are identical to those of SepReformer (e.g., 3.7, 14.2, and 59.4) despite the added Gated MLP and DWC1d modules; please clarify whether these counts are rounded, or whether the new modules replace other parameters.
  5. [Figure 3, Section 4.2] The inference-time and memory measurement conditions should be described fully, including GPU model, batch size, audio length, number of warmup iterations, and whether memory refers to peak allocation or steady-state usage; the phrase single-threaded in Section 4.2 is unclear for GPU inference.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's claims are empirical architecture comparisons, with no fitted parameter or self-citation chain doing the derivational work.

full rationale

The paper's central claims are (1) that replacing quadratic-complexity attention with Gated Focused Linear Attention yields a linear-complexity speech separation model, and (2) that the resulting models match or approach the separation quality of SepReformer and TF-Locoformer while being faster and more memory-efficient. Neither claim is derived from a parameter fitted to the target result. FLA is imported from an external source [22], the gating mechanism is credited to [20], and the base architectures are external baselines [5, 7]. The reported SI-SNRi/SDRi numbers are measured empirically on standard datasets and compared directly with published or reproduced baselines; they are not produced by solving an equation that already contains the answer. The efficiency numbers are wall-clock inference measurements and memory measurements on GPUs, supported by the stated O(N) complexity of linear attention; the speedup factors are not constructed from the model's own weights or from the paper's own definitions. The ablation choices of p=3 and k=7 are hyperparameter selections, not circular predictions, and no claim in the paper reduces to its own input by construction. The only self-citation [17] appears in a broad list of efficient speech-processing designs and is not load-bearing. Concerns about whether all baseline timings were measured under identical conditions are legitimate correctness/comparability risks, but they are not circularity: they concern the reliability of empirical comparisons, not the derivation of the output from the input. Therefore no significant circularity is present.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central efficiency claim depends on the standard linear-attention complexity reduction from the cited FLA paper [22]. The quality claim depends on two hyperparameters (p and k) tuned on WSJ0-2Mix. No new physical or mathematical entities are introduced.

free parameters (2)
  • Focused factor p = 3
    Selected by ablation on WSJ0-2Mix (Table 4); controls the sharpness of the linear attention kernel.
  • DWC1d kernel size k = 7
    Selected by ablation on WSJ0-2Mix (Table 4); controls the local feature integration range.
assumptions (4)
  • standard math Linear attention reduces complexity from O(N^2) to O(N) by computing phi(K)^T V first.
    Based on the associativity of matrix multiplication, stated in Eqs. (2)-(3) of Section 2.1.
  • standard math The rank of phi(Q)phi(K)^T is bounded by min{N, d}.
    Quoted from the FLatten Transformer paper [22] in Section 2.2; used to justify the need for DWC1d.
  • domain assumption Speech separation features have d << N, making the rank bound relevant.
    Paper cites SepReformer-B with d=16 and N=2000 for one second of audio in Section 2.2.
  • ad hoc to paper DWC1d restores attention rank and local modeling in the speech domain.
    This is an adaptation of the 2D DWC from FLA to 1D speech features; the paper does not directly verify rank restoration, only observes improved empirical performance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FLASepformer: Efficient Speech Separation with Gated Focused Linear Attention Transformer." pith.science (2026). https://pith.science/paper/V2X4GBUD

@misc{pith2026250819528,
  author       = {Pith},
  title        = {Pith review of: FLASepformer: Efficient Speech Separation with Gated Focused Linear Attention Transformer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V2X4GBUD}},
  note         = {Machine review of arXiv:2508.19528}
}
read the original abstract

Speech separation always faces the challenge of handling prolonged time sequences. Past methods try to reduce sequence lengths and use the Transformer to capture global information. However, due to the quadratic time complexity of the attention module, memory usage and inference time still increase significantly with longer segments. To tackle this, we introduce Focused Linear Attention and build FLASepformer with linear complexity for efficient speech separation. Inspired by SepReformer and TF-Locoformer, we have two variants: FLA-SepReformer and FLA-TFLocoformer. We also add a new Gated module to improve performance further. Experimental results on various datasets show that FLASepformer matches state-of-the-art performance with less memory consumption and faster inference. FLA-SepReformer-T/B/L increases speed by 2.29x, 1.91x, and 1.49x, with 15.8%, 20.9%, and 31.9% GPU memory usage, proving our model's effectiveness.

Figures

Figures reproduced from arXiv: 2508.19528 by the authors.

Figure 1
Figure 1. Si-SNRi results on WSJ0-2Mix versus Inference Time on RTX A800 GPU (ms/s) for 30s audio mixture. The check mark indicates using DM method for training. The radius of circle is proportional to the parameter size of the model. computing KT V first compared to QKT in Softmax Attention. In SS tasks d << N (channel dimension is much smaller than sequence length), using an efficient linear attention scheme to model long s… view at source ↗
Figure 2
Figure 2. (a) The structure of Gated Focused Linear Attention Module. (b) The structure of FLA-SepReformer. (c) The structure of FLA-TFLocoformer. Q = XWq , K = XWk , V = XWv , Oi = PN j=1 Sim(Qi,Kj ) PN j=1 Sim(Qi,Kj ) Vj (1) which X ∈ R N×d is the input sequence, W∗ ∈ R d×d is the linear layer, and i is the sequence index. Traditional attention uses Softmax attention as the similarity function Sim(Q, K) = exp(QKT /sqrt(d)) … view at source ↗
Figure 3
Figure 3. Inference Time or Memory Usage using various GPU [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 35 canonical work pages

  1. [1]

    Introduction Monaural speech separation (SS) extracts individual speech sources from a single-channel mixture, which is crucial for ad- dressing the cocktail party problem [1,2] and improving speech applications. While previous SS methods achieve good re- sults using neural networks [3–5], SS still faces the challenge of modeling prolonged time sequences,...

    work page

  2. [2]

    Methods 2.1. Vanilla Linear Attention Let’s first review the original attention module, which can be defined as: arXiv:2508.19528v1 [eess.AS] 27 Aug 2025 Linear Add Linear Focused Linear Attention h DWC Concat Kernel Func. MatMul Sum. MatMul & Div. MatMul Norm & Linear Up. + Concat. + Linear Down Conv. Conv-SwiGLU Norm + Gated FLA Conv-SwiGLU T emporal Mo...

    work page Pith review arXiv 2025

  3. [3]

    Dataset We validate our model’s performance using four popular speech separation datasets: WSJ0-2Mix [3], WHAM! [28], WHAMR! [29], and Libri2Mix [30]

    Experimental Setup 3.1. Dataset We validate our model’s performance using four popular speech separation datasets: WSJ0-2Mix [3], WHAM! [28], WHAMR! [29], and Libri2Mix [30]. We train and test all datasets using the full overlap min version with a sampling rate of 8kHz. WSJ0-2Mix is a commonly used benchmark for SS, created from the WSJ0 corpus to generat...

    work page

  4. [4]

    Comparison with previous models We use SI-SNR improvement (SI-SNRi) and SDR improve- ment (SDRi) [33] to evaluate model performance

    Results and Discussion 4.1. Comparison with previous models We use SI-SNR improvement (SI-SNRi) and SDR improve- ment (SDRi) [33] to evaluate model performance. Also, we re- port the number of multiply-accumulate operations (MACs) for 8k samples using pytorch-OpCounter 1. Table 1 shows results 1https://github.com/Lyken17/pytorch-OpCounter (a) RTX A800 GPU...

    work page

  5. [5]

    Although previous meth- ods use STFT and downsampling to reduce speech sequence length, the attention module in those still has O(N 2) time complexity

    Conclusion In this paper, we build FLASepformer, an efficient speech sep- aration model with linear complexity. Although previous meth- ods use STFT and downsampling to reduce speech sequence length, the attention module in those still has O(N 2) time complexity. We improve SepReformer and TF-Locoformer by integrating Focused Linear Attention, creating tw...

    work page

  6. [6]

    Some experiments on the recognition of speech, with one and with two ears,

    E. C. Cherry, “Some experiments on the recognition of speech, with one and with two ears,” Journal of the acoustical society of America, vol. 25, pp. 975–979, 1953

    work page 1953

  7. [7]

    A. S. Bregman, Auditory Scene Analysis: The Perceptual Organi- zation of Sound. MIT Press, 1994

    work page 1994

  8. [8]

    Deep clus- tering: Discriminative embeddings for segmentation and separa- tion,

    J. R. Hershey, Z. Chen, J. Le Roux, and S. Watanabe, “Deep clus- tering: Discriminative embeddings for segmentation and separa- tion,” in Proc. ICASSP. IEEE, 2016, pp. 31–35

    work page 2016

Show all 39 references
  1. [9]

    Multitalker speech separation with utterance-level permutation invariant training of deep recurrent neural networks,

    M. Kolbæk, D. Yu, Z.-H. Tan, and J. Jensen, “Multitalker speech separation with utterance-level permutation invariant training of deep recurrent neural networks,” IEEE/ACM Transactions on Au- dio, Speech, and Language Processing, vol. 25, no. 10, pp. 1901– 1913, 2017

    1901

  2. [10]

    Attention Is All You Need In Speech Separation,

    C. Subakan, M. Ravanelli, S. Cornell, M. Bronzi, and J. Zhong, “Attention Is All You Need In Speech Separation,” in Proc. ICASSP, 2021, pp. 21–25

    2021

  3. [11]

    TF-GRIDNET: Making Time-Frequency Domain Models Great Again for Monaural Speaker Separation,

    Z.-Q. Wang, S. Cornell, S. Choi, Y . Lee, B.-Y . Kim, and S. Watan- abe, “TF-GRIDNET: Making Time-Frequency Domain Models Great Again for Monaural Speaker Separation,” in Proc. ICASSP, 2023, pp. 1–5

    2023

  4. [12]

    TF-Locoformer: Transformer with Local Modeling by Convo- lution for Speech Separation and Enhancement,

    K. Saijo, G. Wichern, F. G. Germain, Z. Pan, and J. Le Roux, “TF-Locoformer: Transformer with Local Modeling by Convo- lution for Speech Separation and Enhancement,” in Proc. Inter- national Workshop on Acoustic Signal Enhancement (IWAENC) , Sep. 2024

    2024

  5. [13]

    Conv-tasnet: Surpassing ideal time– frequency magnitude masking for speech separation,

    Y . Luo and N. Mesgarani, “Conv-tasnet: Surpassing ideal time– frequency magnitude masking for speech separation,”IEEE/ACM transactions on audio, speech, and language processing , vol. 27, no. 8, pp. 1256–1266, 2019

    2019

  6. [14]

    Dual-Path RNN: Effi- cient Long Sequence Modeling for Time-Domain Single-Channel Speech Separation,

    Y . Luo, Z. Chen, and T. Yoshioka, “Dual-Path RNN: Effi- cient Long Sequence Modeling for Time-Domain Single-Channel Speech Separation,” in Proc. ICASSP, 2020, pp. 46–50

    2020

  7. [15]

    Long Short-term Memory,

    S. Hochreiter, “Long Short-term Memory,” Neural Computation MIT-Press, 1997

    1997

  8. [16]

    Dual-Path Transformer Net- work: Direct Context-Aware Modeling for End-to-End Monaural Speech Separation,

    J. Chen, Q. Mao, and D. Liu, “Dual-Path Transformer Net- work: Direct Context-Aware Modeling for End-to-End Monaural Speech Separation,” in Interspeech 2020, 2020, pp. 2642–2646

    2020

  9. [17]

    Attention Is All You Need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention Is All You Need,” in Proc. NIPS 2017, vol. 30, 2017

    2017

  10. [18]

    Sudo RM -RF: Effi- cient Networks for Universal Audio Source Separation,

    E. Tzinis, Z. Wang, and P. Smaragdis, “Sudo RM -RF: Effi- cient Networks for Universal Audio Source Separation,” in Proc. MLSP. IEEE, 2020, pp. 1–6

    2020

  11. [19]

    Sandglasset: A Light Multi-Granularity Self-Attentive Network for Time- Domain Speech Separation,

    M. W. Y . Lam, J. Wang, D. Su, and D. Yu, “Sandglasset: A Light Multi-Granularity Self-Attentive Network for Time- Domain Speech Separation,” in Proc. ICASSP. IEEE, 2021, pp. 5759–5763

    2021

  12. [20]

    An efficient encoder-decoder archi- tecture with top-down attention for speech separation,

    K. Li, R. Yang, and X. Hu, “An efficient encoder-decoder archi- tecture with top-down attention for speech separation,” in Proc. ICLR. OpenReview.net, 2023

    2023

  13. [21]

    TIGER: Time-frequency In- terleaved Gain Extraction and Reconstruction for Efficient Speech Separation,

    M. Xu, K. Li, G. Chen, and X. Hu, “TIGER: Time-frequency In- terleaved Gain Extraction and Reconstruction for Efficient Speech Separation,” in Proc. ICLR. OpenReview.net, 2025

    2025

  14. [22]

    ZipEnhancer: Dual-Path Down-Up Sampling-based Zipformer for Monaural Speech Enhancement,

    H. Wang and B. Tian, “ZipEnhancer: Dual-Path Down-Up Sampling-based Zipformer for Monaural Speech Enhancement,” in Proc. ICASSP, 2025, pp. 1–5

    2025

  15. [23]

    MossFormer: Pushing the Performance Limit of Monaural Speech Separation Using Gated Single-Head Transformer with Convolution-Augmented Joint Self-Attentions,

    S. Zhao and B. Ma, “MossFormer: Pushing the Performance Limit of Monaural Speech Separation Using Gated Single-Head Transformer with Convolution-Augmented Joint Self-Attentions,” in Proc. ICASSP, 2023, pp. 1–5

    2023

  16. [24]

    MossFormer2: Combin- ing Transformer and RNN-Free Recurrent Network for Enhanced Time-Domain Monaural Speech Separation,

    S. Zhao, Y . Ma, C. Ni, C. Zhang, H. Wang, T. H. Nguyen, K. Zhou, J. Q. Yip, D. Ng, and B. Ma, “MossFormer2: Combin- ing Transformer and RNN-Free Recurrent Network for Enhanced Time-Domain Monaural Speech Separation,” in Proc. ICASSP , 2024, pp. 10 356–10 360

    2024

  17. [25]

    Transformer Quality in Linear Time,

    W. Hua, Z. Dai, H. Liu, and Q. V . Le, “Transformer Quality in Linear Time,” inProc. ICML, ser. Proceedings of Machine Learn- ing Research, K. Chaudhuri, S. Jegelka, L. Song, C. Szepesv ´ari, G. Niu, and S. Sabato, Eds., vol. 162. PMLR, 2022, pp. 9099– 9117

    2022

  18. [26]

    Trans- formers are RNNs: Fast Autoregressive Transformers with Linear Attention,

    A. Katharopoulos, A. Vyas, N. Pappas, and F. Fleuret, “Trans- formers are RNNs: Fast Autoregressive Transformers with Linear Attention,” in Proc. ICML, ser. Proceedings of Machine Learning Research, vol. 119. PMLR, 2020, pp. 5156–5165

    2020

  19. [27]

    FLatten Trans- former: Vision Transformer using Focused Linear Attention,

    D. Han, X. Pan, Y . Han, S. Song, and G. Huang, “FLatten Trans- former: Vision Transformer using Focused Linear Attention,” in Proc. ICCV. IEEE, 2023, pp. 5938–5948

    2023

  20. [28]

    Separate and Reconstruct: Asymmetric Encoder-Decoder for Speech Separation,

    U. Shin, S. Lee, T. Kim, and H. Park, “Separate and Reconstruct: Asymmetric Encoder-Decoder for Speech Separation,” in Proc. NeurIPS, A. Globersons, L. Mackey, D. Belgrave, A. Fan, U. Pa- quet, J. M. Tomczak, and C. Zhang, Eds., 2024

    2024

  21. [29]

    EfficientViT: Lightweight Multi-Scale Attention for High-Resolution Dense Prediction,

    H. Cai, J. Li, M. Hu, C. Gan, and S. Han, “EfficientViT: Lightweight Multi-Scale Attention for High-Resolution Dense Prediction,” in Proc. ICCV. IEEE, 2023, pp. 17 256–17 267

    2023

  22. [30]

    Rethink- ing Attention with Performers,

    K. M. Choromanski, V . Likhosherstov, D. Dohan, X. Song, A. Gane, T. Sarl ´os, P. Hawkins, J. Q. Davis, A. Mohiuddin, L. Kaiser, D. B. Belanger, L. J. Colwell, and A. Weller, “Rethink- ing Attention with Performers,” inProc. ICLR. OpenReview.net, 2021

    2021

  23. [31]

    A Neural State-Space Modeling Approach to Efficient Speech Separation,

    C. Chen, C.-H. H. Yang, K. Li, Y . Hu, P.-J. Ku, and E. S. Chng, “A Neural State-Space Modeling Approach to Efficient Speech Separation,” in Interspeech 2023, 2023, pp. 3784–3788

    2023

  24. [32]

    Speech Slytherin: Examining the Performance and Efficiency of Mamba for Speech Separation, Recognition, and Synthesis,

    X. Jiang, Y . A. Li, A. N. Florea, C. Han, and N. Mesgarani, “Speech Slytherin: Examining the Performance and Efficiency of Mamba for Speech Separation, Recognition, and Synthesis,” arXiv preprint arXiv:2407.09732 , 2024. [Online]. Available: https://arxiv.org/abs/2407.09732

    2024 arXiv

  25. [33]

    WHAM!: Extending Speech Separation to Noisy Environments,

    G. Wichern, J. Antognini, M. Flynn, L. R. Zhu, E. McQuinn, D. Crow, E. Manilow, and J. L. Roux, “WHAM!: Extending Speech Separation to Noisy Environments,” in Interspeech 2019, 2019, pp. 1368–1372

    2019

  26. [34]

    WHAMR!: Noisy and Reverberant Single-Channel Speech Sep- aration,

    M. Maciejewski, G. Wichern, E. McQuinn, and J. L. Roux, “WHAMR!: Noisy and Reverberant Single-Channel Speech Sep- aration,” in Proc. ICASSP. IEEE, 2020, pp. 696–700

    2020

  27. [35]

    LibriMix: An Open-Source Dataset for Generalizable Speech Separation,

    J. Cosentino, M. Pariente, S. Cornell, A. Deleforge, and E. Vin- cent, “LibriMix: An Open-Source Dataset for Generalizable Speech Separation,” 2020

    2020

  28. [36]

    Lib- rispeech: An ASR corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- rispeech: An ASR corpus based on public domain audio books,” in Proc. ICASSP, 2015, pp. 5206–5210

    2015

  29. [37]

    ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understanding,

    Y .-J. Lu, X. Chang, C. Li, W. Zhang, S. Cornell, Z. Ni, Y . Ma- suyama, B. Yan, R. Scheibler, Z.-Q. Wang, Y . Tsao, Y . Qian, and S. Watanabe, “ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understanding,” in Inter- speech 2022, 2022, pp. 5458–5462

    2022

  30. [38]

    SDR – Half-baked or Well Done?

    J. L. Roux, S. Wisdom, H. Erdogan, and J. R. Hershey, “SDR – Half-baked or Well Done?” in Proc. ICASSP, 2019, pp. 626–630

    2019

  31. [39]

    Self-attention Does Not Need O(n 2) Memory,

    M. N. Rabe and C. Staats, “Self-attention Does Not Need O(n 2) Memory,” arXiv preprint arXiv:2112.05682 , 2021. [Online]. Available: https://arxiv.org/abs/2112.05682

    2021 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.