REVIEW 4 major objections 5 minor 34 references
A Two-Stage Hierarchical Deep Filtering Framework for Real-Time Speech Enhancement
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read By splitting deep filtering into temporal and frequency stages, HDF-Net reaches 3.01 WB-PESQ with only 0.20M parameters and 0.43 G/s MACs on VoiceBank+DEMAND, matching or beating much larger systems.
desk verdict Solid architectural contribution with clean ablations, but the 'outperforms' claim rests on unverified cross-paper baselines and should be dialed back to 'comparable.' read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the decoupled deep filter. Deep filtering replaces the usual one-to-one spectral mask with a complex filter that sums contributions from surrounding TF bins; HDF-Net splits that filter into temporal deep filtering along preceding frames, which acts like a harmonic comb filter, and frequency deep filtering along neighboring bands, which models local spectral shape. The architecture carries this out with a sub-band fusion module that copies neighboring bands into the channel dimension, a lightweight TAConv module combining pointwise and depthwise convolutions with a causal temporal attention path, and a TACRN encoder-decoder backbone with grouped GRU memory.
What would settle it
Retrain HDF-Net and the Table 3 baselines under one shared training pipeline, using the same crop length, augmentation, and optimizer schedule, and evaluate on a second noise dataset with different noise types and unseen speakers; if HDF-Net no longer matches or beats the larger systems at equalized training conditions, the central claim of superiority at lower cost fails.
Extended reading notes
Core claim
The central claim is that decoupling deep filtering into temporal and frequency components, and placing each in its own stage, lets a small network beat larger systems on VoiceBank+DEMAND. HDF-Net's first stage applies coarse spectral enhancement with temporal deep filtering to clean up periodic structure, and its second stage applies sub-band fusion followed by frequency deep filtering to refine local spectral detail; the second stage outputs a residual correction added to the first-stage result. With filter order 5 and 129 ERB-compressed frequency bands, the model reaches 3.01 WB-PESQ, 4.24 CSIG, 3.52 CBAK, and 3.64 COVL while using far fewer parameters and MACs than comparison systems.
Load-bearing premise
The load-bearing premise is that the baseline results in Table 3 were obtained under training conditions comparable to HDF-Net's and that VoiceBank+DEMAND performance carries over to other noise conditions and speakers; if either premise is wrong, the reported advantage over larger systems could shrink or disappear.
Editorial extensions
If this is right
- On VoiceBank+DEMAND, a model with 0.20M parameters and 0.43 G/s MACs reaches 3.01 WB-PESQ, so low-resource and real-time speech enhancement does not require large networks.
- Two-stage temporal-then-frequency deep filtering outperforms single-stage deep filtering and also outperforms swapping the stage order, confirming that separating the two filter directions is what buys the improvement.
- Removing either the sub-band fusion module or the temporal attention module lowers all four metrics, so both components carry part of the gain.
- Replacing deep filtering with complex ratio masks at both stages drops WB-PESQ to 2.93, indicating the filter approach itself contributes beyond mask estimation.
Reading between the lines
- Because the gain comes from filter diversity rather than extra filtering, the same two-stage decoupling could transfer to other time-frequency restoration tasks, such as dereverberation or target extraction, wherever neighboring-bin information is useful.
- The paper leaves filter order fixed at 5 for both stages; an adaptive or per-band filter order is a natural next experiment, since deep filtering is known to degrade when filter order grows.
- The sub-band fusion module is used only in the second stage; a testable extension would be to feed sub-band features into both stages and check whether the first-stage coarse estimate also benefits.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes HDF-Net, a two-stage hierarchical deep-filtering network for single-channel speech enhancement. In stage 1, a coarse spectral enhancement network performs temporal deep filtering; in stage 2, a fine residual enhancement network performs frequency deep filtering on sub-band features. The authors also introduce a lightweight convolutional module, TAConv, that combines pointwise/depthwise convolutions with a causal temporal attention mechanism. Experiments on VoiceBank+DEMAND report a WB-PESQ of 3.01 with 0.20M parameters and 0.43 G/s MACs, and ablations indicate that the two-stage temporal/frequency decoupling, the sub-band fusion module, and the temporal attention component each contribute to the reported scores.
Significance. If the reported comparison could be trusted, the paper would be a useful contribution: the two-stage decomposition of deep filtering into temporal and frequency components is a sensible way to reduce filter-coefficient prediction complexity, and the ablations support the main architectural choices. The parameter and MACs figures are genuinely low, and the design is real-time-motivated. However, the central empirical claim of outperforming prior systems is not established by the evidence as presented. The baseline scores in Table 3 are not obtained under the authors' training/evaluation protocol, no error bars or significance tests are provided, and the abstract's 'outperforms' wording is contradicted by several metric-level comparisons in Table 3. These issues are fixable within the scope of a revision, so the contribution is promising but not yet ready for acceptance.
major comments (4)
- [Section 3.4, Table 3] The comparison with prior systems is not controlled. The manuscript does not state that the RNNoise, DeepFilterNet, DCCRN, FullSubNet+, CTS-Net, DEMUCS, PHASEN, and GaGNet scores were obtained by retraining under the protocol of Sections 3.1-3.2, or by evaluating official checkpoints with the same STFT settings, random 2-second cropping, loss function, and metric script. Differences in these settings can easily produce PESQ differences on the order of the 0.02-0.07 margin reported here. Please report confidence intervals over multiple training runs, or retrain/evaluate baselines under a common protocol, before claiming superiority.
- [Abstract and Section 3.4, Table 3] The abstract says HDF-Net 'outperforms other advanced systems while using fewer resources,' but Table 3 shows that HDF-Net trails PHASEN on CBAK (3.52 vs. 3.55) and trails GaGNet on CSIG (4.24 vs. 4.26). The conclusion more cautiously says 'comparable performance.' The central claim should be restated to match the evidence, or additional testing must demonstrate that the small advantages are statistically significant.
- [Section 3.4, Table 3 (CompNet row)] The CompNet baseline is unverifiable as cited: reference [26] is Liang et al., 'CompNet: Competitive neural network for palmprint recognition using learnable gabor kernels,' which is not a speech enhancement system. Either replace this with the correct speech-enhancement CompNet reference and reproducible score, or remove the row.
- [Title, Section 3.2, Section 3.4] The 'real-time' claim is not supported by the reported metrics. Parameter count and MACs alone do not establish that the system operates in real time; the paper gives no RTF measurement on a specific hardware platform, no streaming/latency analysis, and no discussion of algorithmic delay despite the causal temporal filter in Eq. (6). Please report an actual real-time factor or clearly delimit the claim to computational complexity.
minor comments (5)
- [Section 2.5, Eq. (9)] The loss weights alpha and beta are never assigned numerical values; reporting them would improve reproducibility.
- [Section 3.2] The filter order is described as 'set to 5,' but Eq. (5) defines separate limits I and J; please clarify whether temporal and frequency orders are both 5 and how this relates to the output shape B×T×F×2×d_f.
- [Section 3.3, Table 2] The comparison between M2 and M3 is used to conclude that temporal deep filtering outperforms frequency deep filtering, but both the stage-1 and stage-2 filtering types differ between these configurations; a cleaner comparison would hold one stage fixed.
- [Figure 2] Figure 2 is visually dense and the text labels for SBF, TACRN, and TAConv are difficult to read; a higher-resolution figure with annotated tensor shapes would improve clarity.
- [Section 3.1] The sentence 'All utterances were re-sampled to 16 kHz' should also state the original sampling rate of VoiceBank+DEMAND to avoid ambiguity.
Circularity Check
No circularity: HDF-Net is an empirical architecture trained with a standard reconstruction loss and evaluated on a held-out test set; all 29 cited works are external to the authors, and no fitted quantity is relabeled as a prediction.
full rationale
The derivation chain is fully empirical and self-contained, so no circular step is found. HDF-Net's filter coefficients are outputs of the TACRN encoders trained with an MSE-based loss (Eqs. 9-11) against the clean target, and the reported WB-PESQ 3.01 is measured on the held-out VoiceBank+DEMAND test set (824 utterances from 2 unseen speakers, Section 3.1); consequently no fitted parameter is relabeled as a prediction. Every cited building block (deep filtering [5], FullSubNet sub-band fusion [3], CRN [15], DPRNN [16]) is prior work by other research groups, and none of the 29 references is authored by Lu, Huang, Yao, Wang, Hong, or Lin Li, so no self-citation chain carries the central claim. The two-stage design and the temporal/frequency filter ordering are justified by internally controlled ablations (Tables 1-2) with matched parameter counts and compute, not by appeal to prior authority, and the removed-component comparisons (w/o SBF, w/o TA, M0-M5) give the claimed contributions independent empirical support. The residual weaknesses the skeptic notes are correctness and verifiability concerns, not circularity: Table 3's baseline rows are copied from earlier papers that used different training protocols (cropping, losses, STFT settings), and the CompNet row points to reference [26], which is actually a palmprint recognition paper rather than the intended speech-enhancement system; these issues could undermine the 'outperforms' comparison but do not constitute a definitional reduction of any result to its own inputs. No equation in the paper defines an output in terms of the claimed result, and no result is imported from the authors' own prior work, so the paper earns a score of 0.
Assumptions & free parameters
free parameters (4)
- Deep filter order =
5
- ERB band mapping =
129 bands (65 linear + 64 ERB)
- Amplitude compression power c =
0.3
- Loss weights alpha and beta =
not reported
assumptions (4)
- domain assumption Additive noise model x(n)=s(n)+z(n)
- domain assumption STFT linearity X=S+Z
- ad hoc to paper Temporal and frequency deep filtering can be decoupled into two stages
- domain assumption ERB compression preserves perceptually relevant information
invented entities (1)
-
TAConv module
Cite this review
Pith. "Pith review of A Two-Stage Hierarchical Deep Filtering Framework for Real-Time Speech Enhancement." pith.science (2026). https://pith.science/paper/YK3FKAUY
@misc{pith2026250601023,
author = {Pith},
title = {Pith review of: A Two-Stage Hierarchical Deep Filtering Framework for Real-Time Speech Enhancement},
year = {2026},
howpublished = {\url{https://pith.science/paper/YK3FKAUY}},
note = {Machine review of arXiv:2506.01023}
}
read the original abstract
This paper proposes a model that integrates sub-band processing and deep filtering to fully exploit information from the target time-frequency (TF) bin and its surrounding TF bins for single-channel speech enhancement. The sub-band module captures surrounding frequency bin information at the input, while the deep filtering module applies filtering at the output to both the target TF bin and its surrounding TF bins. To further improve the model performance, we decouple deep filtering into temporal and frequency components and introduce a two-stage framework, reducing the complexity of filter coefficient prediction at each stage. Additionally, we propose the TAConv module to strengthen convolutional feature extraction. Experimental results demonstrate that the proposed hierarchical deep filtering network (HDF-Net) effectively utilizes surrounding TF bin information and outperforms other advanced systems while using fewer resources.
Figures
Reference graph
Works this paper leans on
-
[26]
Decoupled weight decay regularization,
I. Loshchilov, “Decoupled weight decay regularization,”arXiv preprint arXiv:1711.05101, 2017
arXiv 2017
-
[1]
Introduction In recent years, deep learning methods have achieved remark- able success in single-channel speech enhancement [1, 2, 3, 4]. Mainstream speech enhancement approaches typically apply the short-time Fourier transform (STFT) to convert noisy speech signals from the time domain to the frequency domain, where a deep neural network estimates a spec...
work page Pith review arXiv 2025
-
[2]
Methodology 2.1. The overall architecture The noisy speech signal is denoted asx(n), which can be ex- pressed as: x(n) =s(n) +z(n)(1) wheres(n)is clean speech signal,z(n)is additive noise signal. Typically, noise reduction operates in frequency domain: X(t, f) =S(t, f) +Z(t, f)(2) whereX(t, f)represents the signalx(n)in the frequency do- main,tandfdenote ...
-
[3]
EXPERIMENTS AND RESULTS 3.1. Datasets To evaluate the performance of HDF-Net, we conducted exper- iments on the V oiceBank+DEMAND [20] dataset, which con- tains paired clean and noisy speech. The training set consists of 11,572 utterances from 28 speakers, with signal-to-noise ratios (SNRs) of 0, 5, 10, 15 dB. The test set includes 824 utterances from 2 u...
-
[4]
CONCLUSION In this paper, we propose HDF-Net, a two-stage network that decoupled deep filtering, designed to enhance model perfor- mance by attending to the information of surrounding TF bins. We also introduce the TAConv module, which further captures temporal correlations for speech enhancement tasks. In addi- tion, we employ several techniques to reduc...
-
[5]
Acknowledgements This work was supported in part by the National Natural Science Foundation of China under Grants 62276220 and 62371407, and the Innovation of Policing Science and Tech- nology, Fujian province (Grant number: 2024Y0068)
-
[6]
H. Schroter, A. N. Escalante-B, T. Rosenkranz, and A. Maier, “Deepfilternet: A low complexity speech enhancement frame- work for full-band audio based on deep filtering,” inICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 7407–7411
work page 2022
-
[7]
Cmgan: Conformer-based metric gan for speech enhancement,
R. Cao, S. Abdulatif, and B. Yang, “Cmgan: Conformer-based metric gan for speech enhancement,” inInterspeech 2022, 2022, pp. 936–940
work page 2022
Show all 34 references
-
[8]
Fullsubnet: A full-band and sub-band fusion model for real-time single-channel speech enhancement,
X. Hao, X. Su, R. Horaud, and X. Li, “Fullsubnet: A full-band and sub-band fusion model for real-time single-channel speech enhancement,” inICASSP 2021 - 2021 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 6633–6637
2021
-
[9]
Gtcrn: A speech enhancement model requiring ultralow computational re- sources,
X. Rong, T. Sun, X. Zhang, Y . Hu, C. Zhu, and J. Lu, “Gtcrn: A speech enhancement model requiring ultralow computational re- sources,” inICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024, pp. 971–975
2024
-
[10]
Deep filtering: Signal extraction and reconstruction using complex time-frequency filters,
W. Mack and E. A. P. Habets, “Deep filtering: Signal extraction and reconstruction using complex time-frequency filters,”IEEE Signal Processing Letters, vol. 27, pp. 61–65, 2020
2020
-
[11]
Multi-channel multi-frame adl-mvdr for target speech separation,
Z. Zhang, Y . Xu, M. Yu, S.-X. Zhang, L. Chen, D. S. Williamson, and D. Yu, “Multi-channel multi-frame adl-mvdr for target speech separation,”IEEE/ACM Transactions on Audio, Speech, and Lan- guage Processing, vol. 29, pp. 3526–3540, 2021
2021
-
[12]
A two-stage framework in cross- spectrum domain for real-time speech enhancement,
Y . Zhang, H. Zou, and J. Zhu, “A two-stage framework in cross- spectrum domain for real-time speech enhancement,” inICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024, pp. 12 587–12 591
2024
-
[13]
Two heads are bet- ter than one: A two-stage complex spectral mapping approach for monaural speech enhancement,
A. Li, W. Liu, C. Zheng, C. Fan, and X. Li, “Two heads are bet- ter than one: A two-stage complex spectral mapping approach for monaural speech enhancement,”IEEE/ACM Transactions on Au- dio, Speech, and Language Processing, vol. 29, pp. 1829–1843, 2021
2021
-
[14]
First coarse, fine afterward: A lightweight two-stage complex approach for monau- ral speech enhancement,
F. Dang, H. Chen, Q. Hu, P. Zhang, and Y . Yan, “First coarse, fine afterward: A lightweight two-stage complex approach for monau- ral speech enhancement,”Speech Communication, vol. 146, pp. 32–44, 2023
2023
-
[15]
Dual-path trans- former based on efficient channel attention mechanism for speech enhancement,
Z. Zheng, Y . Liu, J. Liu, K. Niu, and Z. He, “Dual-path trans- former based on efficient channel attention mechanism for speech enhancement,” in2023 International Conference on Wireless Communications and Signal Processing (WCSP), 2023, pp. 7–12
2023
-
[16]
Real and imaginary part interaction network for monaural speech enhancement and de-reverberation,
Z. Zhang, C. He, S. Xu, and M. Wang, “Real and imaginary part interaction network for monaural speech enhancement and de-reverberation,” in2023 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 2023, pp. 972–977
2023
-
[17]
Pointwise convolu- tional neural networks,
B.-S. Hua, M.-K. Tran, and S.-K. Yeung, “Pointwise convolu- tional neural networks,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 984–993
2018
-
[18]
A hybrid dsp/deep learning approach to real-time full-band speech enhancement,
J.-M. Valin, “A hybrid dsp/deep learning approach to real-time full-band speech enhancement,” in2018 IEEE 20th International Workshop on Multimedia Signal Processing (MMSP), 2018, pp. 1–5
2018
-
[19]
B. C. Moore,An introduction to the psychology of hearing. Brill, 2012
2012
-
[20]
A convolutional recurrent neural network for real-time speech enhancement,
K. Tan and D. Wang, “A convolutional recurrent neural network for real-time speech enhancement,” inInterspeech 2018, 2018, pp. 3229–3233
2018
-
[21]
Dual-path rnn: efficient long sequence modeling for time-domain single-channel speech sepa- ration,
Y . Luo, Z. Chen, and T. Yoshioka, “Dual-path rnn: efficient long sequence modeling for time-domain single-channel speech sepa- ration,” inICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 46–50
2020
-
[22]
Dpcrn: Dual-path convo- lution recurrent network for single channel speech enhancement,
X. Le, H. Chen, K. Chen, and J. Lu, “Dpcrn: Dual-path convo- lution recurrent network for single channel speech enhancement,” inInterspeech 2021, 2021, pp. 2811–2815
2021
-
[23]
Time- frequency attention for monaural speech enhancement,
Q. Zhang, Q. Song, Z. Ni, A. Nicolson, and H. Li, “Time- frequency attention for monaural speech enhancement,” in ICASSP 2022 - 2022 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP), 2022, pp. 7852– 7856
2022
-
[24]
Squeeze-and-excitation networks,
J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 7132–7141
2018
-
[25]
Inves- tigating rnn-based speech enhancement methods for noise-robust text-to-speech,
C. V . Botinhao, X. Wang, S. Takaki, and J. Yamagishi, “Inves- tigating rnn-based speech enhancement methods for noise-robust text-to-speech,” in9th ISCA Speech Synthesis Workshop, 2016, pp. 159–165
2016
-
[27]
Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs,
A. W. Rix, J. G. Beerends, M. P. Hollier, and A. P. Hekstra, “Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs,” in 2001 IEEE international conference on acoustics, speech, and signal processing. Proceed...
2001
-
[28]
Evaluation of objective quality measures for speech enhancement,
Y . Hu and P. C. Loizou, “Evaluation of objective quality measures for speech enhancement,”IEEE Transactions on audio, speech, and language processing, vol. 16, no. 1, pp. 229–238, 2007
2007
-
[29]
Dccrn: Deep complex convolution recurrent network for phase-aware speech enhancement,
Y . Hu, Y . Liu, S. Lv, M. Xing, S. Zhang, Y . Fu, J. Wu, B. Zhang, and L. Xie, “Dccrn: Deep complex convolution recurrent network for phase-aware speech enhancement,”Interspeech 2020, 2020
2020
-
[30]
Full- subnet+: Channel attention fullsubnet with complex spectrograms for speech enhancement,
J. Chen, Z. Wang, D. Tuo, Z. Wu, S. Kang, and H. Meng, “Full- subnet+: Channel attention fullsubnet with complex spectrograms for speech enhancement,” inICASSP 2022-2022 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 7857–7861
2022
-
[31]
Compnet: Competitive neural network for palmprint recognition using learnable gabor kernels,
X. Liang, J. Yang, G. Lu, and D. Zhang, “Compnet: Competitive neural network for palmprint recognition using learnable gabor kernels,”IEEE Signal Processing Letters, vol. 28, pp. 1739–1743, 2021
2021
-
[32]
Real time speech en- hancement in the waveform domain,
A. D ´efossez, G. Synnaeve, and Y . Adi, “Real time speech en- hancement in the waveform domain,” inInterspeech 2020, 2020, pp. 3291–3295
2020
-
[33]
Phasen: A phase-and- harmonics-aware speech enhancement network,
D. Yin, C. Luo, Z. Xiong, and W. Zeng, “Phasen: A phase-and- harmonics-aware speech enhancement network,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 05, 2020, pp. 9458–9465
2020
-
[34]
Glance and gaze: A col- laborative learning framework for single-channel speech enhance- ment,
A. Li, C. Zheng, L. Zhang, and X. Li, “Glance and gaze: A col- laborative learning framework for single-channel speech enhance- ment,”Applied Acoustics, vol. 187, p. 108499, 2022
2022
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.