REVIEW 4 major objections 8 minor 32 references
Unified Learnable 2D Convolutional Feature Extraction for ASR
T0 review · 4 major / 8 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read A single 2D convolutional front-end, trained from scratch, matches handcrafted-inspired feature extractors for ASR.
desk verdict A genuine architectural simplification for learnable front-ends, but the parity claim is undersold by weak baselines. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The core object is a unified 2D convolutional front-end: a first layer that turns the 1D waveform into a time-frequency representation (either STFT magnitude or a learnable filterbank), followed by six 3x3 2D convolutional layers with stride 2 in time, each doubling subsampling until the 40 ms frame rate. This single stack absorbs both the classical feature extraction and the VGG-style subsampling block, and it is trained jointly with the acoustic model from scratch.
What would settle it
Re-run Table 1 with SCF and wav2vec baselines trained using continuous tempo perturbation with stronger factors (as in prior work, reference [23]), keeping everything else fixed. If SCF drops below 2.3/5.0 WER on dev clean/other while the 2D front-end stays at 2.5/5.5, the paper's central parity claim fails.
Extended reading notes
Core claim
On its own terms, the paper's central claim is that a single stack of 2D 3x3 convolutions, applied directly to the waveform through a learnable first-layer filterbank, can serve as the entire feature extraction front-end for ASR and perform on par with established learnable front-ends. The generic design removes the arbitrary split between 1D waveform feature extraction and 2D VGG-style subsampling, and moves SpecAugment to the STFT domain before the front-end. The result: WERs of 2.5/5.5 on dev clean/other and 2.9/5.9 on test clean/other, statistically on par with SCF (2.6/5.7, 2.9/6.0) and better than the wav2vec extractor in this setup, at a fraction of the parameter count. Analyses of le
Load-bearing premise
The parity claim rests on the SCF and wav2vec baselines being limited by the authors' weak audio perturbation; with the stronger perturbation settings used in prior work, those baselines improve, and the 2D front-end's parity could disappear.
Editorial extensions
If this is right
- ASR front-ends can be built from a single generic 2D convolutional stack, removing the architectural patchwork of 1D front-end plus VGG block.
- Parameter count before the Conformer drops from 12.4M (SCF) or 5.0M (wav2vec) to as little as 0.3M, making fully learnable front-ends feasible on a single 24GB GPU.
- Moving SpecAugment to the STFT domain makes its hyperparameters independent of the feature extractor's output dimension, so the same augmentation works across different front-ends.
- The learned filterbank still develops Gammatone-like bandpass filters, showing a generic design can rediscover classical auditory filters without explicit prior structure.
- The parameter-efficient configuration (0.3M) loses only about 0.3-0.4 WER absolute versus the larger one, offering a clear compute-accuracy trade-off.
Reading between the lines
- If the parity holds, classical Gammatone/log-Mel structure is not a necessary inductive bias for good ASR features; the model can learn equivalent structure directly from the task loss. A testable extension would be applying this generic front-end to other audio tasks and comparing against task-specific front-ends.
- The observed filter-ordering (adjacent ascending/descending center frequencies) suggests the 2D convolutions impose a locality prior on the feature dimension; initializing the filterbank in a frequency-sorted order might accelerate convergence or improve the small-channel configuration.
- The wav2vec-style 1D extractor may be superseded by the simpler 2D stack in resource-constrained settings; a direct comparison on a larger corpus or with a Transformer-encoder acoustic model would test this more broadly.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a unified, randomly initialized 2D convolutional front-end for ASR that replaces both the handcrafted log-Mel/SCF feature extraction and the subsequent VGG-style subsampling block with a single stack of 2D convolutions. On LibriSpeech with a CTC/Conformer acoustic model, the proposed 2D front-end achieves test WERs of 2.9/5.9 (clean/other) versus 2.9/6.0 for the paper's SCF baseline, while using far fewer parameters before the Conformer (2.3M or 0.3M versus 12.4M for SCF and 5.0M for wav2vec). The paper includes ablations on the first-layer type (STFT vs. filterbank, random vs. Gammatone initialization), the number of 2D layers, subsampling factors, kernel size, and channel count, plus an analysis of learned filter frequency responses. The authors conclude that a generic, from-scratch front-end can match existing supervised learnable feature extractors while being parameter-efficient.
Significance. If the parity claim holds under properly tuned training conditions, the contribution is significant: it challenges the necessity of handcrafted inductive bias in neural front-ends and provides a parameter-efficient alternative for resource-constrained ASR. The paper is clearly written, contains detailed and fairly systematic ablations (Tables 2, Figures 1 and 3), and releases code. The learned-filter analysis is a useful qualitative check. However, the significance is conditional on the central comparison being fair, and the paper itself acknowledges that the baselines are under-tuned relative to prior work. The main scientific value is the architecture and the ablation evidence; the headline claim of matching existing extractors is not yet fully established.
major comments (4)
- [Abstract; §4, Table 1] The central parity claim ('matches the performance of existing supervised learnable feature extractors') is not yet supported by the evidence. The paper states in §4 that audio perturbation was not tuned and only speed perturbation with factors {0.9, 1.0, 1.1} was used, which the authors themselves say may explain why the SCF and wav2vec baselines underperform their published results in [11, 20]. The 2D front-end's best WERs are within 0.1–0.2% absolute of this under-tuned SCF baseline. If SCF/wav2vec were retrained with the stronger perturbation used in prior work, they could match or exceed log Mel, while the 2D front-end remains below log Mel in this paper. The abstract should either restrict the claim to 'under the same training recipe' or the authors should retrain the baselines with matched, tuned augmentation.
- [Table 1; §4] The parity conclusion rests on WER differences of 0.1–0.2% absolute (e.g., 2.9 vs. 2.9 on test-clean and 5.9 vs. 6.0 on test-other). No multiple seeds or confidence intervals are reported. In CTC ASR, such small differences are often within run-to-run variation. Please report multiple runs with mean and standard deviation, or clearly state that the differences are not statistically distinguished. This is load-bearing because the paper's main claim is that the 2D front-end is 'on par with SCF.'
- [§6; §5] The conclusion that 'the remaining difference to log Mel features is not due to its missing structure' is an overreach. The evidence is only that the proposed 2D front-end is competitive with under-tuned SCF/wav2vec baselines under the same weak augmentation. The paper itself lists weak audio perturbation as a possible cause of the gap to log Mel in §5. A controlled comparison with a properly tuned log-Mel baseline is needed to attribute the gap to structure rather than to insufficient regularization or tuning effort. The current wording overstates what the experiments show.
- [Table 1; §2.2] The parameter-efficiency comparison is not fully transparent. The reported count 'parameters before the Conformer' includes the linear projection to the Conformer dimension and, for log Mel and SCF, the VGG-style subsampling block, while the proposed 2D front-end subsumes the subsampling. Thus the large parameter saving (12.4M vs. 2.3M/0.3M) partly reflects the lower-dimensional output representation and the architectural unification, not purely the feature extractor itself. Please provide a breakdown of feature-extractor, subsampling/VGG, and projection parameters, and clarify in the text that the comparison is for the entire front-end including projection.
minor comments (8)
- [Table 1] The STFT row lacks a parameter count, and the SpecAugment column entries ('Features' vs. 'STFT'?) are ambiguous. Please align the columns and clarify what is being reported.
- [Table 2] The table layout is garbled; it is unclear which rows correspond to trainable vs. fixed and Gammatone vs. random initialization. Please reformat the table.
- [Figure 1] The axis labels and cell values are hard to read in the text version. A heatmap with explicit axes and a legend would be clearer.
- [Figure 3] The text says that dropping to kernel size 16 'introduces a severe impairment', but the figure caption says the model with kernel size 16 did not converge. Please reconcile; if it did not converge, the WER should not be presented as a valid comparison point.
- [Abstract; §2.2] The characterization of the front-end as 'generic' and 'from scratch' should be qualified: the first-layer kernel size, stride, and number of channels are tuned (Section 2.2, Figure 3), and the architecture still uses a filterbank-like or STFT-like first layer to create the feature dimension. The claim should say 'randomly initialized' rather than implying no handcrafted influence at all.
- [§4.1] The claim that the learned filters show 'clear groups of filters with adjacent ascending or descending center frequencies' is qualitative and based on visual inspection. Consider providing a quantitative measure of ordering or a more systematic analysis.
- [Reproducibility] The code link is a bare repository path. Please include a commit hash or version and a brief description of how to run the experiments.
- [References] Reference [23] is marked 'to appear'. Please update it with the final publication details if available.
Circularity Check
No circularity: the 2D front-end is trained from scratch and compared to independently re-run baselines; no equation or fitted parameter is reused as a prediction.
full rationale
The paper's central claim is an empirical one: a randomly initialized 2D convolutional front-end performs on par with SCF and wav2vec feature extractors when all are trained from scratch on LibriSpeech. No analytical derivation is offered, and no parameter is fitted to a target and then renamed as a prediction. The ablations (Tables 2, Figure 1, Figure 3) vary architectural choices and measure WER on the same pipeline; that is model selection, not circularity. The SCF and wav2vec baselines are drawn from the authors' prior work, but they are re-implemented and re-trained under the same setup, so the comparison does not reduce to a self-citation; it is an internal benchmark. The limitation that audio perturbation was not tuned (Section 4: 'we do not tune the audio perturbation and only applied speed perturbation with rather limited perturbation factors') is an outside-validity concern about whether the baselines represent their published best, not a circularity, because the reported parity is relative to these re-run baselines rather than defined by them. The observation that learned filters resemble Gammatone filters is an analysis, not a renaming of a known result. Hence no circular step can be quoted and no specific reduction exists.
Assumptions & free parameters
free parameters (6)
- First-layer filterbank kernel size =
256 samples (best)
- First-layer filterbank stride =
10 samples (subsampling factor 10)
- Number of first-layer channels =
128 (best), 8 (parameter-efficient)
- Number of subsequent 2D layers =
6 layers with stride 2
- SpecAugment hyperparameters =
tuned separately for STFT-domain and feature-level
- 2D kernel size =
3x3
assumptions (4)
- domain assumption A single Conformer+CTC model on LibriSpeech is a sufficient yardstick for comparing front-ends
- domain assumption Tuning hyperparameters on dev-other provides unbiased final comparisons
- domain assumption WER differences without repeated runs are meaningful
- standard math The STFT or filterbank decomposition preserves information needed for ASR in this setup
Cite this review
Pith. "Pith review of Unified Learnable 2D Convolutional Feature Extraction for ASR." pith.science (2026). https://pith.science/paper/TQWVTRQZ
@misc{pith2026250910031,
author = {Pith},
title = {Pith review of: Unified Learnable 2D Convolutional Feature Extraction for ASR},
year = {2026},
howpublished = {\url{https://pith.science/paper/TQWVTRQZ}},
note = {Machine review of arXiv:2509.10031}
}
read the original abstract
Neural front-ends represent a promising approach to feature extraction for automatic speech recognition (ASR) systems as they enable to learn specifically tailored features for different tasks. Yet, many of the existing techniques remain heavily influenced by classical methods. While this inductive bias may ease the system design, our work aims to develop a more generic front-end for feature extraction. Furthermore, we seek to unify the front-end architecture contrasting with existing approaches that apply a composition of several layer topologies originating from different sources. The experiments systematically show how to reduce the influence of existing techniques to achieve a generic front-end. The resulting 2D convolutional front-end is parameter-efficient and suitable for a scenario with limited computational resources unlike large models pre-trained on unlabeled audio. The results demonstrate that this generic unified approach is not only feasible but also matches the performance of existing supervised learnable feature extractors.
Figures
Reference graph
Works this paper leans on
-
[1]
Conformer: Convolution-augmented transformer for speech recognition,
A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wuet al., “Conformer: Convolution-augmented transformer for speech recognition,” Preprint arXiv:2005.08100, 2020
arXiv 2005
-
[2]
Comparison of parametric representations for monosyllabic word recognition in contin- uously spoken sentences,
S. Davis and P. Mermelstein, “Comparison of parametric representations for monosyllabic word recognition in contin- uously spoken sentences,”IEEE transactions on acoustics, speech, and signal processing, vol. 28, no. 4, pp. 357–366, 1980
1980
-
[3]
Gamma- tone features and feature combination for large vocabulary speech recognition,
R. Schlüter, I. Bezrukov, H. Wagner, and H. Ney, “Gamma- tone features and feature combination for large vocabulary speech recognition,” inProc. ICASSP, Honolulu, HI, USA, Apr. 2007, pp. 649–652
2007
-
[4]
Convolu- tional, long short-term memory, fully connected deep neural networks,
T. N. Sainath, O. Vinyals, A. Senior, and H. Sak, “Convolu- tional, long short-term memory, fully connected deep neural networks,” inProc. ICASSP, Brisbane, Australia, Apr. 2015, pp. 4580–4584
2015
-
[5]
Learning the speech front-end with raw wave- form CLDNNs,
T. N. Sainath, R. J. Weiss, A. Senior, K. W. Wilson, and O. Vinyals, “Learning the speech front-end with raw wave- form CLDNNs,” inSixteenth Annual Conference of the In- ternational Speech Communication Association, 2015
2015
-
[6]
Learning filterbanks from raw speech for phone recognition,
N. Zeghidour, N. Usunier, I. Kokkinos, T. Schaiz, G. Syn- naeve, and E. Dupoux, “Learning filterbanks from raw speech for phone recognition,” inProc. ICASSP, Calgary, Canada, Apr. 2018, pp. 5509–5513
2018
-
[7]
Acoustic modeling of speech waveform based on multi-resolution, neural network signal processing,
Z. Tüske, R. Schlüter, and H. Ney, “Acoustic modeling of speech waveform based on multi-resolution, neural network signal processing,” inProc. ICASSP, Calgary, Canada, Apr. 2018, pp. 4859–4863
2018
-
[8]
LEAF: A learnable frontend for audio classification,
N. Zeghidour, O. Teboul, F. de Chaumont Quitry, and M. Tagliasacchi, “LEAF: A learnable frontend for audio classification,” inProc. ICLR, Vienna, Austria, May 2021
2021
Show all 32 references
-
[9]
Speaker recognition from raw waveform with SincNet,
M. Ravanelli and Y . Bengio, “Speaker recognition from raw waveform with SincNet,” inProc. SLT, Athens, Greece, Dec. 2018, pp. 1021–1028
2018
-
[10]
Acoustic mod- eling with deep neural networks using raw time signal for LVCSR,
Z. Tüske, P. Golik, R. Schlüter, and H. Ney, “Acoustic mod- eling with deep neural networks using raw time signal for LVCSR,” inProc. Interspeech, Singapore, Sep. 2014, pp. 890–894, ISCA Best Student Paper Award
2014
-
[11]
Comparative analysis of the wav2vec 2.0 feature extractor,
P. Vieting, R. Schlüter, and H. Ney, “Comparative analysis of the wav2vec 2.0 feature extractor,” inITG Conference on Speech Communication, Aachen, Germany, Sep. 2023, pp. 131–135
2023
-
[12]
Perception of speech and sound,
B. Kollmeier, T. Brand, and B. Meyer, “Perception of speech and sound,”Springer handbook of speech processing, pp. 61–82, 2008
2008
-
[13]
Perceptual linear predictive (PLP) analysis of speech,
H. Hermansky, “Perceptual linear predictive (PLP) analysis of speech,”The Journal of the Acoustical Society of America, vol. 87, no. 4, pp. 1738–1752, 1990
1990
-
[14]
Very deep convolutional networks for large-scale image recognition
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition.” Computational and Biological Learning Society, 2015, pp. 1–14
2015
-
[15]
Advances in joint CTC-attention based end-to-end speech recognition with a deep CNN encoder and RNN-LM,
T. Hori, S. Watanabe, Y . Zhang, and W. Chan, “Advances in joint CTC-attention based end-to-end speech recognition with a deep CNN encoder and RNN-LM,” inProc. Inter- speech, 2017, pp. 949–953
2017
-
[16]
Very deep convolu- tional networks for end-to-end speech recognition,
Y . Zhang, W. Chan, and N. Jaitly, “Very deep convolu- tional networks for end-to-end speech recognition,” inProc. ICASSP, New Orleans, LA, USA, Mar. 2017, pp. 4845–4849
2017
-
[17]
ESPnet: End-to-end speech processing toolkit,
S. Watanabe, T. Hori, S. Karita, T. Hayashi, J. Nishitoba, Y . Unno, N. Enrique Yalta Soplin, J. Heymann, M. Wies- ner, N. Chen, A. Renduchintala, and T. Ochiai, “ESPnet: End-to-end speech processing toolkit,” inProc. Interspeech, Hyderabad, India, Sep. 2018, pp. 2207–2211
2018
-
[18]
wav2vec: Unsupervised pre-training for speech recogni- tion,
S. Schneider, A. Baevski, R. Collobert, and M. Auli, “wav2vec: Unsupervised pre-training for speech recogni- tion,” inProc. Interspeech, Graz, Austria, Sep. 2019, pp. 3465–3469
2019
-
[19]
wav2vec 2.0: A framework for self-supervised learning of speech rep- resentations,
A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech rep- resentations,” inAdvances in neural information processing systems, vol. 33, Dec. 2020, pp. 12 449–12 460
2020
-
[20]
On architectures and training for raw waveform feature ex- traction in ASR,
P. Vieting, C. Lüscher, W. Michel, R. Schlüter, and H. Ney, “On architectures and training for raw waveform feature ex- traction in ASR,” inProc. ASRU, Cartagena, Colombia, Dec. 2021, pp. 267–274
2021
-
[21]
HuBERT: How much can a bad teacher bene- fit ASR pre-training?
W.-N. Hsu, Y .-H. H. Tsai, B. Bolte, R. Salakhutdinov, and A. Mohamed, “HuBERT: How much can a bad teacher bene- fit ASR pre-training?” inProc. ICASSP. Toronto, Canada: IEEE, Jun. 2021, pp. 6533–6537
2021
-
[22]
SpecAugment: A simple data aug- mentation method for automatic speech recognition,
D. S. Park, W. Chan, Y . Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V . Le, “SpecAugment: A simple data aug- mentation method for automatic speech recognition,” Graz, Austria, pp. 2613–2617, Sep. 2019
2019
-
[23]
Regularizing learnable feature extraction for auto- matic speech recognition,
P. Vieting, M. Kannen, B. Hilmes, R. Schlüter, and H. Ney, “Regularizing learnable feature extraction for auto- matic speech recognition,” inProc. Interspeech, Rotterdam, Netherlands, Aug. 2025, to appear
2025
-
[24]
Layer normalization,
J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” Preprint arXiv:1607.06450, 2016
2016 arXiv
-
[25]
Gaussian error linear units (GELUs),
D. Hendrycks and K. Gimpel, “Gaussian error linear units (GELUs),” Preprint arXiv:1606.08415, 2018
2018 arXiv
-
[26]
Efficient training of neural transducer for speech recognition,
W. Zhou, W. Michel, R. Schlüter, and H. Ney, “Efficient training of neural transducer for speech recognition,” inProc. Interspeech, Incheon, Korea, Sep. 2022, pp. 2058–2062
2022
-
[27]
Lib- riSpeech: An ASR corpus based on public domain audio books,
V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- riSpeech: An ASR corpus based on public domain audio books,” inProc. ICASSP, Brisbane, Australia, Apr. 2015, pp. 5206–5210
2015
-
[28]
Joint-sequence models for grapheme- to-phoneme conversion,
M. Bisani and H. Ney, “Joint-sequence models for grapheme- to-phoneme conversion,”Speech Communication, vol. 50, no. 5, pp. 434–451, 2008
2008
-
[29]
Self-attention with relative position representations,
P. Shaw, J. Uszkoreit, and A. Vaswani, “Self-attention with relative position representations,” inProc. North American Chapter of the ACL, New Orleans, LA, USA, Jun. 2018, pp. 464–468
2018
-
[30]
Decoupled weight decay regu- larization,
I. Loshchilov and F. Hutter, “Decoupled weight decay regu- larization,” inInternational Conference on Learning Repre- sentations, 2019
2019
-
[31]
Flashlight: Enabling innovation in tools for machine learning,
J. D. Kahn, V . Pratap, T. Likhomanenko, Q. Xu, A. Hannun, J. Cai, P. Tomasello, A. Lee, E. Grave, G. Avidov, B. Steiner, V . Liptchinsky, G. Synnaeve, and R. Collobert, “Flashlight: Enabling innovation in tools for machine learning,” inProc. ICML. Baltimore, MD, USA: PMLR, Ju...
2022
-
[32]
An efficient auditory filterbank based on the gamma- tone function,
R. D. Patterson, I. Nimmo-Smith, J. Holdsworth, P. Rice et al., “An efficient auditory filterbank based on the gamma- tone function,” ina meeting of the IOC Speech Group on Auditory Modelling at RSRE, vol. 2, no. 7, 1987
1987
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.