Pith. sign in

REVIEW 3 major objections 4 minor 59 references

In neural speech models, injecting noise into layer outputs makes layer size a differentiable parameter, letting gradient descent prune redundancies during training.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 09:26 UTC pith:Q64YFPO2

load-bearing objection The empirical results are striking, but the core regularizer in the equations points the wrong way: the λ² penalty drives λ to 0 (retain units), not toward complexity reduction, so the central claim is not established by the derivation as written. the 3 major comments →

arxiv 2601.13704 v3 pith:Q64YFPO2 submitted 2026-01-20 cs.SD cs.AIcs.LGeess.AS

Performance and Complexity Trade-off Optimization of Speech Models During Training

Esteban G\'omez , Tom Backstr\"om This is my paper
classification cs.SD cs.AIcs.LGeess.AS
keywords feature noise injectiondynamic complexity layersmodel compressionstructured pruningspeech processingvoice activity detectionaudio anti-spoofingdifferentiable layer size
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that the size of a neural network layer can be treated as a continuous, differentiable parameter instead of a fixed integer, by adding a controlled amount of noise to the layer's last active feature. This makes it possible for ordinary gradient-based training to simultaneously optimize task accuracy and computational cost, without a separate pruning or architecture-search stage. The authors demonstrate the method on a synthetic filter-bank task, voice activity detection, and audio anti-spoofing. In the anti-spoofing case they report an error-rate drop from 6.26% to 2.53% while cutting computational complexity by nearly 84% and model size by 90%. If correct, the approach replaces heuristic layer-size choices and post-hoc compression with an automatic, data-driven sizing process that happens during training.

Core claim

The central claim is that the effective dimensionality of a layer can be controlled by a continuous parameter K, where the integer part determines how many features are fully active and the fractional part λ = K − ⌊K⌋ controls a noise interpolation on the last active feature. The noise is scaled so that the minimum-mean-squared-error estimate of that feature has a normalized error exactly equal to λ, meaning the vector's information content is a linear function of K. Because the interpolation is differentiable, backpropagation can update K through an L2 penalty on λ, letting the network discover how many features each layer truly needs. Training starts with fixed maximum complexity and then

What carries the argument

Dynamic Complexity Layers (DCL) are the mechanism. A DCL keeps weights for the maximum allowed size but applies the noise-interpolation formula (Eq. 1) so that its effective output dimension is the continuous variable K ∈ [Nmin, Nmax]. Its fractional part λ = K − ⌊K⌋ sets the noise level on the boundary feature, making the number of informative outputs a differentiable quantity. Adaptive Complexity Layers (ACL) adjust their input shape to match a preceding DCL's current size, and both types can be consolidated into fixed-complexity layers at inference. The L2 penalty on λ in the loss function is what encodes the performance-complexity trade-off.

Load-bearing premise

The method's success rests on the assumption that adding noise scaled by λ to the boundary feature is a faithful, differentiable proxy for how much information that feature truly carries in deep nonlinear networks; if the gradient of the task loss with respect to λ points the wrong way, the optimizer may prune features that later turn out to be essential.

What would settle it

Take a network with a known optimal architecture (e.g., a linear filter-bank task where the minimum necessary rank is known exactly), train with the proposed method and large β, and check whether the learned K converges to the true rank for many random initializations. A stronger test: for a given trained model, compare the loss gradient with respect to each λ against a brute-force evaluation where the layer is actually truncated to each integer size; if the signs disagree on more than a small fraction of layers, the proxy is misleading.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Speech models can learn their own layer widths from data, replacing heuristic choices about filter-bank bands, channel counts, and layer capacities.
  • Substantial complexity reduction (up to ~84% in the reported anti-spoofing experiment) can be achieved without a separate post-hoc pruning pass.
  • In the anti-spoofing case, removing redundant capacity improved generalization, lowering EER from 6.26% to 2.53% while shrinking the model by 90%.
  • The method is compatible with existing compression techniques such as quantization and knowledge distillation, potentially compounding gains.
  • Because the optimization is performed in a single training run, the method avoids the iterative retraining loops typical of structured pruning.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The noise-interpolation proxy for feature importance is not obviously limited to speech; the same technique could generalize to vision or NLP models, where structured pruning currently relies on heuristic importance scores.
  • If the learned λ values correlate well with post-hoc importance measures, the method could serve as an interpretability tool, revealing which layers and channels truly carry the task's information.
  • The two-phase training schedule (fixed complexity first, then open λ) suggests a warm-start recipe; a testable extension would be to anneal β or start λ optimization earlier to see if further complexity savings are possible.
  • The claim that reducing redundancy improved anti-spoofing generalization hints that overparameterization itself can hurt on out-of-distribution spoof attacks; this could be probed by comparing trained fixed-size models at the same complexity.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a differentiable method for jointly optimizing model performance and computational complexity during training. The key idea is 'Dynamic Complexity Layers' (DCL): the output dimensionality of a layer is made continuous by writing the last active feature as a noisy interpolation, η = sqrt(1−λ)ξ + sqrt(λ)σξν, with λ = K − floor(K), and then adding an L2 penalty on λ to the training loss. Subsequent 'Adaptive Complexity Layers' (ACL) adjust their input shapes accordingly, and after training all layers are consolidated to fixed shapes. The method is demonstrated on a synthetic filter-bank task, a voice activity detection CRNN, and an ASVspoof 2019 ResNet34 model. The paper reports substantial complexity and model-size reductions, including an 84% FLOP/s reduction and 90% parameter reduction in anti-spoofing, with EER improving from 6.26% to 2.53%.

Significance. If the central idea were made internally consistent, it would be a valuable contribution: a single-run, SGD-compatible alternative to post-hoc pruning and neural architecture search, applicable across layer types. The synthetic experiment is a useful sanity check, and the two speech case studies cover practical architectures. The intent to release code is also a strength, although the URL is currently a placeholder. However, as written, the method section contains a sign/parameterization mismatch that appears to make the proposed regularizer push in the opposite direction of the reported complexity reductions, and the 'effective dimensionality' claim in Section III-A is not supported by the construction. The empirical evaluation is also missing error bars and matched-complexity baselines. These issues are load-bearing for the central claim, so the paper needs substantive revision rather than minor editing.

major comments (3)
  1. There is a fundamental sign/parameterization mismatch. Eq. (1) defines η = sqrt(1−λ)ξ + sqrt(λ)σξν, and Eq. (3) correctly shows that the normalized MMSE error for estimating ξ from η is λ. Thus λ is a noise fraction: λ=0 means the feature is fully retained, λ=1 means it is pure noise. The training losses in Eqs. (6) and (8) add β/N Σ λ². Since ∂(λ²)/∂λ = 2λ > 0, gradient descent drives λ toward 0, i.e., toward maximal retention. This is the opposite of the claimed complexity reduction. Section IV.C's statement that larger β enables more aggressive complexity reduction, and Fig. 2's description of λ=0.7 as 'capacity reduced by 30%', are inconsistent with the formal definitions. Under the alternative reading where λ=K−floor(K), the penalty is a sawtooth with minima at integers; crossing an integer dimension boundary downward incurs a penalty jump, so the regularizer penalizes exactly the d
  2. The text claims 'the effective dimensionality of x_K ... is therefore K.' This does not follow from the construction. The vector x_K has ⌊K⌋−1 fully informative elements and one partially informative element whose signal fraction is 1−λ, so the effective number of informative components is ⌊K⌋−λ, not K. For example, with K=3.7 and λ=0.7 the effective dimensionality is 2.3, not 3.7. Consequently, the mapping from (K,λ) to actual layer size is undefined, and the statements 'λ=0.7 means capacity reduced by 30%' and the 'minimum complexity percentage' labels in Sections V and VI have no rigorous basis. The paper also does not specify how a final fractional K is consolidated into an integer-sized layer at inference (rounding, truncation, or something else). Without this mapping and consolidation rule, the reported FLOP/s and parameter-count reductions cannot be verified.
  3. The empirical evaluation is reported as a single run per configuration, with no standard deviation, no number of random seeds, and no matched-complexity baseline. The headline anti-spoofing result (EER from 6.26% to 2.53% with an 84% FLOP/s reduction and 90% parameter reduction) is one model selected on the development set. To support the claim that dynamic complexity layers discover better architectures than heuristic layer-size choices, the paper should (i) report mean±std over at least three seeds, (ii) compare against standard structured-pruning or width-scaling baselines at the same final FLOP/s and parameter budgets, and (iii) include an ablation with β=0 and with a direct penalty on a conventional complexity measure, so that the contribution of the noise-injection proxy is isolated from the simple effect of the regularizer. Without these, the quantitative claims are not yet establ
minor comments (4)
  1. The sentence 'Conversely, the signal-to-noise ratio is λ^{-1}' is not consistent with Eq. (1). The SNR of η is (1−λ)/λ. This typo contributes to the sign ambiguity and should be corrected.
  2. The abstract states that the code is publicly available, but Footnote 1 says 'URL will be provided upon publication'. Please provide a working repository URL or soften the public-availability claim.
  3. The paper uses 'FLOP/s' throughout, but FLOP/s is a rate (operations per second), whereas the experiments report an operation count per inference (FLOPs). Please use consistent units (e.g., 'FLOPs per inference' or 'MACs per frame').
  4. The comparison to the top-ten ASVspoof 2019 submissions is not apples-to-apples: those are ensemble systems and their complexity is not reported. The paper acknowledges this in passing but still says the result is 'comparable to top-ranked submissions'; please state the single-system, complexity-matched caveat explicitly in the main text.

Circularity Check

0 steps flagged

No significant circularity: empirical results are externally evaluated and no prediction reduces to a fitted input.

full rationale

The paper's derivation chain (Eqs. 1–3) establishes that the MMSE of estimating ξ from η equals λ = K − ⌊K⌋, and the statement that 'the effective dimensionality ... is therefore K' is a definitional restatement of this relationship, not a fitted prediction or a result smuggled in from external data. The headline complexity reductions and EER improvements are measured on held-out sets (VCTK+ESC-50, ASVspoof19 LA evaluation) and FLOPs are computed with the moduleprofiler tool, so these results are not constructed from the training objective. The L2 penalty on λ (Eqs. 6 and 8) is a training mechanism, not a circular input; however, as written it penalizes the fractional part of K and therefore does not by itself create a gradient that reduces the integer layer size ⌊K⌋. That is a substantive correctness/consistency concern, but it is not a circularity in the sense of a prediction being equivalent to its inputs by construction. Self-citations [36] and [48] are background and tooling respectively and are not load-bearing for the central derivation. No fitted parameter is renamed as a prediction, and no uniqueness theorem is imported from the authors' prior work. Score 0.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 0 invented entities

The central claim rests on the noise-importance proxy and the L2 penalty; both are modeling choices. No new physical entities are introduced; DCL and ACL are software abstractions.

free parameters (4)
  • Per-feature retention knob λ_n = Not reported per unit; final layer sizes reported
    Each DCL's fractional widths are learned by SGD with an L2 penalty; these directly determine the final model size.
  • Regularization weight β = 0.5 (VAD), 1.0 (anti-spoofing)
    Chosen by hand; controls the performance-complexity trade-off.
  • Minimum complexity percentage (λ_min) = 50%, 25%, 12.5%, 6.25%
    User-specified lower bound; multiple runs differ only in this value.
  • L1 loss scale in synthetic experiment = 1e-3
    Introduced to prevent large gradients from dominating the small synthetic model.
axioms (3)
  • domain assumption The Gaussian linear MMSE model of feature information (Eqs. 1–3) justifies that effective dimensionality is K and that λ is a linear information knob.
    Exact only for a single linear Gaussian channel; assumed to transfer to trained deep networks without proof.
  • ad hoc to paper L2 penalty on λ is a valid complexity regularizer.
    The objective (Eqs. 6 and 8) assumes minimizing Σλ² approximates minimizing FLOPs; no derivation is given, and final FLOPs are measured post hoc with moduleprofiler.
  • domain assumption Post-training consolidation is safe: units with high noise tolerance can be removed after training.
    The paper tests this empirically but provides no theoretical guarantee that truncating DCL/ACL layers to their final sizes preserves performance.

pith-pipeline@v1.3.0-alltime-deepseek · 16037 in / 12964 out tokens · 127338 ms · 2026-08-03T09:26:19.605345+00:00 · methodology

0 comments
read the original abstract

In speech machine learning, neural network models are typically designed by choosing an architecture with fixed layer sizes and structure. These models are then trained to maximize performance on metrics aligned with the task's objective. While the overall architecture is usually guided by prior knowledge of the task, the sizes of individual layers are often chosen heuristically. However, this approach does not guarantee an optimal trade-off between performance and computational complexity; consequently, post hoc methods such as weight quantization or model pruning are typically employed to reduce computational cost. This occurs because stochastic gradient descent (SGD) methods can only optimize differentiable functions, while factors influencing computational complexity, such as layer sizes and floating-point operations per second (FLOP/s), are non-differentiable and require modifying the model structure during training. We propose a reparameterization technique based on feature noise injection that enables joint optimization of performance and computational complexity during training using SGD-based methods. Unlike traditional pruning methods, our approach allows the model size to be dynamically optimized for a target performance-complexity trade-off, without relying on heuristic criteria to select which weights or structures to remove. We demonstrate the effectiveness of our method through three case studies, including a synthetic example and two practical real-world applications: voice activity detection and audio anti-spoofing. The code related to our work is publicly available to encourage further research.

Figures

Figures reproduced from arXiv: 2601.13704 by Esteban G\'omez, Tom Backstr\"om.

Figure 1
Figure 1. Figure 1: Dynamic (DCL), adaptive (ACL) and fixed (FCL) complexity layers. During training, DCL can modify their output shape, whereas ACL will adapt [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Training progression of a single linear DCL to approximate a 32-band Bark filter bank. The left image shows early results after 100 steps. The middle [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overparameterization factor reduction for different increasing values [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Speech presence probability network. (a) represents the fixed com [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Comparison between CRNNVAD baseline (red line) and DCRNNVAD [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comparison between (a) Fixed complexity ResNet34, and (b) DynResNet34 (ours). Blocks with dotted line borders are resized at training time as [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Comparison of complexity, model size, and performance of DynResNet34 (DRN) models against our baseline ResNet34 (RN). The x-axis shows the [PITH_FULL_IMAGE:figures/full_fig_p009_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

59 extracted references · 7 canonical work pages · 1 internal anchor

  1. [1]

    Speech enhancement algorithms: A systematic literature review,

    S. T. Yousif and B. M. Mahmmod, “Speech enhancement algorithms: A systematic literature review,”Algorithms, vol. 18, no. 5, p. 272,

  2. [2]

    A survey on speech deepfake detection,

    M. Li, Y . Ahmadiadli, and X.-P. Zhang, “A survey on speech deepfake detection,”ACM Computing Surveys, vol. 58, no. 1, pp. 1–39, 2025

    2025

  3. [3]

    Automatic speech recognition using advanced deep learning approaches: A survey,

    H. Kheddar, M. Hemis, and Y . Himeur, “Automatic speech recognition using advanced deep learning approaches: A survey,”Information fusion, vol. 109, p. 102422, 2024

    2024

  4. [5]

    Reshape dimensions network for speaker recognition,

    I. Yakovlev, R. Makarov, A. Balykin, P. Malov, A. Okhotnikov, and N. Torgashov, “Reshape dimensions network for speaker recognition,” inInterspeech 2024. Kos, Greece: ISCA, Sep. 2024, pp. 3235–3239. [Online]. Available: https://doi.org/10.21437/Interspeech.2024-2116

    doi:10.21437/interspeech.2024-2116 2024

  5. [6]

    Exploring the accuracy – energy trade-off in machine learning,

    A. E. Brownlee, J. Adair, S. O. Haraldsson, and J. Jabbo, “Exploring the accuracy – energy trade-off in machine learning,” in2021 IEEE/ACM International Workshop on Genetic Improvement (GI), 2021, pp. 11–18. [Online]. Available: https://doi.org/10.1109/GI52543.2021.00011

    arXiv 2021

  6. [7]

    Dropout: a simple way to prevent neural networks from over- fitting,

    N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhut- dinov, “Dropout: a simple way to prevent neural networks from over- fitting,”The journal of machine learning research, vol. 15, no. 1, pp. 1929–1958, 2014

    1929

  7. [8]

    L2 regularization for learning kernels,

    C. Cortes, M. Mohri, and A. Rostamizadeh, “L2 regularization for learning kernels,” inProceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, ser. UAI ’09. Arlington, Virginia, USA: AUAI Press, 2009, p. 109–116

    2009

  8. [9]

    Quantizing deep convolutional networks for efficient inference: A whitepaper,

    R. Krishnamoorthi, “Quantizing deep convolutional networks for efficient inference: A whitepaper,”arXiv preprint arXiv:1806.08342, vol. 2, 1806. [Online]. Available: https://doi.org/10.48550/arXiv.1806. 08342

    Pith/arXiv arXiv doi:10.48550/arxiv.1806.08342

  9. [10]

    A survey of quantization methods for efficient neural network infer- ence,

    A. Gholami, S. Kim, Z. Dong, Z. Yao, M. W. Mahoney, and K. Keutzer, “A survey of quantization methods for efficient neural network infer- ence,” inLow-power computer vision. Chapman and Hall/CRC, 2022, pp. 291–326

    2022

  10. [11]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,”arXiv preprint arXiv:1503.02531, 2015. [Online]. Available: https://doi.org/10.48550/arXiv.1503.02531

    Pith/arXiv arXiv doi:10.48550/arxiv.1503.02531 2015

  11. [12]

    Knowledge distillation: A survey,

    J. Gou, B. Yu, S. J. Maybank, and D. Tao, “Knowledge distillation: A survey,”International journal of computer vision, vol. 129, no. 6, pp. 1789–1819, 2021

    2021

  12. [13]

    Learning both weights and con- nections for efficient neural network,

    S. Han, J. Pool, J. Tran, and W. Dally, “Learning both weights and con- nections for efficient neural network,”Advances in neural information processing systems, vol. 28, 2015

    2015

  13. [14]

    Hardware acceleration of sparse and irregular tensor computations of ml models: A survey and insights,

    S. Dave, R. Baghdadi, T. Nowatzki, S. Avancha, A. Shrivastava, and B. Li, “Hardware acceleration of sparse and irregular tensor computations of ml models: A survey and insights,”Proceedings of the IEEE, vol. 109, no. 10, pp. 1706–1752, 2021. [Online]. Available: https://doi.org/10.1109/JPROC.2021.3098483

    arXiv 2021

  14. [15]

    Compressing Deep Neural Networks via Layer Fusion

    J. O. Neill, G. V . Steeg, and A. Galstyan, “Compressing deep neural networks via layer fusion,”arXiv preprint arXiv:2007.14917, 2020. [Online]. Available: https://doi.org/10.48550/arXiv.2007.14917

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2007.14917 2007

  15. [16]

    The lottery ticket hypothesis: Finding sparse, trainable neural networks,

    J. Frankle and M. Carbin, “The lottery ticket hypothesis: Finding sparse, trainable neural networks,” inInternational Conference on Learning Representations, 2019

    2019

  16. [17]

    Optimal brain damage,

    Y . LeCun, J. Denker, and S. Solla, “Optimal brain damage,”Advances in neural information processing systems, vol. 2, 1989

    1989

  17. [18]

    Pruning convolutional neural networks for resource efficient inference,

    P. Molchanov, S. Tyree, T. Karras, T. Aila, and J. Kautz, “Pruning convolutional neural networks for resource efficient inference,” inIn- ternational Conference on Learning Representations (ICLR), 2017

    2017

  18. [19]

    To prune, or not to prune: exploring the efficacy of pruning for model compression,

    M. Zhu and S. Gupta, “To prune, or not to prune: exploring the efficacy of pruning for model compression,”International Conference on Learning Representations (ICLR), 2017

    2017

  19. [20]

    The state of sparsity in deep neural networks,

    T. Gale, E. Elsen, and S. Hooker, “The state of sparsity in deep neural networks,”arXiv preprint cs.LG/1902.09574, 2019. [Online]. Available: https://doi.org/10.48550/arXiv.1902.09574

    Pith/arXiv arXiv doi:10.48550/arxiv.1902.09574 1902

  20. [21]

    Dynamic model pruning with feedback,

    T. Lin, S. U. Stich, L. Barba, D. Dmitriev, and M. Jaggi, “Dynamic model pruning with feedback,”International Conference on Learning Representations (ICLR), 2020

    2020

  21. [22]

    A unified framework for soft threshold pruning,

    Y . Chen, Z. Ma, W. Fang, X. Zheng, Z. Yu, and Y . Tian, “A unified framework for soft threshold pruning,”International Conference on Learning Representations (ICLR), 2023

    2023

  22. [23]

    End-to-end speech translation with knowledge distillation,

    Y . Liu, H. Xiong, J. Zhang, Z. He, H. Wu, H. Wang, and C. Zong, “End-to-end speech translation with knowledge distillation,” inInterspeech, 2019, pp. 1128–1132. [Online]. Available: https: //doi.org/10.21437/Interspeech.2019-2582

    work page doi:10.21437/interspeech.2019-2582 2019

  23. [24]

    Distil-whisper: Robust knowledge distillation via large-scale pseudo labelling,

    S. Gandhi, P. V on Platen, and A. M. Rush, “Distil-whisper: Robust knowledge distillation via large-scale pseudo labelling,” arXiv preprint arXiv:2311.00430, 2023. [Online]. Available: https: //doi.org/10.48550/arXiv.2311.00430

    Pith/arXiv arXiv doi:10.48550/arxiv.2311.00430 2023

  24. [25]

    Learning from yourself: A self-distillation method for fake speech detection,

    J. Xue, C. Fan, J. Yi, C. Wang, Z. Wen, D. Zhang, and Z. Lv, “Learning from yourself: A self-distillation method for fake speech detection,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023, pp. 1–5. [Online]. Available: https://doi.org/10.1109/ICASSP49357.2023.10096837

    arXiv 2023

  25. [26]

    Knowledge distillation- based training of speech enhancement for noise-robust automatic speech recognition,

    G. Woo Lee, H. Kook Kim, and D.-J. Kong, “Knowledge distillation- based training of speech enhancement for noise-robust automatic speech recognition,”IEEE Access, vol. 12, pp. 72 707–72 720, 2024. [Online]. Available: https://doi.org/10.1109/ACCESS.2024.3403761

    arXiv 2024

  26. [27]

    Neural architecture search: Insights from 1000 papers,

    C. White, M. Safari, R. Sukthanker, B. Ru, T. Elsken, A. Zela, D. Dey, and F. Hutter, “Neural architecture search: Insights from 1000 papers,”arXiv preprint arXiv:2301.08727, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2301.08727

    Pith/arXiv arXiv doi:10.48550/arxiv.2301.08727 2023

  27. [29]

    Nas-scae: Searching compact attention-based encoders for end-to-end automatic speech recognition,

    Y . Liu, T. Li, P. Zhang, and Y . Yan, “Nas-scae: Searching compact attention-based encoders for end-to-end automatic speech recognition,” inInterspeech, 2022, pp. 1011–1015. [Online]. Available: https://doi.org/10.21437/Interspeech.2022-748

    work page doi:10.21437/interspeech.2022-748 2022

  28. [30]

    Nas-tasnet: Neural architecture search for time-domain speech separation,

    J.-H. Lee, J.-H. Chang, J.-M. Yang, and H.-G. Moon, “Nas-tasnet: Neural architecture search for time-domain speech separation,”IEEE Access, vol. 10, pp. 56 031–56 043, 2022. [Online]. Available: https://doi.org/10.1109/ACCESS.2022.3176003

    arXiv 2022

  29. [31]

    Nas-bench-101: Towards reproducible neural architecture search,

    C. Ying, A. Klein, E. Christiansen, E. Real, K. Murphy, and F. Hutter, “Nas-bench-101: Towards reproducible neural architecture search,” in International conference on machine learning. PMLR, 2019, pp. 7105– 7114

    2019

  30. [32]

    Dynamic neural networks: A survey,

    Y . Han, G. Huang, S. Song, L. Yang, H. Wang, and Y . Wang, “Dynamic neural networks: A survey,”IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 11, pp. 7436–7456,

  31. [33]

    Dynamic nsnet2: Efficient deep noise suppression with early exiting,

    R. Miccini, A. Zniber, C. Laroche, T. Piechowiak, M. Schoeberl, L. Pezzarossa, O. Karrakchou, J. Sparsø, and M. Ghogho, “Dynamic nsnet2: Efficient deep noise suppression with early exiting,” in2023 IEEE 33rd International Workshop on Machine Learning for Signal Processing (MLSP). IEEE, 2023, pp. 1–6. [Online]. Available: https://doi.org/10.1109/MLSP55844....

    arXiv 2023

  32. [34]

    Adaptive slimming for scalable and efficient speech enhancement,

    R. Miccini, M. Kim, C. Laroche, L. Pezzarossa, and P. Smaragdis, “Adaptive slimming for scalable and efficient speech enhancement,” IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2025. [Online]. Available: https://doi.org/10.1109/ W ASPAA66052.2025.11230950 IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, V...

    arXiv 2025

  33. [35]

    A. V . Oppenheim, A. S. Willsky, and S. H. Nawab,Signals & systems. Pearson Educaci ´on, 1997

    1997

  34. [36]

    B ¨ackstr¨om, O

    T. B ¨ackstr¨om, O. R ¨as¨anen, A. Zewoudie, P. P. Zarazaga, L. Koivusalo, S. Das, E. G. Mellado, M. B. Mansali, D. Ramos, S. Kadiri, P. Alku, and M. H. Vali,Introduction to Speech Processing, 2nd ed., 2022. [Online]. Available: https://speechprocessingbook.aalto.fi

    2022

  35. [37]

    Recurrent neural networks for voice activity detection,

    T. Hughes and K. Mierle, “Recurrent neural networks for voice activity detection,” in2013 IEEE International Conference on Acoustics, Speech and Signal Processing, 2013, pp. 7378–7382. [Online]. Available: https://doi.org/10.1109/ICASSP.2013.6639096

    arXiv 2013

  36. [38]

    A convolutional neural network smartphone app for real-time voice activity detection,

    A. Sehgal and N. Kehtarnavaz, “A convolutional neural network smartphone app for real-time voice activity detection,”IEEE Access, vol. 6, pp. 9017–9026, 2018. [Online]. Available: https://doi.org/10. 1109/ACCESS.2018.2800728

    arXiv 2018

  37. [39]

    Real-life voice activity detection with lstm recurrent neural networks and an application to hollywood movies,

    F. Eyben, F. Weninger, S. Squartini, and B. Schuller, “Real-life voice activity detection with lstm recurrent neural networks and an application to hollywood movies,” in2013 IEEE International Conference on Acoustics, Speech and Signal Processing, 2013, pp. 483–487. [Online]. Available: https://doi.org/10.1109/ICASSP.2013.6637694

    arXiv 2013

  38. [40]

    Deep neural networks for voice activity detection,

    S. Mihalache, I.-A. Ivanov, and D. Burileanu, “Deep neural networks for voice activity detection,” in2021 44th International Conference on Telecommunications and Signal Processing (TSP), 2021, pp. 191–194. [Online]. Available: https://doi.org/10.1109/TSP52935.2021.9522670

    arXiv 2021

  39. [41]

    V oice activity detection in the wild: A data-driven approach using teacher-student training,

    H. Dinkel, S. Wang, X. Xu, M. Wu, and K. Yu, “V oice activity detection in the wild: A data-driven approach using teacher-student training,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 1542–1555, 2021. [Online]. Available: https://doi.org/10.1109/TASLP.2021.3073596

    arXiv 2021

  40. [42]

    EARS: An anechoic fullband speech dataset benchmarked for speech enhancement and dereverberation,

    J. Richter, Y .-C. Wu, S. Krenn, S. Welker, B. Lay, S. Watanabe, A. Richard, and T. Gerkmann, “EARS: An anechoic fullband speech dataset benchmarked for speech enhancement and dereverberation,” inISCA Interspeech, 2024, pp. 4873–4877. [Online]. Available: https://doi.org/10.21437/Interspeech.2024-153

    work page doi:10.21437/interspeech.2024-153 2024

  41. [43]

    Pyroomacoustics: A python package for audio room simulation and array processing algorithms,

    R. Scheibler, E. Bezzam, and I. Dokmani ´c, “Pyroomacoustics: A python package for audio room simulation and array processing algorithms,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018, pp. 351–355. [Online]. Available: https://doi.org/10.1109/ICASSP.2018.8461310

    arXiv 2018

  42. [45]

    CSTR VCTK corpus: English multi-speaker corpus for CSTR voice cloning toolkit (version 0.92),

    J. Yamagishi, C. Veaux, and K. MacDonald, “CSTR VCTK corpus: English multi-speaker corpus for CSTR voice cloning toolkit (version 0.92),”Edinburgh DataShare. [Online]. Available: https://datashare.ed.ac.uk/handle/10283/3443

  43. [46]

    ESC: Dataset for Environmental Sound Classification,

    K. J. Piczak, “ESC: Dataset for Environmental Sound Classification,” inProceedings of the 23rd Annual ACM Conference on Multimedia. ACM Press, pp. 1015–1018. [Online]. Available: http://dl.acm.org/ citation.cfm?doid=2733373.2806390

    arXiv

  44. [47]

    Silero V AD: pre-trained enterprise-grade voice activity detec- tor (V AD), number detector and language classifier,

    S. Team, “Silero V AD: pre-trained enterprise-grade voice activity detec- tor (V AD), number detector and language classifier,” https://github.com/ snakers4/silero-vad, 2024

    2024

  45. [48]

    moduleprofiler,

    E. G ´omez, “moduleprofiler,” 2024. [Online]. Available: https://github. com/eagomez2/moduleprofiler

    2024

  46. [49]

    Audio anti-spoofing detection: A survey,

    M. Li, Y . Ahmadiadli, and X.-P. Zhang, “Audio anti-spoofing detection: A survey,”arXiv preprint arXiv:2404.13914, 2024

    Pith/arXiv arXiv 2024

  47. [50]

    Rw-resnet: A novel speech anti-spoofing model using raw waveform,

    Y . Ma, Z. Ren, and S. Xu, “Rw-resnet: A novel speech anti-spoofing model using raw waveform,” inInterspeech, 2021, pp. 4144–4148. [Online]. Available: https://doi.org/10.21437/Interspeech.2021-438

    work page doi:10.21437/interspeech.2021-438 2021

  48. [51]

    End-to-end anti-spoofing with rawnet2,

    H. Tak, J. Patino, M. Todisco, A. Nautsch, N. Evans, and A. Larcher, “End-to-end anti-spoofing with rawnet2,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 6369–6373. [Online]. Available: https://doi.org/10.1109/ ICASSP39728.2021.9414234

    arXiv 2021

  49. [52]

    AASIST: Audio anti-spoofing using integrated spectro-temporal graph attention networks,

    J.-w. Jung, H.-S. Heo, H. Tak, H.-j. Shim, J. S. Chung, B.- J. Lee, H.-J. Yu, and N. Evans, “AASIST: Audio anti-spoofing using integrated spectro-temporal graph attention networks,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 6367–6371. [Online]. Available: https://doi.org/10.1109/ICASSP43922.2022.9747766

    arXiv 2022

  50. [53]

    Audio anti-spoofing based on audio feature fusion,

    J. Zhang, G. Tu, S. Liu, and Z. Cai, “Audio anti-spoofing based on audio feature fusion,”Algorithms, vol. 16, no. 7, p. 317, 2023. [Online]. Available: https://doi.org/10.3390/a16070317

    work page doi:10.3390/a16070317 2023

  51. [54]

    Frame-to-utterance convergence: A spectra-temporal approach for unified spoofing detection,

    A. Khan, K. M. Malik, and S. Nawaz, “Frame-to-utterance convergence: A spectra-temporal approach for unified spoofing detection,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 10 761–10 765. [Online]. Available: https://doi.org/10.1109/ICASSP48485.2024.10447500

    arXiv 2024

  52. [55]

    ASVspoof 5: Crowdsourced speech data, deepfakes, and adversarial attacks at scale,

    X. Wang, H. Delgado, H. Tak, J.-w. Jung, H.-j. Shim, M. Todisco, I. Kukanov, X. Liu, M. Sahidullah, T. Kinnunenet al., “ASVspoof 5: Crowdsourced speech data, deepfakes, and adversarial attacks at scale,” inThe Automatic Speaker Verification Spoofing Countermeasures Workshop (ASVspoof 2024), 2024, pp. 1–8. [Online]. Available: https://doi.org/10.21437/ASVs...

    doi:10.21437/asvspoof.2024-1 2024

  53. [56]

    ASVspoof 2019: Automatic speaker verification spoofing and coun- termeasures challenge evaluation plan,

    J. Yamagishi, M. Todisco, M. Sahidullah, H. Delgado, X. Wang, N. Evans, T. Kinnunen, K. A. Lee, V . Vestman, and A. Nautsch, “ASVspoof 2019: Automatic speaker verification spoofing and coun- termeasures challenge evaluation plan,”ASV Spoof, vol. 13, 2019

    2019

  54. [57]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016. [Online]. Available: https://doi.org/10.1109/CVPR.2016.90

    doi:10.1109/cvpr.2016.90 2016

  55. [58]

    Attentive statistics pooling for deep speaker embedding,

    K. Okabe, T. Koshinaka, and K. Shinoda, “Attentive statistics pooling for deep speaker embedding,”Interspeech, 2018

    2018

  56. [59]

    ASVspoof 2019: Future horizons in spoofed and fake audio detection,

    M. Todisco, X. Wang, V . Vestman, M. Sahidullah, H. Delgado, A. Nautsch, J. Yamagishi, N. Evans, T. Kinnunen, and K. A. Lee, “ASVspoof 2019: Future horizons in spoofed and fake audio detection,” Interspeech, 2019. Esteban G´omezreceived his bachelor’s degree from Universidad de Chile in 2015, a master’s degree from Berklee College of Music in 2017, and a ...

    2019

  57. [62]

    Security and Privacy in Speech Communication

    He is currently pursuing a Ph.D. degree with the Department of Information and Communications Engineering at Aalto University, Finland. He has contributed to the research and development of low- complexity, real-time speech enhancement systems in collaboration with several companies. His re- search interests include antispoofing systems and real-time spee...

    2001

  58. [2021]

    Available: https://doi.ieeecomputersociety.org/10.1109/ TPAMI.2021.3117837

    [Online]. Available: https://doi.ieeecomputersociety.org/10.1109/ TPAMI.2021.3117837

    arXiv 2021

  59. [2025]

    Available: https://doi.org/10.3390/a18050272

    [Online]. Available: https://doi.org/10.3390/a18050272

    work page doi:10.3390/a18050272