Pith. sign in

REVIEW 3 major objections 6 minor 55 references

Unfolding A Few Structures for The Many: Memory-Efficient Compression of Conformer and Speech Foundation Models

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A small seed model, trained only at its shallowest and deepest unfoldings, can be executed at any depth in between and match individually trained systems, cutting parameters by up to 35%.

desk verdict A solid weight-sharing compression result with plausible WER gains, but the training-memory claim overreaches because activations scale with unfolded depth. read the letter →

arxiv 2505.21237 v1 pith:6OIJLWVH submitted 2025-05-27 cs.SD eess.AS

Zhaoqing Li , Haoning Xu , Xurong Xie , Zengrui Jin , Tianzi Wang , Xunying Liu This is my paper
classification cs.SDeess.AS
keywords networkstructureunfoldingmodelcompressionspeechrecognitionConformerwav2vec2HuBERTself-distillationweightsharing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Network structure unfolding compresses an ASR or speech foundation model by training a compact seed model and letting it share its physical layers across repeated uses. The paper claims that jointly training the seed at its own depth and at the maximum unfolded depth, with a Kullback-Leibler (KL) self-distillation term, makes every intermediate depth usable from one stored set of weights. On Switchboard, a 6-layer foldable Conformer unfolded to depth 12 reaches 12.64% average word error rate versus 12.86% for the 12-layer baseline, a 35% parameter reduction without a statistically significant WER increase; an 8-layer foldable wav2vec2 reaches 5.48% test-clean word error rate versus 5.41% for the baseline, a 30% reduction. The practical payoff is that one small model can serve devices with different compute budgets instead of training and storing one system per depth.

What carries the argument

The carrying mechanism is the foldable network: a set of $n_p$ physical layers that can each be repeated $k$ times, with all repetitions sharing the same weights, so the model's logical depth $n_f$ can be changed at inference without changing the stored parameters. The training recipe is Eq. (4), which combines the seed loss at depth $n_p$, the unfolded loss at maximum depth $N_f$, and a KL-divergence self-distillation term from the deepest system to the seed. This joint criterion is what lets intermediate depths work: the shared weights are pulled to behave correctly both as a shallow model and as a deep model, while the KL term anchors their output distributions.

What would settle it

Train a foldable model with $n_p=6$ and $N_f=12$ using Eq. (4), then evaluate at every intermediate depth 7 through 11 on the same evaluation sets; if any depth shows a statistically significant WER increase over an individually trained model of that depth by the paper's MAPSSWE test at $\alpha=0.05$, the arbitrary-depth claim is falsified. The paper reports only even depths and says odd depths follow similar trends without providing the numbers.

Watch

Extended reading notes

Core claim

The central discovery is that a shared-weight network can be made foldable to any logical depth between its physical depth $n_p$ and a maximum $N_f$ by the joint criterion $L = L_F^{N_f} + \alpha_p L_P^{n_p} + \alpha_{kl} D_{KL}(SG(p^{N_f}) \| p^{n_p})$, where the first term trains the deepest unfolded path, the second trains the seed, and the KL term makes the deepest system's output distribution guide the seed. The authors find empirically that these two endpoint losses plus the regularizer suffice to make all middle depths $F^{n_p}_{n_f}$ work, even though no loss term directly supervises them. They demonstrate this for supervised Conformer systems and for wav2vec2 and HuBERT speech foundation models, achieving 35% and 30% parameter reductions respectively with no statistically significant WER increase over the 12-layer baselines.

Load-bearing premise

The load-bearing premise is that minimizing only the seed loss and the deepest-unfolded loss, plus the KL term, is enough to make every intermediate depth work, even though no training signal directly supervises those intermediate depths.

Editorial extensions

If this is right

  • A single stored seed model replaces a family of separately trained and stored systems of different depths, since it can be executed at any depth from $n_p$ to $N_f$.
  • The method is model-agnostic: the same recipe works for supervised Conformer and for self-supervised wav2vec2 and HuBERT backbones with minimal changes.
  • Unfolding path choice is not critical: for a 6-layer model unfolded to depth 8, all 15 possible paths give similar WER, so a device can pick any path.
  • Training memory is bounded by the physical layers, not the unfolded depth, because only the shared weights and their activations are loaded.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The endpoint-only training recipe suggests that intermediate depths of a deep weight-shared network can be regularized implicitly by the extremes; if true, the same trick may transfer to other transformer-based modalities beyond speech.
  • As the depth range $N_f - n_p$ grows, intermediate depths may degrade unless $\alpha_{kl}$ is scaled or scheduled; adapting $\alpha_{kl}$ to the range is a natural testable extension.
  • The paper reports even depths for Conformer and wav2vec2 and states odd depths behave similarly without giving numbers; checking depths 7, 9, and 11 would sharpen the arbitrary-depth claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a 'foldable network' approach to ASR model compression. A compact seed model with n_p physical layers is trained jointly with an unfolded version of maximum logical depth N_f, using a KL self-distillation term between the deepest unfolded system and the seed. The resulting network is claimed to execute at arbitrary intermediate depths with accuracy comparable to individually trained models of those depths. Experiments on Conformer (Switchboard 300h) and wav2vec2/HuBERT (LibriSpeech 100h) report 35% and 30% parameter reductions with no statistically significant WER increase, plus favorable comparisons against prior SSL compression methods. The paper concludes that the approach requires 'minimal memory' during both training and inference.

Significance. If the memory-efficiency claim were fully substantiated, this would be a practically useful contribution: a single compact model that could be deployed at multiple depths, avoiding separate training and storage of many systems. The paper has clear strengths: the headline parameter-reduction figures are directly supported by the parameter counts in Tables 1 and 2, the ablation in Table 4 shows that both the joint training term and the KL regularization contribute, and the models are evaluated with standard recipes and a statistical significance test (MAPSSWE). However, the central 'memory-efficient training' claim is not verified by any measurement, and one of the comparison tables appears internally inconsistent. These issues must be addressed before the central claims can be considered established.

major comments (3)
  1. [Section 3.1, Eq. (4), Tables 1–2] The paper's central memory-efficiency claim is not established by the reported experiments. The text in Section 3.1 states that during training 'only the physical layers of the network are loaded into memory' and that unfolding scales model complexity 'without expanding the physical memory footprint.' This conflates parameter storage with total training memory. For a shared layer repeated N_f times, standard backpropagation must retain the forward activations of every repetition to compute gradients of the shared weights, unless gradient checkpointing or reversible layers are used. Neither is described anywhere in the manuscript. Consequently, peak activation memory during the joint training of Eq. (4) is expected to scale with the logical depth N_f, not with the seed's physical layer count. The 'loaded in memory' columns in Tables 1 and 2 report only parameter counts, and no peak-memory measurement is provided. Please report actual peak GPU memory (e.g., from caching allocator statistics) for training the foldable systems versus the corresponding all-physical baselines, and state how the backward pass through repeated layers is implemented. Without this, the Abstract's 'minimal memory' claim and the title's 'memory-efficient' claim are not supported.
  2. [Table 3 versus Table 2] Table 3 lists F312-W2V2 and F312-HuBERT with test-clean WERs of 18.15% and 17.92%, respectively, but the same configuration appears in Table 2 as T1 (3 physical layers unfolded to 12) with a test-clean WER of 8.15%. This factor-of-two discrepancy is not explained anywhere. Since Table 3 is the basis for the claim that the proposed method 'significantly outperforms' previous SSL compression methods, the inconsistency is load-bearing. Please correct the numbers or clarify what exactly is being compared, including whether the evaluation protocol, fine-tuning data, or model definition differs from Table 2.
  3. [Section 3.2, Eq. (4), Figs. 2–3] The claim that the seed model can be unfolded to 'arbitrary logical depths' is stronger than the evidence. The joint objective in Eq. (4) contains loss terms only for the seed depth n_p and the maximum depth N_f; intermediate depths are not supervised during training. The paper shows empirically in Figs. 2 and 3 and Tables 1 and 2 that several intermediate depths work well, which is valuable, but the term 'arbitrary' is not justified by the data: only even depths are reported in the tables, and the text's assertion that similar trends hold for odd depths is not accompanied by results. Please either provide odd-depth results and standard deviations for all supported depths, or qualify the claim to 'a range of logical depths' and state that intermediate-depth behavior is an empirical property of the trained foldable networks.
minor comments (6)
  1. [Section 3.2] The sentence 'with depths between N_f and n_p (i.e., N_f < n_f < np)' has the inequality reversed; it should read n_p < n_f < N_f.
  2. [Table 4 and Section 4.4] Table 4 is hard to read: several rows (Sys. 1, 3, 5) do not show the number of physical layers or parameter count consistently, and the caption/text says Sys. 0 and 1 contain 12 layers while the table lists differing structures. Please make the table self-contained.
  3. [Table 3] The comparison with prior work mixes training data amounts (960h vs 100h) and fine-tuning protocols; please state this limitation explicitly in the text or add a column for the fine-tuning setup.
  4. [Section 3.1 and Eq. (4)] The manuscript does not provide a pseudocode or implementation description of the joint training loop; adding one would clarify how the unfolded forward and backward passes are scheduled and would help readers implement Eq. (4).
  5. [Figs. 2 and 3] The standard deviations across different unfolding paths are plotted but their numerical values are not given; please provide them in a table or appendix, especially for the SSL models where the deviations are said to be 'too small to show.'
  6. [Section 1] The paper claims a 'first' small-to-large perspective; given the prior weight-sharing, once-for-all, and slimmable-network literature cited in [34–40], please sharpen the novelty statement and explicitly contrast the proposed method with those approaches.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims are empirical evaluations against independently trained baselines, not predictions reduced from fitted inputs or self-citations.

full rationale

Walking the claimed derivation chain: the method is defined by Eq. (2) (joint seed and deepest-unfolded losses) and Eq. (4) (adding a stop-gradient KL regularizer). The central results in Tables 1-3 and Figs. 2-3 are WER measurements of trained foldable systems compared with individually trained all-physical systems at equal executed depth. The intermediate-depth WERs are post-hoc evaluations, not training targets, validation criteria, or fitted outputs; therefore there is no fitting-to-prediction circularity. The KL term is an internal training regularizer applied with a stop-gradient, not a criterion used to select or report the claimed results. Hyperparameters alpha_p and alpha_kl are scalar balancing weights chosen from loss magnitudes, and no derivation depends on them beyond being fixed constants. The layer-sensitivity heuristic in Sec. 4.3 is a pruning selection rule; the reported WERs are measurements after training, not the selection signal. The paper's own text makes the intermediate-depth property an empirical observation, not a theorem: 'we found in our experiments that Eq. (2) containing only the two explicit loss terms is enough to also enable all middle systems' (Sec. 3.2), so the weakness flagged by the reader is a generalization/optimization risk, not circularity. Self-citations [8,28,39] appear in background or prior-compression contexts and are not load-bearing for the unfoldability claim. The most serious concern in the paper, the Section 3.1 claim that unfolding scales complexity 'without expanding the physical memory footprint,' is a potential validity gap because activation memory for repeated shared layers is not measured or discussed; however, an unverified memory claim is a correctness issue, not a derivation that reduces to its own inputs. On the circularity axis, the paper is self-contained against external baselines and the reported results are empirical, so no circular step meets the evidence bar.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on four unproven domain assumptions: that tied-weight repetition is expressive enough, that endpoint-only training transfers to intermediate depths, that leave-one-out layer sensitivity is a good drop criterion, and that one KL term helps all paths. The only free parameters are the two loss-weight scalars in Eq. (4); no new physical entities or fitted constants are introduced.

free parameters (2)
  • alpha_p (joint training scale) = 0.25 (Conformer), 0.7 (SSL)
    Scales the seed-model loss term in Eq. (4); manually picked from observed loss magnitudes, no search or sensitivity study reported.
  • alpha_kl (self-distillation scale) = 0.1 (Conformer), 0.005 (SSL)
    Scales the KL regularization term in Eq. (4); hand-set, not ablated across values.
assumptions (4)
  • domain assumption Shared-weight repetition preserves representational capacity.
    The central mechanism assumes that applying the same N physical layers k times produces a function close to k independent layers; this is an empirical premise, tested only indirectly by WER.
  • domain assumption Endpoint training transfers to intermediate depths.
    Eq. (4) contains only n_p and N_f losses; no loss for n_f between them. The claim that all intermediate depths work is validated observationally rather than by construction.
  • domain assumption Layer sensitivity by leave-one-out WER identifies safe layers to drop.
    For SSL models, the choice of which physical layers remain is set by measuring WER after dropping each layer; this assumes single-layer importance is additive and stable under foldable training.
  • domain assumption KL from deepest to seed is a useful regularizer for all unfolded paths.
    The KL term only matches the deepest teacher to the smallest seed; its benefit for intermediate depths is assumed and not directly optimized.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unfolding A Few Structures for The Many: Memory-Efficient Compression of Conformer and Speech Foundation Models." pith.science (2026). https://pith.science/paper/6OIJLWVH

@misc{pith2026250521237,
  author       = {Pith},
  title        = {Pith review of: Unfolding A Few Structures for The Many: Memory-Efficient Compression of Conformer and Speech Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6OIJLWVH}},
  note         = {Machine review of arXiv:2505.21237}
}
read the original abstract

This paper presents a novel memory-efficient model compression approach for Conformer ASR and speech foundation systems. Our approach features a unique "small-to-large" design. A compact "seed" model containing a few Conformer or Transformer blocks is trained and unfolded many times to emulate the performance of larger uncompressed models with different logical depths. The seed model and many unfolded paths are jointly trained within a single unfolding cycle. The KL-divergence between the largest unfolded and smallest seed models is used in a self-distillation process to minimize their performance disparity. Experimental results show that our foldable model produces ASR performance comparable to individually constructed Conformer and wav2vec2/HuBERT speech foundation models under various depth configurations, while requiring only minimal memory and storage. Conformer and wav2vec2 models with a reduction of 35% and 30% parameters are obtained without loss of performance, respectively.

Figures

Figures reproduced from arXiv: 2505.21237 by the authors.

Figure 1
Figure 1. Diagram of a foldable network. Systems (a), (b), and (c) share the same N physical layers, and they can (un)fold to each other by changing the number of repeating times of the physical layers, without expanding parameters and memory. 3.2. Foldable Network Previous studies using weight-sharing blocks [43,44], which are based on a manually defined sharing pattern with a fixed model depth. Our work aims to learn a comp… view at source ↗
Figure 2
Figure 2. Averaged WER of Conformer systems versus executed encoder depths (# physical layers + unfolded depths). The fold￾able and all-physical models are the same ones in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. WER (test-clean) of wav2vec2 versus executed en￾coder depths. The foldable and compact models are the same ones in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 52 canonical work pages

  1. [1]

    Unfolding A Few Structures for The Many: Memory-Efficient Compression of Conformer and Speech Foundation Models

    Introduction The field of automatic speech recognition (ASR) systems is witnessing a significant surge in model size, driven in part by the recent proliferation of self-supervised learning (SSL) mod- els such as wav2vec2.0 [1], HuBERT [2], data2vec [3], and WavLM [4]. This increase creates an urgent demand for ef- fective and efficient model compression t...

    work page Pith review arXiv 2025

  2. [2]

    Speech Models For supervised learning cases, we consider the convolution- augmented Transformer (Conformer [41]), an end-to-end (E2E) ASR model. A Conformer encoder comprises multiple stack- ing blocks 1, where each block is composed of the following modules in sequence: a feed-forward module (FFN), a self- attention module (MHSA), a convolution module (C...

    work page

  3. [3]

    layers” and “blocks

    Methods 3.1. Network Structure Unfolding To benefit from both the memory efficiency of small models and the better modeling capacity of deeper models, we present a network structure unfolding approach that allows a small model to reuse its network structures to enhance the model capability and maintain a low memory consumption. Suppose we have a small mod...

    work page

  4. [4]

    Experimental Setup Models and Data.For supervised learning, we take Conformer AED system configured with the ESPnet [45] recipe2

    Experiments 4.1. Experimental Setup Models and Data.For supervised learning, we take Conformer AED system configured with the ESPnet [45] recipe2. We train Conformer with 300-hr Switchboard corpus [46] and evalu- ate on NIST Hub5’00, RT02, and RT03 evaluation sets. For SSL cases, the wav2vec2-base-100h/HuBERT-base are down- loaded from Huggingface 3. We f...

    work page

  5. [5]

    Conclusion This paper introduces a novel architecture compression ap- proach for ASR systems to effectively address essential chal- lenges related to memory efficiency, model performance, and adaptability to various edge device requirements. It enables the development of a single compact model that can unfold to vari- ous logical depths, achieving perform...

    work page

  6. [6]

    14200220, 14200021, 14200324, Innovation Technology Fund grant No

    Acknowledgements This research is supported by Hong Kong RGC GRF grant No. 14200220, 14200021, 14200324, Innovation Technology Fund grant No. ITS/218/21, and Youth Innovation Promotion Asso- ciation CAS Grant (2023119)

    work page

  7. [7]

    2-bit conformer quantization for automatic speech recognition,

    O. Rybakov, P. Meadowlarket al., “2-bit conformer quantization for automatic speech recognition,” inInterspeech, 2023

    work page 2023

  8. [8]

    2 and 3), and the 8-layer system that trained at the unfolded depth of 12 that can not be folded back (i.e., Sys

    This is also the case for the 8-layer system that trained at the depth of 8 but can not be unfolded (i.e., Sys. 2 and 3), and the 8-layer system that trained at the unfolded depth of 12 that can not be folded back (i.e., Sys. 4 and 5). In contrast, by using our joint training framework, the 8-layer foldable system (i.e., Sys. E0 and E1) can be executed at...

    work page

Show all 55 references
  1. [9]

    wav2vec 2.0: A framework for self- supervised learning of speech representations,

    A. Baevski, Y . Zhouet al., “wav2vec 2.0: A framework for self- supervised learning of speech representations,” inNeurIPS, 2020

    2020

  2. [10]

    HuBERT: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolteet al., “HuBERT: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM T-ASLP, vol. 29, pp. 3451–3460, 2021

    2021

  3. [11]

    Data2vec: A general framework for self-supervised learning in speech, vision and language,

    A. Baevski, W.-N. Hsuet al., “Data2vec: A general framework for self-supervised learning in speech, vision and language,” in ICML, 2022

    2022

  4. [12]

    WavLM: Large-scale self-supervised pre-training for full stack speech processing,

    S. Chen, C. Wanget al., “WavLM: Large-scale self-supervised pre-training for full stack speech processing,”IEEE J-STSP, vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  5. [13]

    4-bit conformer with native quan- tization aware training for speech recognition,

    S. Ding, P. Meadowlarket al., “4-bit conformer with native quan- tization aware training for speech recognition,” inInterspeech, 2022

    2022

  6. [14]

    Integer-only zero-shot quantization for efficient speech recognition,

    S. Kim, A. Gholamiet al., “Integer-only zero-shot quantization for efficient speech recognition,” inICASSP, 2022

    2022

  7. [15]

    Effective and efficient mixed precision quan- tization of speech foundation models,

    H. Xu, Z. Liet al., “Effective and efficient mixed precision quan- tization of speech foundation models,” inICASSP, 2025

    2025

  8. [16]

    Multi-stage progressive com- pression of conformer transducer for on-device speech recogni- tion,

    J. Rathod, N. Dawalatabadet al., “Multi-stage progressive com- pression of conformer transducer for on-device speech recogni- tion,” inInterspeech, 2022

    2022

  9. [17]

    DistilHuBERT: Speech represen- tation learning by layer-wise distillation of hidden-unit bert,

    H.-J. Chang, S.-w. Yanget al., “DistilHuBERT: Speech represen- tation learning by layer-wise distillation of hidden-unit bert,” in ICASSP, 2022

    2022

  10. [18]

    FitHuBERT: Going thinner and deeper for knowledge distillation of speech self-supervised learning,

    Y . Lee, K. Janget al., “FitHuBERT: Going thinner and deeper for knowledge distillation of speech self-supervised learning,” in Interspeech, 2022

    2022

  11. [19]

    Deep versus wide: An analysis of student architectures for task-agnostic knowledge distillation of self-supervised speech models,

    T. Ashihara, T. Moriyaet al., “Deep versus wide: An analysis of student architectures for task-agnostic knowledge distillation of self-supervised speech models,” inInterspeech, 2022

    2022

  12. [20]

    DistillW2V2: A small and streaming wav2vec 2.0 based asr model,

    Y . Fu, Y . Kanget al., “DistillW2V2: A small and streaming wav2vec 2.0 based asr model,”arXiv preprint arXiv:2303.09278, 2023

    2023 arXiv

  13. [21]

    Distilling HuBERT with LSTMs via decoupled knowledge distillation,

    D. de Oliveira and T. Gerkmann, “Distilling HuBERT with LSTMs via decoupled knowledge distillation,” inICASSP, 2024

    2024

  14. [22]

    Conformer-based on-device streaming speech recognition with KD compression and two-pass architec- ture,

    J. Park, S. Jinet al., “Conformer-based on-device streaming speech recognition with KD compression and two-pass architec- ture,” inIEEE SLT, 2023

    2023

  15. [23]

    Dynamic sparsity neural networks for au- tomatic speech recognition,

    Z. Wu, D. Zhaoet al., “Dynamic sparsity neural networks for au- tomatic speech recognition,” inICASSP, 2021

    2021

  16. [24]

    PARP: Prune, adjust and re-prune for self-supervised speech recognition,

    C.-I. J. Lai, Y . Zhanget al., “PARP: Prune, adjust and re-prune for self-supervised speech recognition,” inNeurIPS, 2021

    2021

  17. [25]

    Layer pruning on demand with intermediate ctc,

    J. Lee, J. Kanget al., “Layer pruning on demand with intermediate ctc,” inInterspeech, 2021

    2021

  18. [26]

    DPHuBERT: Joint distillation and prun- ing of self-supervised speech models,

    Y . Peng, Y . Sudoet al., “DPHuBERT: Joint distillation and prun- ing of self-supervised speech models,” inInterspeech, 2023

    2023

  19. [27]

    Accurate and structured pruning for efficient automatic speech recognition,

    H. Jiang, L. L. Zhanget al., “Accurate and structured pruning for efficient automatic speech recognition,” inInterspeech, 2023

    2023

  20. [28]

    PADA: Pruning assisted domain adaptation for self-supervised speech representations,

    V . S. Lodagala, S. Ghoshet al., “PADA: Pruning assisted domain adaptation for self-supervised speech representations,” inIEEE SLT, 2023

    2023

  21. [29]

    Structured pruning of self-supervised pre-trained models for speech recognition and understanding,

    Y . Peng, K. Kimet al., “Structured pruning of self-supervised pre-trained models for speech recognition and understanding,” in ICASSP, 2023

    2023

  22. [30]

    Task-agnostic structured pruning of speech representation models,

    H. Wang, S. Wanget al., “Task-agnostic structured pruning of speech representation models,” inInterspeech, 2023

    2023

  23. [31]

    SparseW A V: Fast and accurate one-shot un- structured pruning for large speech foundation models,

    T. Gu, B. Liuet al., “SparseW A V: Fast and accurate one-shot un- structured pruning for large speech foundation models,” inInter- speech, 2024

    2024

  24. [32]

    Semi-orthogonal low-rank matrix fac- torization for deep neural networks,

    D. Povey, G. Chenget al., “Semi-orthogonal low-rank matrix fac- torization for deep neural networks,” inInterspeech, 2018

    2018

  25. [33]

    Neural architecture search for LF-MMI trained time delay neural networks,

    S. Hu, X. Xieet al., “Neural architecture search for LF-MMI trained time delay neural networks,”IEEE/ACM T-ASLP, vol. 30, pp. 1093–1107, 2022

    2022

  26. [34]

    Efficient conformer-based speech recognition with linear attention,

    S. Li, M. Xuet al., “Efficient conformer-based speech recognition with linear attention,” inAPSIPA ASC, 2021

    2021

  27. [35]

    Lossless 4-bit quantization of architecture compressed conformer asr systems on the 300-hr switchboard cor- pus,

    Z. Li, T. Wanget al., “Lossless 4-bit quantization of architecture compressed conformer asr systems on the 300-hr switchboard cor- pus,” inInterspeech, 2023

    2023

  28. [36]

    ALBERT: A lite bert for self-supervised learning of language representations,

    Z. Lan, M. Chenet al., “ALBERT: A lite bert for self-supervised learning of language representations,” inICLR, 2020

    2020

  29. [37]

    Extremely low footprint end-to-end asr system for smart device,

    Z. Gao, Y . Yaoet al., “Extremely low footprint end-to-end asr system for smart device,” inInterspeech, 2021

    2021

  30. [38]

    Weight-sharing supernet for searching specialized acoustic event classification networks across device constraints,

    G.-T. Lin, Q. Tanget al., “Weight-sharing supernet for searching specialized acoustic event classification networks across device constraints,” inICASSP, 2023

    2023

  31. [39]

    Sharing low rank conformer weights for tiny always-on ambient speech recognition models,

    S. M. Hernandez, D. Zhaoet al., “Sharing low rank conformer weights for tiny always-on ambient speech recognition models,” inICASSP, 2023

    2023

  32. [40]

    ResidualTransformer: Residual low-rank learning with weight-sharing for transformer layers,

    Y . Wang and J. Li, “ResidualTransformer: Residual low-rank learning with weight-sharing for transformer layers,” inICASSP, 2024

    2024

  33. [41]

    Once for all: Train one network and spe- cialize it for efficient deployment,

    H. Cai, C. Ganet al., “Once for all: Train one network and spe- cialize it for efficient deployment,” inICLR, 2020

    2020

  34. [42]

    Universally slimmable networks and im- proved training techniques,

    J. Yu and T. S. Huang, “Universally slimmable networks and im- proved training techniques,” inICCV, 2019

    2019

  35. [43]

    Self-distillation: Towards efficient and compact neural networks,

    L. Zhang, C. Baoet al., “Self-distillation: Towards efficient and compact neural networks,”IEEE T-PAMI, vol. 44, no. 8, pp. 4388– 4403, 2021

    2021

  36. [44]

    Collaborative training of acoustic en- coders for speech recognition,

    V . Nagaraja, Y . Shiet al., “Collaborative training of acoustic en- coders for speech recognition,” inInterspeech, 2021

    2021

  37. [45]

    LightHuBERT: Lightweight and con- figurable speech representation learning with once-for-all hidden- unit bert,

    R. Wang, Q. Baiet al., “LightHuBERT: Lightweight and con- figurable speech representation learning with once-for-all hidden- unit bert,” inInterspeech, 2022

    2022

  38. [46]

    One-pass multiple conformer and foundation speech systems compression and quantization using an all-in-one neural model,

    Z. Li, H. Xuet al., “One-pass multiple conformer and foundation speech systems compression and quantization using an all-in-one neural model,” inInterspeech, 2024

    2024

  39. [47]

    Small-footprint slimmable networks for keyword spotting,

    Z. Akhtar, M. O. Khursheedet al., “Small-footprint slimmable networks for keyword spotting,” inICASSP, 2023

    2023

  40. [48]

    Conformer: Convolution-augmented transformer for speech recognition,

    A. Gulati, J. Qinet al., “Conformer: Convolution-augmented transformer for speech recognition,” inInterspeech, 2020

    2020

  41. [49]

    Hybrid CTC/attention architecture for end-to-end speech recognition,

    S. Watanabe, T. Horiet al., “Hybrid CTC/attention architecture for end-to-end speech recognition,”IEEE Journal of Selected Top- ics in Signal Processing, vol. 11, no. 8, pp. 1240–1253, 2017

    2017

  42. [50]

    Improving transformer-based speech recognition systems with compressed structure and speech at- tributes augmentation

    S. Li, R. Dabreet al., “Improving transformer-based speech recognition systems with compressed structure and speech at- tributes augmentation.” inInterspeech, 2019

    2019

  43. [51]

    Non-autoregressive asr with self-conditioned folded encoders,

    T. Komatsu, “Non-autoregressive asr with self-conditioned folded encoders,” inICASSP, 2022

    2022

  44. [52]

    Espnet: End-to-end speech process- ing toolkit,

    S. Watanabe, T. Horiet al., “Espnet: End-to-end speech process- ing toolkit,”arXiv preprint arXiv:1804.00015, 2018

    2018 arXiv

  45. [53]

    SWITCHBOARD: Tele- phone speech corpus for research and development,

    J. J. Godfrey, E. C. Hollimanet al., “SWITCHBOARD: Tele- phone speech corpus for research and development,” inICASSP, 1992

    1992

  46. [54]

    LibriSpeech: an asr corpus based on public domain audio books,

    V . Panayotov, G. Chenet al., “LibriSpeech: an asr corpus based on public domain audio books,” inICASSP, 2015

    2015

  47. [55]

    Some statistical issues in the comparison of speech recognition algorithms,

    L. Gillick and S. J. Cox, “Some statistical issues in the comparison of speech recognition algorithms,” inICASSP, 1989

    1989

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.