Pith. sign in

REVIEW 5 major objections 6 minor 4 cited by

How to Learn a New Language? An Efficient Solution for Self-Supervised Learning Models Unseen Languages Adaption in Low-Resource Scenario

T0 review · 5 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adding an intermediate warm-up on linguistically similar source languages lets a frozen speech model adapt to unseen low-resource languages with only 1–5% of parameters updated, improving CER/PER by up to 28% relative to standard…

desk verdict A practical warm-up recipe for adapter-based ASR adaptation that mostly works, but the headline 28% gain is aggregate and the source-selection rule is only validated on the Seen Set. read the letter →

arxiv 2411.18217 v2 pith:4IH6V5JP submitted 2024-11-27 cs.SD cs.CLeess.AS

Shih-Heng Wang , Zih-Ching Chen , Jiatong Shi , Ming-To Chuang , Guan-Ting Lin , Kuan-Po Huang , David Harwath , Shang-Wen Li
show 1 more author
Hung-yi Lee
This is my paper · ORCID
classification cs.SDcs.CLeess.AS
keywords low-resourceASRself-supervisedspeechadapterparameter-efficientfine-tuningtransferlearningmultilinguallanguageadaptationmeta-learning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a frozen self-supervised speech model can adapt to an unseen low-resource language far better if, before fine-tuning on the target language, one first warms up the adapter and downstream head on a handful of high-resource languages that are linguistically close to the target. The warm-up, called Intermediate Adaptation (IA), can use either multitask learning or first-order MAML, and it changes only 1–5% of the model's parameters. On the ML-SUPERB benchmark, the proposed pipeline beats conventional efficient fine-tuning with up to a 28% relative reduction in Character/Phoneme Error Rate, and it matches or exceeds full fine-tuning while updating far fewer parameters. A sympathetic reader would care because it offers a cheap recipe for expanding speech recognition to low-resource languages without retraining the large pre-trained model.

What carries the argument

The load-bearing pieces are the similarity score $\mathrm{Sim}(l,T)=\sum_{j} D(\mathrm{LCA}(l,t_j))$, which ranks candidate source languages by the depth $D$ of the lowest common ancestor node with each target language in a linguistic tree, and the Intermediate Adaptation objective $\hat{\theta}_{a+d} = \mathrm{IA}(\theta,S)$, realized either by multitask learning over source languages or by first-order MAML with an inner support-set update and outer query-set update. These produce a warmed-up initialization of the bottleneck adapters (size 32) plus the CTC transformer downstream head, which are then fine-tuned per target language while the SSL backbone stays frozen.

What would settle it

On the Unseen Set, compare IA-MTL with source languages selected by the proposed LCA-depth ranking against sources selected randomly (or by measured representation similarity from the frozen SSL model) over many seeds; if the LCA-selected sources do not consistently beat random or embedding-based selection, the source-selection criterion is not carrying the claimed benefit.

Watch

Extended reading notes

Core claim

The central claim is that the domain mismatch between a pre-trained speech SSL model and an unseen low-resource target language can be bridged by an intermediate adaptation step that tunes only the adapter and downstream model on selected source languages before the final target fine-tuning. The paper proposes to select source languages by the depth of their lowest common ancestor with the target languages in a linguistic tree, and to instantiate the warm-up either as multitask learning over all sources or as MAML over sampled source-language batches. After warm-up, the adapter and downstream head are fine-tuned per target language with the SSL backbone frozen. The authors report that this initialization consistently outperforms training from random initialization (PEFT), from frozen SSL features alone (Freeze FT), and from a single-stage joint source-target MTL, and that it can match or surpass full fine-tuning while using under 6% of the tunable parameters.

Load-bearing premise

The recipe only pays off if the depth of the lowest common ancestor in a linguistic tree is a good proxy for how much acoustic and phonetic knowledge transfers from a source language to the target language; the paper validates this ranking only on the Seen Set, not on the Unseen Set.

Editorial extensions

If this is right

  • Adding IA before PEFT yields up to a 28% relative CER/PER improvement over direct PEFT on unseen languages, and matches or beats full fine-tuning with far fewer updated parameters.
  • Linguistically-similar source selection via LCA depth beats random source selection in both the 10-minute and 1-hour low-resource settings.
  • The benefit holds across different SSL backbones, including a monolingual English HuBERT, a trilingual mHuBERT, and a 128-language XLSR-128.
  • Increasing the number of source languages helps up to M=20 and then plateaus, implying that the closest languages do most of the work.
  • After IA, the CTC head must be reinitialized because the character or phoneme sets of source and target languages differ.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If LCA-based selection truly tracks acoustic transferability, then composing it with embedding-space similarity from the frozen SSL model could yield better source rankings; this is a testable extension the paper does not run.
  • Because the SSL backbone is frozen, the warmed-up adapter could be reused across many target languages, suggesting the recipe scales to hundreds of languages without per-language backbone training.
  • A limitation the authors state is that the target set must be known in advance; a universal source set selected by average similarity to all remaining languages would make the method deployable without prior target knowledge.
  • The warm-up changes only the initialization point of fine-tuning, so it may also improve performance when the target has a few hours of data rather than just 10 minutes or 1 hour.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes an Intermediate Adaptation (IA) stage inserted before parameter-efficient fine-tuning (PEFT) for low-resource ASR with frozen SSL models. Source languages are selected by summing the depth of the lowest common ancestor in a linguistic tree over the target-language set (Eq. (2)); the adapter and downstream CTC model are then warmed up on those source languages via multitask learning (MTL) or first-order MAML, and finally adapted to each target language with PEFT. The experiments cover ML-SUPERB Seen and Unseen language sets with HuBERT-base, mHuBERT-base, and XLSR-128, and the paper reports that the IA variants outperform PEFT, Freeze-FT, and S&T-MTL baselines, with up to 28% relative CER/PER improvement while updating only 1-5% of model parameters.

Significance. If the reported numbers hold, the contribution is practically useful: it gives a low-cost recipe for adapting frozen SSL ASR models to low-resource languages and connects linguistic typology to adapter initialization. The core comparison is well controlled in design: the frozen backbone, adapter architecture, downstream model, and data budget are matched across IA and PEFT, and the S&T-MTL baseline controls for the use of additional source-language data. The source-language selection hypothesis based on linguistic-tree closeness is interesting and falsifiable. However, the manuscript as submitted does not yet provide sufficient evidence for the central claim: the Unseen-Set results are only reported in aggregate, the selection rule is not validated on the Unseen Set, and the numerical table supporting the headline numbers is missing from the submitted text.

major comments (5)
  1. [§4.1, Table 3] The central quantitative claim cannot be checked from the submitted manuscript: Table 3 contains only a caption and no numerical CER/PER entries, even though the text cites relative improvements of up to 28% and 20% over the PEFT baseline. Please include the complete table with all model/condition rows and absolute CER/PER values, and reconcile the cited percentages with the individual cells.
  2. [§2.1, Eq. (2); §3.2, Table 1(II)] The source-selection rule is validated only on the Seen Set (Table 4a), but the main 'unseen language' claim rests on the Unseen Set. For that set, Eq. (2) selects ten Bantu source languages (xho, ven, ssw, sot, sna, nso, nbl, nya, lug, kin) even though only four of the twenty targets (umb, zul, tsn, tso) are Bantu. This is the expected failure mode of summing LCA depths over a heterogeneous target set: a dense branch can dominate the score. Because no per-language CER/PER is reported for the Unseen Set, the aggregate improvement over PEFT may be carried by those four Bantu targets, and the abstract's 'adapting to unseen languages' would then be an overstatement. Please report per-target results for the Unseen Set and validate the selection criterion on that set, for example by comparing Eq. (2) against per-target selection or against random selection within the Unseen-Set condition, for both IA-MTL and IA-MAML.
  3. [§4.3, Table 4a footnote] The validation of the source-selection method is internally inconsistent. The footnote to Table 4a lists the M=10 source languages as {nbl, ssw, ven, mal, ben, mri, sot, nep, sin, jav}, but Table 1(I) defines the Seen-Set M=10 source set as {ltz, nor, spa, por, oci, nld, glg, cat, ast, afr}. It is therefore unclear whether Table 4a compares the proposed selection on the Seen Set or on some other language pool. Please correct the footnote or the table, and state explicitly the pool from which both random and proposed selections are drawn.
  4. [§4.1, Tables 3–4] All reported CER/PER values appear to come from a single training run, and the experimental setup does not state the number of seeds. Low-resource ASR comparisons are noisy, and the headline gains (e.g., 28% relative over PEFT) could be within run-to-run variance. Please report mean and standard deviation over at least three random seeds, or provide per-language paired results, and state the number of runs in the experimental setup.
  5. [§2.2, §4.3] Table 4a validates the source-language selection only under IA-MTL, while the other headline variant IA-MAML uses the same source set. Since MAML's bi-level optimization can behave differently from MTL with respect to source-language relatedness, the claim that Eq. (2) is the right selection rule for the proposed pipeline is only partially supported. Please validate the selection rule for both adaptation algorithms, or explicitly restrict the source-selection claim to the MTL variant.
minor comments (6)
  1. [§2.1, Eq. (2)] The depth function D and the handling of languages missing from the linguistic tree are not specified; please state the convention for the root depth and the behavior for out-of-tree languages.
  2. [§3.2, Table 1] The Unseen Set is described as containing '20 endangered languages', but the list includes epo, tok, kea, sun, and others that are not commonly classified as endangered; please check the label or the language list.
  3. [§3.3, Algorithm 1] Since the paper uses first-order MAML, Algorithm 1 should state explicitly which gradient terms are treated as constant in line 8; otherwise readers may assume full second-order MAML.
  4. [§6, References] Reference [34] is cited as 'NACCL' and should be 'NAACL'; reference [25] lists an author as 'C. Zih-Ching' and should be checked for consistency with the author list.
  5. [§3.4, §4.1] The paper describes the solution as efficient but only reports parameter counts; please also report wall-clock time or training FLOPs for IA versus PEFT, since IA trains on M source languages and the computational overhead is part of the efficiency claim.
  6. [§2.1] The linguistic-tree resource is not identified: reference [32] is a grapheme-to-phoneme paper, not the tree itself. Please cite the actual tree source (e.g., a language database) and explain how the tree topology is obtained.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the IA claim is an empirical result on the external ML-SUPERB benchmark, not a derivation from its own inputs.

full rationale

The paper's central claim is that an Intermediate Adaptation (IA) warm-up on selected source languages improves adapter-based PEFT for unseen low-resource ASR targets. The evaluation is an external benchmark comparison: error rates are measured on ML-SUPERB against independent baselines (Full FT, Freeze FT, PEFT, S&T-MTL), with the SSL backbone frozen. The source-language selection rule in Eq. (2) is a heuristic score based on depth of the lowest common ancestor in an external linguistic tree; it has no parameters fitted to the target CER/PER, and the paper validates it against random selection on the Seen Set (Table 4a). Choosing M=10 is a hyperparameter selected on the Seen Set, but the reported improvements are measured on held-out target test sets, so they are not forced by construction. The only notable self-citation is [1], used to support the premise that linguistically close languages share acoustic traits; this premise is not the derivation of the result, and the experimental gains provide independent evidence. The paper's disclosed limitation that the target language set must be known beforehand does not make the claim circular, because 'unseen' refers to the SSL model's pre-training data and no target speech data is used during IA. No equation in the paper reduces to its own inputs by definition.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

No fundamentally new mechanisms are introduced; the free parameters are standard hyperparameters chosen by hand, and the axioms are domain assumptions about language similarity, the use of MAML/MTL algorithms, and the benchmark representativeness. Nothing counts as an invented entity in the sense of a new particle, force, or conserved quantity.

free parameters (3)
  • Number of source languages M = M = 10 (default), M = 20 (best on Seen Set)
    Chosen via a sweep on the Seen Set (Table 4b); M=20 gives the lowest CER/PER but M=10 is the default due to stated computational constraints.
  • MAML outer and inner learning rates = alpha = 0.001, beta = 0.0001
    Hand-picked FOMAML hyperparameters in Sec. 3.3; no sensitivity analysis or tuning curves are provided.
  • MTL learning rate = 1e-4
    Hand-picked learning rate for the MTL warm-up in Sec. 3.3, with no reported sensitivity analysis.
assumptions (3)
  • domain assumption The linguistic tree from [32] reflects usable acoustic and phonetic similarity between languages.
    Used in Sec. 2.1 to rank source languages via LCA depth; the assumption that 'linguistically close' languages share acoustic traits is stated with citation [1].
  • standard math MAML and MTL optimization, as standardly defined in [33] and common practice, converge to an initialization useful for target-language fine-tuning.
    Sec. 2.2 assumes the optimization objectives in Eqs. (3)-(4) and Algorithm 1 produce a beneficial warm start; no convergence guarantee or analysis is provided.
  • domain assumption ML-SUPERB's 10-minute and 1-hour per-language subsets are representative low-resource training conditions.
    All conclusions in Sec. 4 depend on this benchmark setup from [15] as a proxy for real low-resource ASR.

how reviews work

0 comments
Cite this review

Pith. "Pith review of How to Learn a New Language? An Efficient Solution for Self-Supervised Learning Models Unseen Languages Adaption in Low-Resource Scenario." pith.science (2026). https://pith.science/paper/4IH6V5JP

@misc{pith2026241118217,
  author       = {Pith},
  title        = {Pith review of: How to Learn a New Language? An Efficient Solution for Self-Supervised Learning Models Unseen Languages Adaption in Low-Resource Scenario},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4IH6V5JP}},
  note         = {Machine review of arXiv:2411.18217}
}
read the original abstract

The utilization of speech Self-Supervised Learning (SSL) models achieves impressive performance on Automatic Speech Recognition (ASR). However, in low-resource language ASR, they encounter the domain mismatch problem between pre-trained and low-resource languages. Typical solutions like fine-tuning the SSL model suffer from high computation costs while using frozen SSL models as feature extractors comes with poor performance. To handle these issues, we extend a conventional efficient fine-tuning scheme based on the adapter. We add an extra intermediate adaptation to warm up the adapter and downstream model initialization. Remarkably, we update only 1-5% of the total model parameters to achieve the adaptation. Experimental results on the ML-SUPERB dataset show that our solution outperforms conventional efficient fine-tuning. It achieves up to a 28% relative improvement in the Character/Phoneme error rate when adapting to unseen languages.

Figures

Figures reproduced from arXiv: 2411.18217 by the authors.

Figure 1
Figure 1. Pipeline of our solution. Before fine-tuning the adapter and downstream model (omitted in the figure) to each target language, we warm up them with Intermediate Adapta￾tion. cost. However, they could still encounter difficulties in trans￾ferring to an unseen language in low-resource scenarios due to domain mismatch. Some existing works aim to solve the domain mismatch problem. For example, [28] introduces continual … view at source ↗
Figure 2
Figure 2. Our source language selection process with the lin￾guistic tree. Based on the topology of the example linguis￾tic tree, we pick “Luxembourgish” and “Ndebele” instead of “Manx Gaelic” as source languages for IA because they are linguistically closer to our target languages “English” and “Swedish”. tree. For example, in [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. REDDIT: Correcting Model-Generated Timestamp Drift in ASR without Forgetting via Replay-Based Distribution Editing

    cs.CL 2026-07 conditional novelty 7.0 of 10

    REDDIT corrects non-speech-induced timestamp drift in autoregressive ASR by editing timestamp targets under cached replay context while anchoring non-timestamp behavior to the frozen base distribution.

  2. Dissecting Sensitivity to Training Language in Self-Supervised Speech Learning Using Neural Audio Codec Tokens

    cs.SD 2026-07 conditional novelty 5.0 of 10

    Downstream ASR/SER performance is largely insensitive to the neural-audio-codec training language but strongly depends on the SSL pre-training language in codec-based self-supervised speech learning.

  3. Fake-Mamba: Real-Time Speech Deepfake Detection Using Bidirectional Mamba as Self-Attention's Alternative

    eess.AS 2025-08 unverdicted novelty 5.0 of 10

    Fake-Mamba reports EERs of 0.97%, 1.74%, and 5.85% on three speech deepfake benchmarks, but the provided full text is an unrelated paper, so the claims cannot be verified.

  4. Exploring Dynamic Parameters for Vietnamese Gender-Independent ASR

    eess.AS 2025-07 conditional novelty 4.0 of 10

    Polar-ratio features derived from spectral subband centroids, plus a pseudo-F0 feature, lower word error rates and improve gender independence in Vietnamese ASR over a 6-MFCC baseline.

Reference graph

Works this paper leans on

39 extracted references · 37 canonical work pages · cited by 4 Pith papers

  1. [27]

    Prefix-tuning: Optimizing continuous prompts for generation,

    X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” in ACL, 2021, pp. 4582–4597

    work page 2021

  2. [1]

    Introduction Self-Supervised Learning models (SSL models) [1–6] pre- trained with speech-only data have achieved significant im- provements for Automatic Speech Recognition (ASR) in main- stream languages. [7–13]. However, employing SSL models on low-resource language ASR may encounter the problem of domain mismatch between pre-trained and low-resource la...

    arXiv 2025

  3. [2]

    Luxembourgish

    Methodology We focus on effectively adapting SSL models to each unseen target language in low-resource scenarios. As illustrated in Fig- ure 1, before fine-tuning on each target language, we add an extra Intermediate Adaptation (IA) step. IA warms up the adapter and downstream model with source languages to facil- itate adaptation to each unseen target la...

    work page

  4. [3]

    Dataset We evaluate our solution using ML-SUPERB [15], a benchmark for multilingual ASR with speech SSL models

    Experimental Setups 3.1. Dataset We evaluate our solution using ML-SUPERB [15], a benchmark for multilingual ASR with speech SSL models. ML-SUPERB is supported by 143 languages. For each language, ML-SUPERB provides 10-minute and 1-hour settings. The duration means the training data size employed in fine-tuning for each language. For evaluation metrics, w...

    work page

  5. [4]

    Main Result Table 3 presents the results of our IA variants and baselines from the Seen Set and the Unseen Set

    Result & Analysis 4.1. Main Result Table 3 presents the results of our IA variants and baselines from the Seen Set and the Unseen Set. Remarkably, two IA variants (IA-MAML, IA-MTL) consistently outperform other baselines (Freeze-FT, PEFT, S&T -MTL) on both sets, while IA-MTL slightly outperforms IA-MAML. In the 10-minute and 1-hour setting, IA variants ac...

    work page

  6. [5]

    Our so- lution adds an extra Intermediate Adaptation (IA) to warm up the adapter and downstream model initialization

    Conclusion & Limitation In this work, we propose an efficient solution for adapting SSL models to unseen language in low-resource scenarios. Our so- lution adds an extra Intermediate Adaptation (IA) to warm up the adapter and downstream model initialization. With this en- hanced initialization, the model can adapt to unseen target lan- guages more easily....

    work page

  7. [6]

    Cross-Lingual Transfer for Speech Processing Using Acoustic Language Similarity,

    P. Wu, J. Shi, Y . Zhong et al. , “Cross-Lingual Transfer for Speech Processing Using Acoustic Language Similarity,” ASRU, pp. 1050–1057, 2021

    work page 2021

  8. [7]

    Massively Multilingual Adversarial Speech Recognition,

    O. Adams et al. , “Massively Multilingual Adversarial Speech Recognition,” in Proceedings of NAACL-HLT, 2019, pp. 96–108

    work page 2019

Show all 39 references
  1. [8]

    Large-Scale End-to-End Multilingual Speech Recognition and Language Identification with Multi-Task Learning,

    W. Hou, Y . Dong, B. Zhuang et al. , “Large-Scale End-to-End Multilingual Speech Recognition and Language Identification with Multi-Task Learning,” inInterspeech, 2020

    2020

  2. [9]

    Low Resource ASR: The Surprising Effec- tiveness of High Resource Transliteration,

    S. Khare and othersj, “Low Resource ASR: The Surprising Effec- tiveness of High Resource Transliteration,” inInterspeech, 2021

    2021

  3. [10]

    A Survey of Multilingual Models for Automatic Speech Recognition,

    H. Yadav and S. Sitaram, “A Survey of Multilingual Models for Automatic Speech Recognition,” in LREC, 2022, pp. 5071–5079

    2022

  4. [11]

    Cross-lingual Automatic Speech Recognition Ex- ploiting Articulatory Features,

    Q. Zhan et al., “Cross-lingual Automatic Speech Recognition Ex- ploiting Articulatory Features,” inAPSIPA ASC), 2019, pp. 1912– 1916

    2019

  5. [12]

    wav2vec 2.0: A frame- work for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamed et al., “wav2vec 2.0: A frame- work for self-supervised learning of speech representations,” Ad- vances in Neural Information Processing Systems , vol. 33, pp. 12 449–12 460, 2020. 2Source languages M=5 {cat, mar, guj, kan, tam}, M=10 {nbl, ssw...

    2020

  6. [13]

    HuBERT: Self-supervised speech representa- tion learning by masked prediction of hidden units,

    W.-N. Hsu et al., “HuBERT: Self-supervised speech representa- tion learning by masked prediction of hidden units,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 29, pp. 3451–3460, 2021

    2021

  7. [14]

    XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale,

    A. Babu, C. Wang, A. Tjandra et al., “XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale,” in Proc. Interspeech, 2022, pp. 2278–2282

    2022

  8. [15]

    W2v-BERT: Combining contrastive learning and masked language modeling for self-supervised speech pre- training,

    Y .-A. Chung et al., “W2v-BERT: Combining contrastive learning and masked language modeling for self-supervised speech pre- training,” in ASRU, 2021, pp. 244–250

    2021

  9. [16]

    WavLM: Large-scale self-supervised pre-training for full stack speech processing,

    “WavLM: Large-scale self-supervised pre-training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  10. [17]

    Self-supervised speech representation learn- ing: A review,

    A. Mohamed et al., “Self-supervised speech representation learn- ing: A review,” IEEE Journal of Selected Topics in Signal Pro- cessing, vol. 16, no. 6, pp. 1179–1210, 2022

    2022

  11. [18]

    Multi- resolution huBERT: Multi-resolution speech self-supervised learning with masked unit prediction,

    J. Shi, H. Inaguma, X. Ma, I. Kulikov, and A. Sun, “Multi- resolution huBERT: Multi-resolution speech self-supervised learning with masked unit prediction,” in Proc. ICLR, 2024

    2024

  12. [19]

    Ana- lyzing the factors affecting usefulness of self-supervised pre- trained representations for speech recognition,

    A. Seth, L. V . Prasad, S. Ghosh, and S. Umesh, “Ana- lyzing the factors affecting usefulness of self-supervised pre- trained representations for speech recognition,” arXiv preprint arXiv:2203.16973, 2022

    2022 arXiv

  13. [20]

    ML-SUPERB: Multilingual Speech Universal PER- formance Benchmark,

    J. Shi et al., “ML-SUPERB: Multilingual Speech Universal PER- formance Benchmark,” in Proc. INTERSPEECH, 2023, pp. 884– 888

    2023

  14. [21]

    Studying the impact of language model size for low-resource ASR,

    Z. Liu, J. Spence, and E. Prudhommeaux, “Studying the impact of language model size for low-resource ASR,” in ACL, 2023

    2023

  15. [22]

    A Novel Self-training Approach for Low-resource Speech Recognition,

    S. Singh et al., “A Novel Self-training Approach for Low-resource Speech Recognition,” arXiv preprint arXiv:2308.05269, 2023

    2023 arXiv

  16. [23]

    SUPERB: Speech Processing Universal PERformance Benchmark,

    S. wen Yang, P.-H. Chi, Y .-S. Chuang et al., “SUPERB: Speech Processing Universal PERformance Benchmark,” in Proc. Inter- speech, 2021, pp. 1194–1198

    2021

  17. [24]

    The power of scale for parameter-efficient prompt tuning,

    B. Lester et al. , “The power of scale for parameter-efficient prompt tuning,” in EMNLP, 2021, pp. 3045–3059

    2021

  18. [25]

    Parameter-efficient transfer learning for NLP,

    N. Houlsby et al., “Parameter-efficient transfer learning for NLP,” in ICML, 2019, pp. 2790–2799

    2019

  19. [26]

    AdapterBias: Parameter-efficient token- dependent representation shift for adapters in NLP tasks,

    C.-L. Fu et al. , “AdapterBias: Parameter-efficient token- dependent representation shift for adapters in NLP tasks,” inFind- ings of NAACL, 2022, pp. 2608–2621

    2022

  20. [28]

    BitFit: Sim- ple parameter-efficient fine-tuning for transformer-based masked language-models,

    E. Ben Zaken, Y . Goldberg, and S. Ravfogel, “BitFit: Sim- ple parameter-efficient fine-tuning for transformer-based masked language-models,” in ACL, 2022, pp. 1–9

    2022

  21. [29]

    Exploring efficient-tuning methods in self- supervised speech models,

    Z.-C. Chen et al. , “Exploring efficient-tuning methods in self- supervised speech models,” in SLT, 2023, pp. 1120–1127

    2023

  22. [30]

    Chapter: Exploiting Convolutional Neural Network Adapters for Self-Supervised Speech Models,

    C. Zih-Ching et al., “Chapter: Exploiting Convolutional Neural Network Adapters for Self-Supervised Speech Models,” in ICAS- SPW, 2023, pp. 1–5

    2023

  23. [31]

    Efficient Adapter Transfer of Self-Supervised Speech Models for Automatic Speech Recognition,

    B. Thomas et al., “Efficient Adapter Transfer of Self-Supervised Speech Models for Automatic Speech Recognition,” in ICASSP, 2022, pp. 7102–7106

    2022

  24. [32]

    Adapter pre-training for improved speech recog- nition in unseen domains using low resource adapter tuning of self-supervised models,

    S. Udupa et al., “Adapter pre-training for improved speech recog- nition in unseen domains using low resource adapter tuning of self-supervised models,” in Interspeech 2024 , 2024, pp. 2529– 2533

    2024

  25. [33]

    An Adapter Based Pre- Training for Efficient and Scalable Self-Supervised Speech Rep- resentation Learning,

    S. Kessler, B. Thomas, and S. Karout, “An Adapter Based Pre- Training for Efficient and Scalable Self-Supervised Speech Rep- resentation Learning,” in ICASSP, 2022, pp. 3179–3183

    2022

  26. [34]

    Exploiting adapters for cross-lingual low-resource speech recognition,

    W. Hou et al., “Exploiting adapters for cross-lingual low-resource speech recognition,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 30, pp. 317–329, 2021

    2021

  27. [35]

    Efficient domain adaptation for speech foundation models,

    B. Li et al., “Efficient domain adaptation for speech foundation models,” in ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023, pp. 1–5

    2023

  28. [36]

    Peft: State-of-the-art parameter-efficient fine-tuning methods,

    S. Mangrulkar, S. Gugger, L. Debut, Y . Belkada, S. Paul, and B. Bossan, “Peft: State-of-the-art parameter-efficient fine-tuning methods,” https://github.com/huggingface/peft, 2022

    2022

  29. [37]

    Zero-shot learning for grapheme to phoneme conver- sion with language ensemble,

    X. Li et al., “Zero-shot learning for grapheme to phoneme conver- sion with language ensemble,” in Findings of the ACL, 2022, pp. 2106–2115

    2022

  30. [38]

    Model-Agnostic Meta-Learning for Fast Adapta- tion of Deep Networks,

    C. Finn et al., “Model-Agnostic Meta-Learning for Fast Adapta- tion of Deep Networks,” in Proceedings of the 34th International Conference on Machine Learning - Volume 70 , ser. ICML’17, 2017, p. 1126–1135

    2017

  31. [39]

    Textless Speech-to- Speech Translation on Real Data,

    A. Lee, H. Gong, P.-A. Duquenne et al. , “Textless Speech-to- Speech Translation on Real Data,” inNACCL, 2022, pp. 860–872

    2022

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.