pith. sign in

arxiv: 2606.31642 · v1 · pith:36J2XRVJnew · submitted 2026-06-30 · 💻 cs.CL

Tone-Conditioned Curriculum Learning for Low-Resource Bantu Speech Recognition

Kesego Mokgosi , Vukosi Marivate , Sitwala Mundia , Unarine Netshifhefhe , Tsholofelo Hope Mogale , Thapelo Sindane This is my paper

Pith reviewed 2026-07-01 05:27 UTC · model grok-4.3

classification 💻 cs.CL
keywords speech recognitionBantu languagescurriculum learningtone conditioninglow-resource ASRtransfer learningautomatic speech recognition
0
0 comments X

The pith

A tone-conditioned curriculum improves transfer performance for automatic speech recognition on six Southern Bantu languages.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to reduce unusable error rates in foundation ASR models for Southern Bantu languages by conditioning curriculum training on tone. It combines hybrid difficulty scoring with gated adapters that receive tonal statistics as input during staged training on community data. Performance is then measured on transfer to the separate NCHLT evaluation sets. A reader would care because these languages have more than 80 million speakers yet current zero-shot models exceed 100 percent word error rate, limiting applications in education and public services. The results also show that architecture preference varies by language group and that tone conditioning contributes to the reported gains.

Core claim

The central claim is that a tone-conditioned curriculum framework built on hybrid difficulty scoring and gated adapters driven by tonal statistics enables better transfer from community corpora to standard benchmarks. W2V-BERT with tone conditioning reaches 28.41 percent average word error rate across datasets and 23.79 percent on Xitsonga transfer. W2V-BERT outperforms Whisper on Nguni languages while Whisper performs better on Sotho-Tswana languages, and no single model works equally well for all six languages.

What carries the argument

Tone-conditioned curriculum that uses tonal statistics both to compute hybrid difficulty scores and to control gated adapters during staged training.

If this is right

  • W2V-BERT should be selected for Nguni languages and Whisper for Sotho-Tswana languages rather than assuming one base model fits all.
  • Deployment requires per-language model selection followed by validation across multiple corpora instead of relying on a single universal model.
  • Training on community data with tone conditioning can produce measurable transfer improvements to standard evaluation sets for these languages.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same tonal statistics could be tested as input features for automated selection of the best base model per language.
  • The curriculum might be applied to additional tonal languages outside the six studied here to check whether the transfer benefit generalizes.

Load-bearing premise

Tonal statistics extracted from the speech data can be used to steer difficulty scoring and adapter gating in a way that produces robust gains on transfer to a held-out evaluation corpus.

What would settle it

Train the same models with and without the tone-conditioning components and check whether the word error rate advantage on the NCHLT transfer sets disappears.

Figures

Figures reproduced from arXiv: 2606.31642 by Kesego Mokgosi, Sitwala Mundia, Thapelo Sindane, Tsholofelo Hope Mogale, Unarine Netshifhefhe, Vukosi Marivate.

Figure 1
Figure 1. Figure 1: Overview of the proposed framework. (a) Pretraining phase computes hybrid difficulty scores from WER and tonal features. (b) Fine-tuning phase uses curriculum scheduling with gated tone conditioned adapters. (c) Inference applies the trained model. ing studies have described the phenomenon well but rarely in￾corporated tonal complexity directly into curriculum design or adapter gating. In low-resource Afri… view at source ↗
read the original abstract

Southern Bantu languages are spoken by over 80 million people, yet current foundation ASR models still produce zero-shot WER above 100%, which limits practical use in education and public services. We addressed this gap with a tone conditioned curriculum framework for 6 Southern Bantu languages that combined hybrid difficulty scoring, gated adapters driven by tonal statistics and staged curriculum training. We trained on a community corpus and tested transfer to NCHLT to measure robustness beyond matched evaluation. Results revealed clear interactions between architecture and language, with W2V-BERT outperforming Whisper on Nguni languages by 3 to 4 WER points whilst Whisper performed better on Sotho-Tswana languages. W2V-BERT with tone conditioning reached 28.41% average WER across datasets and 23.79% on Xitsonga transfer. No single model suited all 6 languages, so deployment should pair model selection per language with validation across corpora.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper proposes a tone-conditioned curriculum learning framework for low-resource ASR on 6 Southern Bantu languages. It combines hybrid difficulty scoring, gated adapters driven by tonal statistics, and staged training on a community corpus, with transfer testing on the NCHLT corpus. The work reports architecture-language interactions (W2V-BERT better on Nguni, Whisper on Sotho-Tswana) and specific transfer WERs (28.41% average and 23.79% on Xitsonga for W2V-BERT with tone conditioning), concluding that model selection must be language- and corpus-specific.

Significance. If the tone-conditioning component is shown to drive the reported gains, the approach could provide a practical route to improving transfer performance in tonal low-resource languages where zero-shot foundation models fail. The emphasis on cross-corpus robustness and per-language model selection is a useful deployment-oriented takeaway. However, the current presentation supplies no experimental details, baselines, or ablations, so the significance cannot yet be assessed.

major comments (2)
  1. [Abstract] Abstract: the central claim attributes the 28.41% average WER and 23.79% Xitsonga transfer result to the tone-conditioned curriculum (hybrid difficulty scoring + gated adapters + staged training), yet no ablation or control experiment isolating the contribution of the tonal statistics is reported. Without this, it is impossible to determine whether the tone input is causally responsible for gains beyond standard curriculum training and adapters.
  2. [Abstract] Abstract: the manuscript states concrete WER numbers and architecture-language interactions but supplies no experimental details on training data sizes, hyper-parameters, statistical significance tests, error bars, or the exact definition of the hybrid difficulty scoring function. These omissions make the results unverifiable from the given text.
minor comments (1)
  1. [Abstract] Abstract: the phrase 'zero-shot WER above 100%' is imprecise; WER is bounded at 100% by definition, so the intended meaning (e.g., 'effectively unusable' or 'high deletion rates') should be clarified.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments highlighting the need for stronger evidence on the role of tone conditioning and for complete experimental details. We address each point below and will revise the manuscript to improve verifiability.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim attributes the 28.41% average WER and 23.79% Xitsonga transfer result to the tone-conditioned curriculum (hybrid difficulty scoring + gated adapters + staged training), yet no ablation or control experiment isolating the contribution of the tonal statistics is reported. Without this, it is impossible to determine whether the tone input is causally responsible for gains beyond standard curriculum training and adapters.

    Authors: We agree that an ablation isolating the tonal statistics is required to support the causal claim. In the revised manuscript we will add a controlled ablation comparing the full tone-conditioned curriculum against an otherwise identical curriculum that omits the tonal input to the gated adapters. This will directly test whether the tonal statistics contribute beyond standard curriculum learning and adapters. revision: yes

  2. Referee: [Abstract] Abstract: the manuscript states concrete WER numbers and architecture-language interactions but supplies no experimental details on training data sizes, hyper-parameters, statistical significance tests, error bars, or the exact definition of the hybrid difficulty scoring function. These omissions make the results unverifiable from the given text.

    Authors: We accept that the current text lacks these details. The revised version will include a new Experimental Setup subsection that reports per-language training data sizes, the full hyper-parameter configuration, results of statistical significance tests, error bars on reported WERs, and the precise mathematical definition of the hybrid difficulty scoring function. revision: yes

Circularity Check

0 steps flagged

Empirical ASR results with no derivation chain or self-referential reductions

full rationale

The paper reports direct empirical WER measurements (28.41% avg, 23.79% Xitsonga) from training W2V-BERT/Whisper variants on a community corpus and evaluating transfer to NCHLT. The abstract and available text describe a combined system (hybrid difficulty scoring + gated adapters + staged training) but contain no equations, fitted parameters renamed as predictions, uniqueness theorems, or self-citations that bear load on the central claim. No load-bearing step reduces by construction to its own inputs; results are presented as measured outcomes rather than derived quantities.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract contains no mathematical derivations, free parameters, axioms, or invented entities; the contribution is described at the level of an empirical training framework.

pith-pipeline@v0.9.1-grok · 5720 in / 1138 out tokens · 30923 ms · 2026-07-01T05:27:37.487017+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 3 canonical work pages · 2 internal anchors

  1. [1]

    They remain primarily oral, and tone and prosody encode grammatical and cultural distinctions not fully captured in writing

    Introduction Southern Bantu languages such as isiZulu, isiXhosa, Sesotho, Setswana, Tshivenda and Xitsonga are among the most widely spoken in sub-Saharan Africa [1] and hold official status in South Africa. They remain primarily oral, and tone and prosody encode grammatical and cultural distinctions not fully captured in writing. As communities expand la...

  2. [2]

    A hybrid difficulty scoring function combining WER with morphotonal features for curriculum learning

  3. [3]

    Gated tone conditioned adapters with 2.1M parameters that modulate encoder representations based on tonal statistics per utterance

  4. [4]

    Evaluation across Whisper, W2V-BERT and MMS on 2 datasets, revealing architecture preferences by language fam- ily

  5. [5]

    Generalisation analysis across datasets showing W2V-BERT transfers better for Nguni languages

  6. [6]

    Tone-Conditioned Curriculum Learning for Low-Resource Bantu Speech Recognition

    Related Work Curriculum learning in ASR orders training samples by esti- mated difficulty so models first learn stable patterns and then harder cases [10, 11]. Difficulty has been defined by du- ration, signal-to-noise ratio, model loss and recognition er- ror [9]. Reported gains include lower optimisation variance in low-resource settings and a 41.02% WE...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  7. [7]

    The pipeline comprised 3 phases, namely hybrid difficulty scoring, staged fine-tuning with gated adapters and inference

    Methodology Figure 1 presents the tone conditioned curriculum framework, guided by 2 constraints, namely limited Southern Bantu training data and preservation of tonal linguistic structure. The pipeline comprised 3 phases, namely hybrid difficulty scoring, staged fine-tuning with gated adapters and inference. 3.1. Hybrid Difficulty Scoring To formalise cu...

    2000

  8. [8]

    Experiments 4.1. Dataset We used the Swivuriso/za-african-next-voices dataset [26], a corpus developed by speaker communities with read speech from 6 Southern Bantu languages; isiZulu with 1,810 samples, isiXhosa with 2,470, Sesotho with 2,717, Setswana with 4,299, Tshivenda with 1,192 and Xitsonga with 3,500. The dev test split contained 15,988 samples a...

  9. [9]

    Conclusions We presented a tone conditioned curriculum framework for 6 Southern Bantu languages and evaluated 12 configurations across Whisper, W2V-BERT and MMS on both Swivuriso and NCHLT. The central finding was that architecture suitability varied by language family, with W2V-BERT achieving lower error rates on Nguni languages whereas Whisper held an a...

  10. [10]

    We ac- knowledge the support and engagement of Masakhane, Lan- frica, the University of Pretoria (ITS and Research Office)

    Acknowledgements We thank all collaborators, institutions, and partners who con- tributed to the African Next V oices project (funded through a grant from Gates Foundation and a gift from Meta). We ac- knowledge the support and engagement of Masakhane, Lan- frica, the University of Pretoria (ITS and Research Office). The authors acknowledge the Data Scien...

  11. [11]

    Generative AI Use Disclosure Generative AI tools were used solely to assist with language editing and clarity of presentation. All research ideas, method- ology, experiments as well as interpretations were conceived and carried out by the authors, who take full responsibility for the originality, validity, and integrity of the work

  12. [12]

    Nchlt auxiliary speech data for asr technology development in south africa,

    J. Badenhorst and F. de Wet, “Nchlt auxiliary speech data for asr technology development in south africa,”Data in Brief, vol. 41, p. 107860, 2022

    2022

  13. [13]

    Navajo speech recognition using low-resource language models,

    E. M. Sutherland, M. R. Chee, and M. S. Pattichis, “Navajo speech recognition using low-resource language models,” in2024 58th Asilomar Conference on Signals, Systems, and Computers. IEEE, 2024, pp. 679–683

    2024

  14. [14]

    Opportunities and challenges of automatic speech recognition systems for low- resource language speakers,

    T. Reitmaier, E. Wallington, D. Kalarikalayil Raju, O. Klejch, J. Pearson, M. Jones, P. Bell, and S. Robinson, “Opportunities and challenges of automatic speech recognition systems for low- resource language speakers,” inProceedings of the 2022 CHI con- ference on human factors in computing systems, 2022, pp. 1–17

    2022

  15. [15]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” inInternational conference on machine learning. PMLR, 2023, pp. 28 492–28 518

    2023

  16. [16]

    Scaling speech technology to 1,000+ languages,

    V . Pratap, A. Tjandra, B. Shi, P. Tomasello, A. Babu, S. Kundu, A. Elkahky, Z. Ni, A. Vyas, M. Fazel-Zarandiet al., “Scaling speech technology to 1,000+ languages,”Journal of Machine Learning Research, vol. 25, no. 97, pp. 1–52, 2024

    2024

  17. [17]

    Influences on tone in sepedi, a south- ern bantu language,

    S. Zerbian and E. Barnard, “Influences on tone in sepedi, a south- ern bantu language,” inInterspeech 2008, 2008, p. 625

    2008

  18. [18]

    A study on the perception of tone and intonation in sesotho,

    H. Mixdorff, L. Mohasi, M. Machobane, and T. Niesler, “A study on the perception of tone and intonation in sesotho,” inInter- speech 2011, 2011, pp. 3181–3184

    2011

  19. [19]

    An acoustic analysis of tone in sesotho

    L. Mohasi, H. Mixdorff, and T. Niesler, “An acoustic analysis of tone in sesotho.” inICPhS, vol. 17, 2011, pp. 17–21

    2011

  20. [20]

    Comparison and Analysis of New Curriculum Criteria for End- to-End ASR,

    Georgios Karakasidis and Tam ´as Gr ´osz and Mikko Kurimo, “Comparison and Analysis of New Curriculum Criteria for End- to-End ASR,” inInterspeech 2022, 2022, pp. 66–70

    2022

  21. [21]

    Curriculum learning based ap- proaches for robust end-to-end far-field speech recognition,

    S. Ranjan and J. H. Hansen, “Curriculum learning based ap- proaches for robust end-to-end far-field speech recognition,” Speech Communication, vol. 132, pp. 123–131, 2021

    2021

  22. [22]

    Cur- riculum optimization for low-resource speech recognition,

    A. Kuznetsova, A. Kumar, J. D. Fox, and F. M. Tyers, “Cur- riculum optimization for low-resource speech recognition,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 8187– 8191

    2022

  23. [23]

    Cba-whisper: Curriculum learning-based adalora fine-tuning on whisper for low-resource dysarthric speech recognition,

    T. Tan, X. Chen, X. Le, W. Fan, X. Xia, C. Huang, and J. Lu, “Cba-whisper: Curriculum learning-based adalora fine-tuning on whisper for low-resource dysarthric speech recognition,” inProc. Interspeech 2025, 2025, pp. 3309–3313

    2025

  24. [24]

    Dysarthric speech recognition us- ing curriculum learning and articulatory feature embedding,

    I.-T. Hsieh, C.-H. Wuet al., “Dysarthric speech recognition us- ing curriculum learning and articulatory feature embedding,” in Proceedings of the Interspeech, 2024, pp. 1300–1304

    2024

  25. [25]

    Curriculum learning based ap- proaches for noise robust speaker recognition,

    S. Ranjan, John HL and Hansen, “Curriculum learning based ap- proaches for noise robust speaker recognition,”IEEE/ACM Trans- actions on Audio, Speech, and Language Processing, vol. 26, no. 1, pp. 197–210, 2017

    2017

  26. [26]

    Curriculum pre-training for end-to-end speech translation,

    C. Wang, Y . Wu, S. Liu, M. Zhou, and Z. Yang, “Curriculum pre-training for end-to-end speech translation,” inProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 3728–3738

    2020

  27. [27]

    Exploring effective data utilization for low-resource speech recognition,

    Z. Zhou, W. Wang, W. Zhang, and Y . Qian, “Exploring effective data utilization for low-resource speech recognition,” inICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 8192–8196

    2022

  28. [28]

    Phonemic tran- scription of low-resource tonal languages,

    O. Adams, T. Cohn, G. Neubig, and A. Michaud, “Phonemic tran- scription of low-resource tonal languages,” inProceedings of the Australasian Language Technology Association Workshop 2017, 2017, pp. 53–60

    2017

  29. [29]

    Robust mizo digit recognition using data aug- mentation and tonal information,

    B. D. Sarma, A. Dey, W. Lalhminghlui, P. Gogoi, P. Sarmah, and S. R. M. Prasanna, “Robust mizo digit recognition using data aug- mentation and tonal information,” inSpeech Prosody 2018, 2018, pp. 621–625

    2018

  30. [30]

    Using tone informa- tion in cantonese continuous speech recognition,

    T. Lee, W. Lau, Y . W. Wong, and P. Ching, “Using tone informa- tion in cantonese continuous speech recognition,”ACM Transac- tions on Asian Language Information Processing (TALIP), vol. 1, no. 1, pp. 83–102, 2002

    2002

  31. [31]

    Explicit tone transcription improves asr per- formance in extremely low-resource languages: A case study in bribri,

    R. Coto-Solano, “Explicit tone transcription improves asr per- formance in extremely low-resource languages: A case study in bribri,” inProceedings of the first workshop on natural language processing for Indigenous languages of the Americas, 2021, pp. 173–184

    2021

  32. [32]

    The tnt team system descriptions of cantonese and mongolian for iarpa openasr20,

    J. Zhao, Z. Lv, A. Han, G.-B. Wang, G. Shi, J. Kang, J. Yan, P. Hu, S. Huang, and W.-Q. Zhang, “The tnt team system descriptions of cantonese and mongolian for iarpa openasr20,” inInterspeech 2021, 2021, pp. 4344–4348

    2021

  33. [33]

    Multilingual self- supervised speech representations improve the speech recognition of low-resource african languages with codeswitching,

    T. Ogunremi, C. D. Manning, and D. Jurafsky, “Multilingual self- supervised speech representations improve the speech recognition of low-resource african languages with codeswitching,” inPro- ceedings of the 6th Workshop on Computational Approaches to Linguistic Code-Switching, 2023, pp. 83–88

    2023

  34. [34]

    Kinspeak: Improving speech recognition for kin- yarwanda via semi-supervised learning methods,

    A. Nzeyimana, “Kinspeak: Improving speech recognition for kin- yarwanda via semi-supervised learning methods,”arXiv preprint arXiv:2308.11863, 2023

    work page arXiv 2023

  35. [35]

    Fast and efficient multilingual self-supervised pre-training for low-resource speech recognition,

    Z. Zhang, W. Wang, and Y . Qian, “Fast and efficient multilingual self-supervised pre-training for low-resource speech recognition,” inProc. Interspeech, 2023

    2023

  36. [36]

    Openasr21: The second open challenge for automatic speech recognition of low-resource lan- guages

    K. Peterson, A. Tong, and Y . Yu, “Openasr21: The second open challenge for automatic speech recognition of low-resource lan- guages.” inINTERSPEECH, 2022, pp. 4895–4899

    2022

  37. [37]

    Swivuriso: The South African Next Voices Multilingual Speech Dataset

    V . Marivate, K. Olaleye, S. Mundia, A. Bakainga, U. Net- shifhefhe, M. Milanzie, T. H. Mogale, T. Sindane, Z. Abdul- rasaq, K. Mokgosi, C. Okorie, N. Z. V . Wyk, G. Morrissey, D. Dunbar, F. Smit, T. Chidi, R. Mabuya, A. Bukula, R. Mlambo, T. Macucwa, I. Abdulmumin, , and S. Rananga, “Swivuriso: The south african next voices multilingual speech dataset,”a...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  38. [38]

    Thai dialect corpus and transfer-based curriculum learning investigation for dialect automatic speech recognition,

    A. Suwanbandit, B. Naowarat, O. Sangpetch, and E. Chuang- suwanich, “Thai dialect corpus and transfer-based curriculum learning investigation for dialect automatic speech recognition,” inProc. Interspeech, vol. 2, 2023

    2023

  39. [39]

    Curriculum-based transfer learning for an effec- tive end-to-end spoken language understanding and domain porta- bility,

    A. Caubri `ere, N. Tomashenko, A. Laurent, E. Morin, N. Camelin, and Y . Est`eve, “Curriculum-based transfer learning for an effec- tive end-to-end spoken language understanding and domain porta- bility,” inInterspeech 2019, 2019, pp. 1198–1202

    2019