Pith. sign in

REVIEW 1 major objections 2 minor 27 references

Building an ASR Solution for Training and Assessing Children's Reading

T0 review · 1 major / 2 minor · reviewed 2026-07-01 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A Bambara-adapted Fast-Conformer ASR halves word error rate for children's reading assessment.

desk verdict This paper gives a practical pipeline and benchmark for Bambara child reading ASR but the gains may not hold if the 60-speaker set overlaps across splits. read the letter →

arxiv 2606.31508 v1 pith:E2BXY2NM submitted 2026-06-30 cs.CL cs.SD

Yacouba Diarra , Nouhoum Souleymane Coulibaly , Mamadou Dembele , Aymane Dembele , Michael Leventhal This is my paper · ORCID
classification cs.CLcs.SD
keywords ASRBambarachildren'sreadingspeechrecognitionliteracyassessmentFast-Conformerlow-resourcelanguages
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper builds a complete open-source system for using automatic speech recognition to assess children's reading in Bambara. Starting with mobile data collection of 55 hours from 60 children, the authors create a public benchmark and fine-tune models to improve recognition accuracy. Their adapted Soloni model cuts word error rate from 0.42 to 0.22 and character error rate from 0.15 to 0.08 compared to QuartzNet. This reduction supports more reliable literacy evaluation in classrooms where manual scoring varies. Analysis of results shows younger children drive most errors, and classroom tests confirm the app's usability.

What carries the argument

Soloni, the Bambara-adapted Fast-Conformer ASR framework using TDT and CTC decoders, which is fine-tuned on the collected child speech data to perform reading assessment.

What would settle it

Running the model on reading recordings from additional Bambara-speaking children outside the original 60 would show if the error rates stay low or rise significantly.

Watch

Extended reading notes

Core claim

By adapting the Fast-Conformer architecture into Soloni with TDT and CTC decoders and training on collected Bambara child reading data, the system achieves a word error rate of 0.22 and character error rate of 0.08 on the benchmark, outperforming the QuartzNet baseline, while also demonstrating that repeated readings aid one architecture more than the other and that errors cluster among readers under age 10.

Load-bearing premise

The speech data collected from 60 children over 55 hours represents the target population well enough for the models to work in actual classroom reading assessments.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 2 minor

Summary. The paper presents an end-to-end open-source ASR pipeline for Bambara children's reading assessment. It covers collection of 55 hours of speech from 60 children via a mobile app, construction of a public benchmark, fine-tuning of a Bambara-adapted Fast-Conformer (Soloni) with TDT/CTC decoders versus QuartzNet, reporting WER reduction from 0.42 to 0.22 and CER from 0.15 to 0.08 on the benchmark, analysis of repeated readings and SpecAugment effects, disaggregation showing higher errors for children under 10, and 10 classroom validation trials.

Significance. If the benchmark gains are shown to be speaker-independent, the work supplies a rare public resource and adapted models for low-resource child ASR in an African language, supporting reproducible literacy tools. The integrated data-to-app workflow and identification of age-specific error patterns are constructive contributions that could seed further targeted data collection.

major comments (1)
  1. [Abstract / benchmark construction] Abstract and benchmark construction section: The WER/CER gains (0.42→0.22 WER, 0.15→0.08 CER) are reported on a benchmark derived from the same 60 children, yet no details are given on the train/test partition (e.g., whether it is speaker-independent). If child-level overlap exists, the Soloni advantage over QuartzNet may partly reflect memorization of individual vocal characteristics rather than robust generalization to new readers, directly weakening the central claim that the model substantially outperforms on a representative isolated benchmark.
minor comments (2)
  1. [Abstract] Abstract: The statement that repeated readings 'substantially improve QuartzNet but add only marginal gains for Soloni' would benefit from explicit numeric deltas or a table row to allow readers to assess the architecture-dependent effect size.
  2. [Abstract] Abstract: No mention of statistical significance testing, confidence intervals, or error analysis methodology is provided to support the numeric performance claims.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback and the recommendation for major revision. We address the single major comment below and will update the manuscript to strengthen the presentation of the benchmark.

read point-by-point responses

  1. Referee: [Abstract / benchmark construction] Abstract and benchmark construction section: The WER/CER gains (0.42→0.22 WER, 0.15→0.08 CER) are reported on a benchmark derived from the same 60 children, yet no details are given on the train/test partition (e.g., whether it is speaker-independent). If child-level overlap exists, the Soloni advantage over QuartzNet may partly reflect memorization of individual vocal characteristics rather than robust generalization to new readers, directly weakening the central claim that the model substantially outperforms on a representative isolated benchmark.

    Authors: We agree that explicit details on the train/test partition are required to substantiate claims of generalization. The current manuscript does not specify the partitioning strategy or confirm that the split is speaker-independent at the child level. In the revised version we will expand the benchmark construction section with a clear description of the split (including the proportion of children held out for testing) and will state whether it is performed on a per-child basis to ensure no speaker overlap between train and test sets. This addition will directly address the concern and allow readers to evaluate the robustness of the reported gains. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: purely empirical fine-tuning on newly collected speech data

full rationale

The paper reports standard ASR fine-tuning experiments on 55 hours of newly collected Bambara child-reading speech from 60 speakers. Performance metrics (WER/CER reductions) are measured outcomes on a constructed benchmark; no equations, self-definitional relations, fitted parameters renamed as predictions, or load-bearing self-citations appear. The derivation chain consists of data collection, benchmark construction, model adaptation, and empirical evaluation, all externally falsifiable against the held-out test utterances. No step reduces to its inputs by construction.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

The central claim rests on standard ASR training assumptions and the representativeness of a small collected dataset; no new entities or ad-hoc axioms are introduced beyond typical domain assumptions for speech recognition evaluation.

assumptions (1)
  • domain assumption WER and CER on isolated benchmark sentences accurately measure real-world reading assessment quality
    The paper uses these metrics to claim the system supports classroom use.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Building an ASR Solution for Training and Assessing Children's Reading." pith.science (2026). https://pith.science/paper/E2BXY2NM

@misc{pith2026260631508,
  author       = {Pith},
  title        = {Pith review of: Building an ASR Solution for Training and Assessing Children's Reading},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E2BXY2NM}},
  note         = {Machine review of arXiv:2606.31508}
}
read the original abstract

Automatic speech recognition for children's reading remains underdeveloped for most African languages, including Bambara, despite its potential value for reproducible literacy assessment. We present an open-source system for assessing children's reading in Bambara, developed through an end-to-end process linking field data collection, benchmark construction, model adaptation, a reading application, and classroom validation. A mobile collection and assessment app was used to collect 55 hours of raw reading speech from 60 children, from which we construct a public benchmark for Bambara child-reading assessment. Fine-tuning experiments compare Soloni, a Bambara-adapted Fast-Conformer ASR framework with TDT and CTC decoders, with QuartzNet, a compact convolutional ASR architecture. The best Soloni model reduces WER from 0.42 to 0.22 and CER from 0.15 to 0.08, substantially outperforming QuartzNet on the isolated benchmark. The experiments further show that repeated readings of the same texts provide architecture-dependent benefits: they substantially improve QuartzNet but add only marginal gains for Soloni, while SpecAugment regulates training without exceeding the best unaugmented configuration. Disaggregated analysis identifies children under 10 as the main source of residual errors, motivating targeted collection from younger readers. Ten classroom trials supported continued use of the application.

Figures

Figures reproduced from arXiv: 2606.31508 by the authors.

Figure 1
Figure 1. Overall performance by model (WER and CER), sorted by [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. WER by age cohort before (Soloni-114M base) and after fine [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 27 canonical work pages

  1. [1]

    Automatic speech recognition (ASR) for African low-resource languages: A systematic literature review,

    S. H. Imam et al., "Automatic speech recognition (ASR) for African low-resource languages: A systematic literature review," arXiv:2510.01145, 2025

    work page arXiv 2025

  2. [2]

    Dealing with the hard facts of low-resource African NLP,

    Y. Diarra et al., "Dealing with the hard facts of low-resource African NLP," arXiv:2501.10315 [cs.CL], 2025

    work page arXiv 2025

  3. [3]

    A formant modification method for improved ASR of children's speech,

    H. Kathania et al., "A formant modification method for improved ASR of children's speech," in Proc. Interspeech, 2022

    work page 2022

  4. [4]

    Automatic speech recognition (ASR) systems for children: A review,

    V. Bhardwaj et al., "Automatic speech recognition (ASR) systems for children: A review," Applied Sciences, vol. 12, no. 9, Art. no. 4419, 2022

    work page 2022

  5. [5]

    Pearce, Delivering Education for All in Mali

    C. Pearce, Delivering Education for All in Mali. Oxford, U.K.: Oxfam International, 2009. [Online]. Available: https://www-cdn.oxfam.org/s3fs-public/file_attachments/delivering- education-for-all-mali-report-07-06-09_4.pdf

    work page 2009

  6. [6]

    Protecting education from attack through strong coordination in Mali,

    Global Education Cluster, "Protecting education from attack through strong coordination in Mali," May 2025. [Online]. Available: https://www.educationcluster.net/news/protecting-education-attack- through-strong-coordination-mali

    work page 2025

  7. [7]

    World Bank and UNESCO Institute for Statistics, Mali Learning Poverty Brief, Washington, DC, USA: World Bank, Apr. 2024. [Online]. Available: https://documents1.worldbank.org/curated/en/099062524153069090/pd f/P17920911989040bd19f27126a7249becfd.pdf

    work page arXiv 2024

  8. [8]

    Speech enabled reading fluency assessment: A validation study,

    M. van der Velde et al., "Speech enabled reading fluency assessment: A validation study," International Journal of Artificial Intelligence in Education, 2025

    work page 2025

Show all 27 references
  1. [9]

    Automatic assessment of oral reading accuracy for reading diagnostics,

    B. Molenaar, C. Tejedor-Garcia, H. Strik, and C. Cucchiarini, "Automatic assessment of oral reading accuracy for reading diagnostics," arXiv:2306.03444, 2023

    2023

  2. [10]

    Deep learning for assessment of oral reading fluency,

    M. Vaidya, B. K. Sahoo, and P. Rao, "Deep learning for assessment of oral reading fluency," arXiv:2405.19426, 2024

    2024

  3. [11]

    SpecAugment: A simple data augmentation method for automatic speech recognition,

    D. S. Park et al., "SpecAugment: A simple data augmentation method for automatic speech recognition," in Proc. Interspeech, 2019, pp. 2613-2617

    2019

  4. [12]

    Where are we at with automatic speech recognition for the Bambara language?

    S. Diallo, Y. Diarra, M. K. Keita, P. A. Kamaté, A. B. Kampo, and A. Ouattara, "Where are we at with automatic speech recognition for the Bambara language?" arXiv:2601.04122 [cs.CL], 2026

    2026 arXiv

  5. [13]

    Kunnafonidilaw ka Cadeau: An ASR dataset of present-day Bambara,

    M. Leventhal, Y. Diarra, N. S. Coulibaly, P. A. Kamaté, M. A. Tall, A. Dembélé, and E. E. Koné, "Kunnafonidilaw ka Cadeau: An ASR dataset of present-day Bambara," in Proc. AfricaNLP, 2026

    2026

  6. [14]

    Supporting literacy assessment in West Africa: Using state-of-the-art speech models to assess oral reading fluency,

    O. Henkel, H. Horne-Robinson, L. Hills, B. Roberts, and J. McGrane, "Supporting literacy assessment in West Africa: Using state-of-the-art speech models to assess oral reading fluency," International Journal of Artificial Intelligence in Education, vol. 35, no. 1, pp. 282-303, 2025

    2025

  7. [15]

    Bamako, Mali: RobotsMali, 2026 [Online]

    RobotsMali AI4D Lab, An bɛ kalan Benchmark Technical Report V0. Bamako, Mali: RobotsMali, 2026 [Online]. Available: https://docs.google.com/document/d/1iTh8wZZX3Wm765za8xHk_KH _iGlCPwxE6zDC5Ri9Bpg/edit?usp=sharing

    2026

  8. [16]

    an-be-kalan-bench dataset,

    RobotsMali, "an-be-kalan-bench dataset," Hugging Face Datasets,

  9. [17]

    Available: https://huggingface.co/datasets/RobotsMali/an-be-kalan-bench

    [Online]. Available: https://huggingface.co/datasets/RobotsMali/an-be-kalan-bench

  10. [18]

    GAIfE: Using GenAI to improve literacy in low- resourced settings,

    A. Tapo et al., "GAIfE: Using GenAI to improve literacy in low- resourced settings," in Proc. 2025 Conf., 2025, pp. 7914-7929

    2025

  11. [19]

    QuartzNet: Deep automatic speech recognition with 1D time-channel separable convolutions,

    S. Kriman et al., "QuartzNet: Deep automatic speech recognition with 1D time-channel separable convolutions," in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 6124- 6128

    2020

  12. [20]

    Efficient sequence transduction by jointly predicting tokens and durations,

    H. Xu, F. Jia, S. Majumdar, H. Huang, S. Watanabe, and B. Ginsburg, "Efficient sequence transduction by jointly predicting tokens and durations," in Proc. 40th Int. Conf. Machine Learning (ICML), 2023, pp. 38462-38484

    2023

  13. [21]

    Fast conformer with linearly scalable attention for efficient speech recognition,

    N. Rekesh et al., "Fast conformer with linearly scalable attention for efficient speech recognition," in Proc. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2023

    2023

  14. [22]

    Jeli-ASR,

    RobotsMali, “Jeli-ASR,” Hugging Face Datasets, 2025. [Online]. Available: https://huggingface.co/datasets/RobotsMali/jeli-asr. [Accessed: Jun. 29, 2026]

    2025

  15. [23]

    African Next Voices – Bambara (AfVoices),

    RobotsMali, “African Next Voices – Bambara (AfVoices),” Hugging Face Datasets, 2025. [Online]. Available: https://huggingface.co/datasets/RobotsMali/afvoices. [Accessed: Jun. 29, 2026]

    2025

  16. [24]

    Individuals using the Internet (% of population) – Mali,

    World Bank, "Individuals using the Internet (% of population) – Mali," World Development Indicators, data from International Telecommunication Union, 2024. [Online]. Available: https://data.worldbank.org/indicator/IT.NET.USER.ZS?locations=ML

    2024

  17. [25]

    Washington, DC, USA: World Bank, 2023

    World Bank Group, Digital Economy for Mali Country Diagnostic Report. Washington, DC, USA: World Bank, 2023. [Online]. Available: https://documents.worldbank.org/en/publication/documents- reports/documentdetail/099320408092229246

    2023

  18. [26]

    Dissecting user-perceived latency of on-device E2E speech recognition,

    Y. Shangguan et al., "Dissecting user-perceived latency of on-device E2E speech recognition," in Proc. Interspeech, 2021, pp. 4553-4557

    2021

  19. [27]

    Effects of feedback in a computer-based assessment for learning,

    F. M. van der Kleij, T. J. H. M. Eggen, C. F. Timmers, and B. P. Veldkamp, "Effects of feedback in a computer-based assessment for learning," Computers & Education, vol. 58, no. 1, pp. 263-272, 2012

    2012

Pith tools

Reviewed July 1, 2026 · model on record in the stance chip above.