Pith. sign in

REVIEW 4 major objections 4 minor 16 references

Natural language processing for African languages

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This dissertation shows that curated data plus a single multilingual adaptation step can bring 21 African languages to competitive NLP performance.

desk verdict A useful, honest resource-compilation dissertation whose main empirical claim (AfroXLMR gains) is overstated by a real domain confound, but the released corpora and models deserve serious engagement. read the letter →

arxiv 2507.00297 v1 pith:42245LDJ submitted 2025-06-30 cs.CL cs.AI

David Ifeoluwa Adelani This is my paper · ORCID
classification cs.CLcs.AI
keywords Africanlanguageslow-resourceNLPnamedentityrecognitionmachinetranslationmultilingualpre-trainedlanguagemodelswordembeddingsdataqualitytransferlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This dissertation argues that the under-representation of African languages in NLP is not a fixed scarcity but a solvable mismatch between what the web contains and what current models are trained on. It shows that massively crawled multilingual corpora yield poor word embeddings for African languages, and that small, carefully curated corpora trained with the right architecture improve similarity-task correlations by roughly 170-180%. It then extends the argument to pre-trained language models, introducing multilingual adaptive fine-tuning (MAFT) to produce a single model, AfroXLMR, that beats other multilingual PLMs on named entity recognition, news topic classification, and sentiment analysis for African languages. Finally, it contributes human-annotated NER and machine-translation datasets covering 21 African languages, turning those languages from evaluation blind spots into benchmarked participants.

What carries the argument

The load-bearing mechanism is multilingual adaptive fine-tuning (MAFT): taking an existing multilingual masked language model such as XLM-R and continuing its masked-language-model training on a mix of monolingual texts from 17 African languages plus English, French, and Arabic, without changing the training objective. Combined with vocabulary reduction, which removes embedding tokens for scripts irrelevant to the target languages before adaptation, MAFT yields a single adapted model (AfroXLMR) whose downstream fine-tuning on NER, news topic classification, and sentiment analysis is competitive with, and sometimes better than, training a separate language-adapted model for each language. The dissertation also treats corpus curation as machinery: it shows that the noise level of the pre-training corpus, not its size, determines embedding quality for Twi and Yorùbá.

What would settle it

Compare AfroXLMR against an identical MAFT model adapted only on clean, domain-mismatched texts such as curated religious corpora in the same languages: if news-domain scores drop to the XLM-R baseline, the MAFT gain is domain adaptation, not language adaptation. Alternatively, run the same adaptation recipe on the line-cleaned web corpus for a set of control languages and measure the downstream F1 gap; the language-adaptation interpretation predicts no gap when corpus quality is held constant.

Watch

Extended reading notes

Core claim

The central claim is that African languages can be brought into NLP with attainable resources if the data is curated and the model is adapted to many languages at once rather than one at a time. The dissertation demonstrates this in three moves: (1) pre-trained FastText embeddings for Twi and Yorùbá score near chance on word-similarity because their training text is tiny or polluted by other languages, while embeddings trained on a few hundred thousand curated tokens improve correlation with human judgements from about 0.14 to 0.39-0.44; (2) applying MAFT, which continues masked-language-model training on monolingual text from 20 languages, creates AfroXLMR, which outperforms existing multilingual PLMs and matches per-language adaptation while needing only one model; (3) the released datasets, MasakhaNER for 21 languages, MAFAND-MT for 21 languages, and MENYO-20k for Yorùbá-English, provide the evaluation infrastructure that makes such measurements possible. The dissertation also finds that fine-tuning large pre-trained translation models on a few thousand high-quality translations is the most effective route to African news translation.

Load-bearing premise

The claim that adaptation to African languages, rather than domain overlap with the evaluation data, drives AfroXLMR's improvements rests on the assumption that the web-crawled adaptation corpus is clean enough and general enough to count as language adaptation; if it is as noisy as the Twi and Yorùbá corpora documented in Chapter 4, part of the measured gain could be domain adaptation to news and social media text.

Editorial extensions

If this is right

  • With only 2,000 labelled sentences, a fine-tuned XLM-R reaches about 80 F1 on average across African-language NER, roughly doubling the data efficiency of a BiLSTM-CRF with word embeddings.
  • MAFT gives practitioners one adapted model instead of one per language; on NER, news topic classification, and sentiment analysis it is within about 0.5 F1 of per-language adaptation while cutting storage needs by a factor of 13.
  • Trimming the vocabulary to 70k tokens before MAFT halves the model size while losing only 1-2 F1 points, which makes the model practical for limited GPU memory.
  • For machine translation into a new domain, fine-tuning a large pre-trained model such as M2M-100 on a few thousand high-quality parallel sentences outperforms training from scratch or using noisy web data.
  • The released datasets for 21 African languages provide public NER and MT benchmarks, making supervised, weakly-supervised, and transfer-learning evaluation possible for these languages.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If AfroXLMR's gains partly reflect domain overlap between the web-crawled adaptation text and the news/social-media evaluation sets, then AfroXLMR may transfer less strongly to non-news domains such as health or legal text; a domain-matched control would test this directly.
  • The vocabulary-reduction recipe, dropping tokens for non-target scripts before continued pre-training, is portable to other multilingual models and scripts, so the same compression could be tried for models serving other low-resource language clusters.
  • The finding that 2,000 NER sentences suffice with PLMs suggests annotation budgets are better spent on more languages or more tasks than on ever-larger per-language datasets.
  • If the data-quality result generalizes beyond FastText, the common practice of scraping ever more web text for low-resource languages may be less valuable than building smaller curated corpora in partnership with native speakers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This dissertation addresses data and modeling gaps for Sub-Saharan African languages in NLP. It surveys language characteristics and resources (Chapters 2-3), shows with word-similarity and NER experiments that corpus quality matters more than quantity for Yoruba and Twi embeddings (Chapter 4), and proposes multilingual adaptive fine-tuning (MAFT) to adapt XLM-R to African languages, releasing the AfroXLMR family and an ANTC news-topic corpus (Chapter 5). It studies distant supervision with noisy-label learning for Hausa and Yoruba NER (Chapter 6), presents the MasakhaNER 1.0 and 2.0 human-annotated NER datasets for 10 and 21 languages with supervised and cross-lingual transfer results (Chapters 7-8), and introduces MENYO-20k Yoruba-English and MAFAND-MT 21-language news translation corpora, showing that fine-tuning large pretrained MT models on small high-quality parallel data is effective (Chapters 9-10). The abstract and headline claims state that these resources are large-scale and that AfroXLMR is the best multilingual PLM on African-language benchmarks.

Significance. If the empirical results hold, the dissertation's released resources—MasakhaNER, MasakhaNER 2.0, MENYO-20k, MAFAND-MT, the curated embeddings, ANTC, and the AfroXLMR model family—are substantial and reusable assets for 20+ African languages, and the participatory data-creation model is an important methodological contribution. The chapter-level multi-run experiments, public URLs to code/models/datasets, and explicit hyperparameter reporting make much of the work reproducible. The central scientific claims about data quality and the utility of transfer learning are credible in direction. However, the headline 'best multilingual PLM' claim is currently stronger than the evidence because of a domain confound and an incomplete comparison set, and the 'large-scale' wording overstates the size of the human-annotated datasets. These are fixable with rewording and additional controls.

major comments (4)
  1. [Section 5.3.1, Table 5.1; Section 5.6, Table 5.10] The claim that AfroXLMR 'produces the best performance compared to all other multilingual PLMs' is partly confounded with domain adaptation. The MAFT corpus in Table 5.1 is mostly mC4/Common Crawl augmented with news sources (VOA, BBC, Isolezwe, Alaroye/Asejere/Awikonko for Yoruba), cleaned only at line level, while two of the three headline evaluations (MasakhaNER and news-topic classification) are news-domain. Section 5.6 demonstrates that switching language adapters/SFT from Wikipedia to news-domain corpora improves zero-shot NER by +3.7 to +4.2 F1, so domain overlap is a large and known effect. The reported +2.6 (NER), +3.6 (news topic), and +1.5 (sentiment) average gains of MAFT over XLM-R-base therefore do not isolate language specialization. Please run a control with a non-news MAFT corpus (e.g., Wikipedia/JW300-only) or with news-matched and non-news-matched evaluation, and restrict the claim accordingly. For sentiment (Twitter), the domain mismatch is different, so the same caveat should be stated explicitly.
  2. [Abstract; Section 1.1; Tables 5.4-5.6] The abstract's 'large scale human-annotated labelled datasets' is inconsistent with Section 1.1, which describes 'small labelled datasets like 2k sentences for NER and 2k-5k parallel sentences for MT.' The contribution is scale in language coverage, not corpus size. In addition, the headline averages in Tables 5.4, 5.5, 5.6, and 5.8 are reported without standard deviations or significance tests even though the text says they are averages over five runs; several decisive comparisons are small (e.g., AfriBERTa with versus without MAFT on sentiment, +0.8 F1 in Table 5.6). Please add variance estimates and a significance statement, and align the 'large scale' wording with the actual dataset sizes.
  3. [Section 5.1; Tables 5.4-5.6] The claim that AfroXLMR outperforms 'all other multilingual PLMs' is not supported by the comparison set. The main tables compare only XLM-R-base/large, XLM-R-miniLM, and AfriBERTa; mBERT appears only in the cross-lingual transfer experiments, and other multilingual PLMs such as RemBERT, InfoXLM, and mT5, as well as later Africa-centric models, are absent. Please either expand the baselines to justify the universal wording or rephrase the claim to be relative to the evaluated models.
  4. [Section 3.4.1 and Table 3.4] Section 3.4.1 states that the XLM-R fine-tuning uses XLM-R-base for all training sizes, but Table 3.4 labels the 4000-sentence column as 'XLM-R large'. Please clarify which model was used for the N=4000 condition; if XLM-R-large was used, the comparison at that data size is not controlled and the conclusion that the PLM advantage shrinks with more training data would need to be re-stated.
minor comments (4)
  1. [Section 3.3.1, Table 3.3] The XLM-R row in Table 3.3 lists 13 African languages although the text (and Section 5.3) states that XLM-R covers eight African languages; please correct the list or the count.
  2. [Section 5.3.2.2, Table 5.2] The Somali ANTC entry states six classes but lists only five category labels; also Section 5.7 says the new news-topic corpus covers four African languages while Section 5.1 and Table 5.2 list five (Lingala, Naija, Somali, Malagasy, isiZulu). Please make these numbers consistent.
  3. [Section 4.5.2.3, Table 4.5] The text says 'fine-tuning the pre-trained multilingual-uncased BERT for 10 epochs on the NER task gives an F1 score of 27,' but Table 4.5 reports 27.1 for the pre-trained (not fine-tuned) model and 56.4 for the fine-tuned model; please correct the textual description to match the table.
  4. [Tables 2.4 and 2.5] The Somali rows in Tables 2.4 and 2.5 appear to have been copied from the Sesotho/Shona rows (identical consonant list; 'strongly suffixing' with noun classes absent is dubious for Somali, which is Cushitic), and the Kirundi row in Table 2.5 uses the code 'kin' also used for Kinyarwanda. Please verify these entries.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: this is empirical resource creation with external benchmarks and community-annotated datasets; the MAFT news-domain overlap is a confound, not a definitional reduction.

full rationale

The dissertation is primarily an empirical resource-creation effort. Chapter 4 builds curated word/contextual embeddings and evaluates them on translated wordsim-353 pairs and a natively annotated NER corpus; Chapter 5 performs multilingual adaptive fine-tuning on mC4/news corpora and compares against external PLMs (XLM-R, AfriBERTa, XLM-R-miniLM, XLM-R-large) on MasakhaNER, ANTC, and NaijaSenti; Chapters 6-10 release human-annotated NER/MT datasets and compare supervised, weakly supervised, and transfer baselines. No quantity is defined in terms of the quantity it is claimed to predict, and no fitted parameter is relabeled as a prediction. The main self-citations (Adelani et al. 2021b, 2022b; Alabi et al. 2022) point to publicly released, community-annotated datasets and model checkpoints that are externally falsifiable and have been used as benchmarks beyond this thesis; they are not invoked as an unverified uniqueness theorem or as the sole justification for a central choice. The one substantive limitation is that the MAFT adaptation text in Table 5.1 is largely mC4 plus news sources, while several downstream evaluations are news-domain, so part of the measured improvement could reflect domain adaptation rather than language specialization. That is an experimental confound and an interpretation risk, not a circular reduction: the reported gains are not forced by construction, and the chapter itself shows in Section 5.6 that domain overlap has a large effect, which makes the issue inspectable. The claim that AfroXLMR outperforms other multilingual PLMs is supported by tables with external baselines; any overstatement due to domain overlap is a correctness concern, not circularity.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The dissertation is empirical ML, so there are no invented physical entities and no formal axioms beyond standard ML practice. Free parameters are training and vocabulary choices, not fitted physical constants. The load-bearing premises are about data validity: wordsim translations, MLM transfer, and participatory annotation quality.

free parameters (2)
  • Vocabulary reduction top-k sizes = k_amh=52,000; k_other=60,000; merged vocabulary 70,609
    Hand-chosen thresholds for AfroXLMR-small vocabulary pruning in Section 5.5.1; they are coverage-based but not optimized per language and affect all downstream scores.
  • MAFT training schedule = 3 epochs, learning rate 5e-5, batch size 10 (or 32 for AfriBERTa)
    Fixed hyperparameters for multilingual adaptive fine-tuning; no tuning curve or sensitivity analysis is given, so their effect on the state-of-the-art claim is unknown.
assumptions (3)
  • domain assumption Cross-language wordsim-353 translations preserve similarity structure, so Spearman correlations measure embedding quality.
    Chapter 4 uses wordsim-353, originally English, translated and scored by native speakers for Yoruba and Twi; the validity of this cross-lingual transfer of scores is assumed rather than validated independently.
  • domain assumption Continued masked-language-model training on monolingual texts specializes a PLM for downstream tasks in those languages.
    Chapter 5's MAFT method relies on the MLM objective transferring to NER, topic classification, and sentiment; this is a standard empirical assumption, and no analysis of representation shift is provided.
  • domain assumption Participatory annotations from Masakhane are reliable gold labels for NER and MT evaluation.
    Chapters 7, 8, and 10 treat the community-labeled datasets as ground truth for F1 and BLEU; the provided text does not report inter-annotator agreement or adjudication statistics for these datasets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Natural language processing for African languages." pith.science (2026). https://pith.science/paper/42245LDJ

@misc{pith2026250700297,
  author       = {Pith},
  title        = {Pith review of: Natural language processing for African languages},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/42245LDJ}},
  note         = {Machine review of arXiv:2507.00297}
}
read the original abstract

Recent advances in word embeddings and language models use large-scale, unlabelled data and self-supervised learning to boost NLP performance. Multilingual models, often trained on web-sourced data like Wikipedia, face challenges: few low-resource languages are included, their data is often noisy, and lack of labeled datasets makes it hard to evaluate performance outside high-resource languages like English. In this dissertation, we focus on languages spoken in Sub-Saharan Africa where all the indigenous languages in this region can be regarded as low-resourced in terms of the availability of labelled data for NLP tasks and unlabelled data found on the web. We analyse the noise in the publicly available corpora, and curate a high-quality corpus, demonstrating that the quality of semantic representations learned in word embeddings does not only depend on the amount of data but on the quality of pre-training data. We demonstrate empirically the limitations of word embeddings, and the opportunities the multilingual pre-trained language model (PLM) offers especially for languages unseen during pre-training and low-resource scenarios. We further study how to adapt and specialize multilingual PLMs to unseen African languages using a small amount of monolingual texts. To address the under-representation of the African languages in NLP research, we developed large scale human-annotated labelled datasets for 21 African languages in two impactful NLP tasks: named entity recognition and machine translation. We conduct an extensive empirical evaluation using state-of-the-art methods across supervised, weakly-supervised, and transfer learning settings.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 7 canonical work pages

  1. [217]

    A Focus on Neural Machine Translation for African Languages

    doi: 10 . 18653 / v1 / 2021 . americasnlp - 1 . 23. url: https : //aclanthology.org/2021.americasnlp-1.23. Magnini, Bernardo et al. (May 2008). “Evaluation of Natural Language Tools for Italian: EVALITA 2007.” In: Proceedings of the Sixth Inter- national Conference on Language Resources and Evaluation (LREC’ 08). Marrakech, Morocco: European Language Reso...

    work page Pith review arXiv 2019

  2. [742]

    RoBERTa: A Robustly Optimized BERT Pretrain- ing Approach

    doi: 10.1162/tacl_a_00343. url: https://aclanthology.org/ 2020.tacl-1.47. Liu, Yinhan, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke S. Zettlemoyer, and Veselin Stoyanov (2019). “RoBERTa: A Robustly Optimized BERT Pretrain- ing Approach.” In: ArXiv abs/1907.11692. Liu, Zihan, Genta Indra Winata, and Pascale Fung ...

    arXiv 2019

  3. [1034]

    TinyBERT: Distilling BERT for Natural Language Understanding

    doi: 10.18653/v1/N18-1093 . url: https://www.aclweb.org/ anthology/N18-1093. Jiao, Xiaoqi, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu (Nov. 2020). “TinyBERT: Distilling BERT for Natural Language Understanding.” In: Findings of the Associa- tion for Computational Linguistics: EMNLP 2020. Online: Association for Comput...

    work page arXiv 2019

  4. [1060]

    TALEN: Tool for Annota- tion of Low-resource ENtities

    url: https://www.aclweb.org/anthology/K19-1060. Mayhew, Stephen and Dan Roth ( 2018). “TALEN: Tool for Annota- tion of Low-resource ENtities.” In: Proceedings of ACL 2018: System Demonstrations. url: http://aclweb.org/anthology/P18-4014. Mayhew, Stephen, Tatiana Tsygankova, and Dan Roth (Nov. 2019). “ner and pos when nothing is capitalized.” In: Proceedin...

    arXiv 2017

  5. [1077]

    Predicting Performance for Natu- ral Language Processing Tasks

    url: https://aclanthology.org/D19-1077. Xia, Mengzhou, Antonios Anastasopoulos, Ruochen Xu, Yiming Yang, and Graham Neubig (July 2020). “Predicting Performance for Natu- ral Language Processing Tasks.” In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Online: Associ- ation for Computational Linguistics, pp. 8625–...

    arXiv 2015

  6. [1201]

    DeBER- TaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing

    url: https://www.aclweb.org/anthology/D09-1124. He, Pengcheng, Jianfeng Gao, and Weizhu Chen ( 2021b). DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient- Disentangled Embedding Sharing. arXiv: 2111.09543 [cs.CL]. He, Pengcheng, Jianfeng Gao, and Weizhu Chen ( 2021a). “DeBER- TaV3: Improving DeBERTa using ELECTRA-Style Pre-Trainin...

    arXiv 2011

  7. [2018]

    Exploring Amharic Sentiment Analy- sis from Social Media Texts: Building Annotation Tools and Clas- sification Models

    url: https://www.aclweb.org/anthology/C18-1183. Yimam, Seid Muhie, Hizkiel Mitiku Alemayehu, Abinew Ayele, and Chris Biemann (Dec. 2020). “Exploring Amharic Sentiment Analy- sis from Social Media Texts: Building Annotation Tools and Clas- sification Models.” In: Proceedings of the 28th International Confer- ence on Computational Linguistics. Barcelona, Sp...

    doi:10.18653/v1/2020.coling-main.91 2020

  8. [2020]

    Online: Association for Computational Linguistics, pp. 2649–

    work page

Show all 16 references
  1. [2021]

    Online: Association for Computational Linguistics, pp. 2140–

  2. [2151]

    MiniLM: Deep Self-Attention Distillation for Task- Agnostic Compression of Pre-Trained Transformers

    doi: 10 . 18653 / v1 / 2021 . findings - acl . 188. url: https : //aclanthology.org/2021.findings-acl.188. Wang, Wenhui, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou (2020a). “MiniLM: Deep Self-Attention Distillation for Task- Agnostic Compression of Pre-Trained Tran...

    2020

  3. [2656]

    IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding

    doi: 10 . 18653 / v1 / 2020 . findings - emnlp . 240. url: https : //aclanthology.org/2020.findings-emnlp.240. Wilie, Bryan et al. (Dec. 2020). “IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding.” In: Proceedings of the 1st Conference of...

    2020 arXiv

  4. [2799]

    Universal Language Model Fine-tuning for Text Classification

    url: https : / / proceedings . mlr . press / v97 / houlsby19a . html. Howard, Jeremy and Sebastian Ruder ( 2018). “Universal Language Model Fine-tuning for Text Classification.” In: Proceedings of ACL 2018. Hu, Edward J, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi L...

    2022 arXiv

  5. [5446]

    Learning with Noisy Labels for Sentence-level Sentiment Classifi- cation

    url: https://aclanthology.org/W18-5446. Wang, Hao, Bing Liu, Chaozhuo Li, Yan Yang, and Tianrui Li (2019). “Learning with Noisy Labels for Sentence-level Sentiment Classifi- cation.” In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and...

    2019 doi

  6. [5903]

    Named- entity recognition for a low-resource language using pre-trained language model

    doi: 10.3390/fi13110275. url: https://www.mdpi.com/1999- 5903/13/11/275. Yohannes, Hailemariam Mehari and Toshiyuki Amagasa (2022). “Named- entity recognition for a low-resource language using pre-trained language model.” In: Proceedings of the 37th ACM/SIGAPP Symposium on App...

    2022 doi

  7. [6312]

    LLaMA: Open and Efficient Foundation Language Models

    url: https://aclanthology.org/W18-6312. Touvron, Hugo et al. (2023a). “LLaMA: Open and Efficient Foundation Language Models.” In: ArXiv abs/2302.13971. url: https://api. semanticscholar.org/CorpusID:257219404. Touvron, Hugo et al. (2023b). “Llama 2: Open Foundation and Fine- T...

    2023 arXiv

  8. [7032]

    Grammaticalization in Nigerian Pidgin

    isbn: 979-10-95546-34-4. url: https : / / aclanthology . org / 2020.lrec-1.868. Offiong Mensah, Eyo (2012). “Grammaticalization in Nigerian Pidgin.” In: Íkala, revista de lenguaje y cultura 17.2, pp. 167–179. Ogayo, Perez, Graham Neubig, and Alan W. Black ( 2022). “Building Af...

    2012 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.