REVIEW 5 major objections 5 minor 14 references
Yankari: A Monolingual Yoruba Dataset
T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read This paper introduces Yankari, a 51,407-document, 30-million-token monolingual Yoruba corpus from 13 non-religious web sources, and claims it is the first large-scale, ethically curated resource of its kind.
desk verdict Useful new Yoruba corpus, but the paper promises more evidence than it delivers; the quality claim needs external checks or clearer criteria. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism that carries the argument is the curation pipeline, not any one algorithm. Source selection chooses 13 publicly accessible domains with an eye to news, blogs, culture, sports, and general knowledge; HTML parsing converts pages to structured text; exact document matching plus MinHash locality-sensitive hashing, a hashing technique that efficiently finds nearly identical text blocks, removes duplicate paragraphs; and a manual native-speaker review, performed by the author, excludes machine-translated, offensive, or non-Yoruba content. Each stage targets a failure documented in prior corpora: religious over-representation, duplication, machine translation, and the legal and ethical restrictions of JW300. The pipeline is what makes Yankari's claimed quality a property of the dataset's construction rather than a matter of luck.
What would settle it
An independent audit would settle it: take a random sample of Yankari documents, have at least two native Yoruba speakers who did not build the corpus label each for machine-translation artifacts, non-Yoruba content, and duplicates, and compare their labels with an automatic duplicate detector; if the flagged share approaches the 18% repetition or 24.48% duplication figures reported for Wura or CulturaX's Yoruba subset, or if annotators disagree substantially, the paper's quality claim is refuted.
Extended reading notes
Core claim
The paper's core claim is that Yankari is the first large-scale monolingual Yoruba dataset built without the religious skew that dominates prior resources: 51,407 documents, 30,438,702 tokens, an average of 592 tokens per document, across 13 sources led by yo.wikipedia.org, alaroye.org, and BBC Yoruba. The author asserts that by excluding JW300-derived and other restricted content, filtering out machine-translated pages through manual native-speaker review, and removing exact and near-duplicate paragraphs with MinHash and LSH, Yankari supplies a cleaner and more representative sample of written Yoruba. As evidence, the paper reports corpus statistics, domain distribution, and a quality audit of the existing Wura dataset showing 18.01% repetition of a single word, only 45% unique entries after cleaning, and formatting and language errors. The intended consequence is that Yankari can support language modeling, machine translation, text classification, and comparative linguistic studies without the ethical and legal problems tied to religious corpora.
Load-bearing premise
The dataset's quality rests on the author's own manual review of documents for machine-translated or inappropriate content, with no published criteria, no second reviewer, and no independent check; if that review is inconsistent, Yankari's advantage over earlier Yoruba corpora is not established.
Editorial extensions
If this is right
- If Yankari is as clean and diverse as claimed, Yoruba NLP gains a training corpus that is not dominated by religious text, which should improve natural language generation and text classification.
- Researchers can build on Yankari without the legal and ethical exposure that comes from JW300-derived content, because restricted sources were explicitly excluded.
- The size and domain spread make Yankari a credible base for language modeling and for comparative studies of written Yoruba against other corpora.
- The documented pipeline, including deduplication and manual filtering steps, provides a template for creating similar ethically sourced datasets for other low-resource languages.
- The paper's planned downstream evaluations in language modeling, machine translation, and classification would directly test whether curation translates into better model performance.
Reading between the lines
- Because every source is a website, Yankari will still under-represent spoken Yoruba, informal text-speak, and regional orthographic variants; models trained only on it may miss everyday conversational usage, an extension of the representation biases the paper itself lists.
- The per-document URL and source metadata make Yankari a natural seed for benchmark tasks: holding out one domain and testing transfer across the other twelve would measure how domain diversity affects generalization.
- The paper's comparison with Wura suggests an independent quantitative check: running the same repetition and duplication analysis on Yankari and publishing the numbers would let the non-religious, low-duplication claim be verified without trusting the curation process.
- To make the pipeline reproducible for other languages, the manual exclusion step would need written guidelines and inter-annotator agreement measures; without them, the method cannot be cleanly transplanted.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Yankari, a monolingual Yoruba dataset of 51,407 documents and 30,438,702 tokens collected from 13 web sources. The methodology includes HTML parsing, exact and near-duplicate deduplication, manual filtering to exclude machine-translated and inappropriate content, and an ethical stance against using restricted sources such as JW300. The authors claim Yankari is the first large-scale, non-religious-domain monolingual resource for Yoruba and argue that it addresses quality and ethical problems in existing corpora such as Wura and Yorùbá Text C3. The Abstract promises 'thorough automated evaluations' and comparisons with existing resources, but the body contains no such evaluations: §4.8 reports only manual spot checks, and §6 defers downstream evaluation to future work. The paper is transparent about several limitations, but its central claims of cleanliness, authenticity, and superiority over prior resources are not supported by the evidence presented.
Significance. If the quality and authenticity claims were verified, Yankari would be a valuable addition to Yoruba NLP resources, particularly because it deliberately avoids religious and machine-translated content and provides a source-attributed, JSONL-formatted corpus. The paper's ethical motivation, including the critique of JW300 usage, is a strength, and the acknowledgment of limitations such as internet bias and diacritization issues is commendable. However, the manuscript currently provides no quantitative evaluation, no reproducible audit trail for the manual filtering, no code for the Wura analysis, and no versioned dataset identifier. The resource may be useful, but the paper does not yet meet the evidentiary standard for its own claims.
major comments (5)
- [Abstract; §4.8; §6] The Abstract states that the paper provides 'thorough automated evaluations of the dataset, demonstrating its quality compared to existing resources,' but the body does not contain these evaluations. §4.8 describes only native-speaker involvement and spot checks, §4.9 lists qualitative limitations, and §6 explicitly defers downstream evaluation to future work. No quality metrics, no comparisons with Wura or other Yoruba corpora, and no language-model or probing experiments are reported. Because the central claim of the paper is that Yankari is clean, diverse, and superior to existing resources, this missing evidence is load-bearing. The authors should either add the promised automated evaluations or revise the Abstract and Section 1 to reflect what the paper actually provides.
- [§4.6] The exclusion of machine-translated and inappropriate content is described as 'primarily a manual process conducted by the author, a native Yoruba speaker,' with no explicit exclusion criteria, no inter-annotator agreement, no audit trail, and no examples of excluded documents. Since the paper's value proposition rests on authenticity and quality—'avoiding ... machine-translated content' and 'rigorous quality control'—this procedure must be specified to a reproducible degree. The authors should state the concrete criteria used to identify 'unnatural phrasing' and 'translation artifacts,' report the number of documents removed at each filtering step, and provide a sample of excluded items. Without this, the quality claim cannot be independently checked.
- [§4.2] The analysis of the Wura dataset reports specific figures (18.01% of documents contain 'asteroidi', 17,103 unique entries after cleaning, and 45% of the original dataset remaining) but provides no methodology, code, or scripts for how these numbers were obtained. This analysis is used to motivate Yankari's curation approach, so the figures need to be reproducible. Please specify the deduplication method, the cleaning steps, and the exact computation behind each percentage, or release the analysis scripts.
- [§4.3.2 and §4.6] The pipeline description omits the values of the free parameters that determine the dataset's composition: the MinHash/LSH similarity threshold for near-duplicate paragraph removal and the threshold for removing 'very short texts' are never specified. Because these thresholds directly affect the reported 51,407 documents and 30,438,702 tokens, their absence makes the dataset statistics non-reproducible even with access to the same sources. Please state the threshold values and the heuristics used for short-text removal.
- [Dataset availability; §4.7] The dataset is released via a Hugging Face URL, but the paper provides no version identifier, commit hash, or data card describing fields, license, and provenance. Since the paper does not include the full source URL list or processing code, an independent audit cannot confirm that the published artifact matches the described statistics. A persistent versioned identifier and a datasheet should be provided.
minor comments (5)
- [Title; §1] There are inconsistent orthographic renderings of 'Yoruba' (e.g., 'Y oruba' in the title and 'Yorùbá' in §2.1.1); please normalize the orthography throughout the manuscript.
- [Table 1] The numeric columns in Table 1 are not aligned and the 'Total' row uses a space separator; the formatting should be cleaned for readability.
- [§4.7; Dataset link] The sample JSON entry contains unusual spacing in the text field ('O ma s e o ! Ijamba oko o f u r u f u gba emi eeyan marun - un') and the dataset URL in the main text contains a space ('Y ANKARI'); these appear to be rendering artifacts that should be fixed.
- [References] Several references lack full publication details or consistent formatting (e.g., Adelani et al. 2021, Alabi et al. 2020); please harmonize the reference list and add stable identifiers where available.
- [§3; §4.5] The claim of 'first large-scale, non-religious domain monolingual resource' should be qualified, since Wura contains a larger number of Yoruba documents (approximately 68,000), even if it is multilingual and religious-heavy; clarify the comparison criteria. In addition, the 'diverse sources' claim is weakened by the fact that the top three domains account for about 69% of documents; consider reporting an effective diversity metric or discussing this skew.
Circularity Check
No significant circularity: the dataset paper makes empirical claims about a released artifact; no derivation reduces to its own inputs.
full rationale
Yankari is a dataset-description paper, not a derivation or prediction pipeline. There are no equations, no fitted parameters, and no quantity that is defined in terms of another quantity it is then said to predict. The core claims are empirical assertions about an externally released Hugging Face artifact: document counts, token counts, source diversity, and the absence of religious or machine-translated content. These claims are checkable against the published dataset by any third party. The closest concern is that the quality filtering in §4.6 was a manual process performed by the author, a native Yoruba speaker, with no inter-annotator agreement or external audit, and that the abstract's promise of 'thorough automated evaluations' is not reflected in the body, which describes only spot checks and manual review. That is a missing-evidence or rigor problem, not a circularity problem: the manuscript does not define quality as the author's own judgment in a way that makes the dataset's value equivalent to its input. There are no self-citations carrying the argument, no imported uniqueness theorem, and no known result renamed as a new contribution. Under the rule that circularity requires a quoted reduction of the claimed result to its own inputs, no such reduction is present, so the score is 0.
Assumptions & free parameters
free parameters (2)
- MinHash/LSH similarity threshold
- Short-text removal threshold
assumptions (3)
- domain assumption Web-scraped written Yoruba text from the listed 13 sources is a sufficiently representative sample of contemporary Yoruba for NLP.
- domain assumption The author's native-speaker manual filtering reliably distinguishes natural Yoruba from machine-translated or inappropriate content.
- ad hoc to paper The published Hugging Face dataset matches the statistics and content described in the paper.
Cite this review
Pith. "Pith review of Yankari: A Monolingual Yoruba Dataset." pith.science (2026). https://pith.science/paper/CX6QJDOS
@misc{pith2026241203334,
author = {Pith},
title = {Pith review of: Yankari: A Monolingual Yoruba Dataset},
year = {2026},
howpublished = {\url{https://pith.science/paper/CX6QJDOS}},
note = {Machine review of arXiv:2412.03334}
}
read the original abstract
This paper presents Yankari, a large-scale monolingual dataset for the Yoruba language, aimed at addressing the critical gap in Natural Language Processing (NLP) resources for this important West African language. Despite being spoken by over 30 million people, Yoruba has been severely underrepresented in NLP research and applications. We detail our methodology for creating this dataset, which includes careful source selection, automated quality control, and rigorous data cleaning processes. The Yankari dataset comprises 51,407 documents from 13 diverse sources, totaling over 30 million tokens. Our approach focuses on ethical data collection practices, avoiding problematic sources and addressing issues prevalent in existing datasets. We provide thorough automated evaluations of the dataset, demonstrating its quality compared to existing resources. The Yankari dataset represents a significant advancement in Yoruba language resources, providing a foundation for developing more accurate NLP models, supporting comparative linguistic studies, and contributing to the digital accessibility of the Yoruba language.
Reference graph
Works this paper leans on
-
[1]
online" 'onlinestring :=
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Ife Adebara, AbdelRahim Elmadany, and Muhammad Abdul-Mageed. 2024. https://arxiv.org/abs/2401.01053 Cheetah: Natural language generation for 517 african languages . Preprint, arXiv:2401.01053
arXiv 2024
-
[4]
David Adelani et al. 2021. Menyo-20k: A multi-domain english - yoruba corpus for machine translation and domain adaptation. In Proceedings of the Fourth Workshop on Technologies for MT of Low Resource Languages
work page 2021
-
[5]
Z eljko Agi \'c and Ivan Vuli \'c . 2019. https://doi.org/10.18653/v1/P19-1310 JW 300: A wide-coverage parallel corpus for low-resource languages . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3204--3210, Florence, Italy. Association for Computational Linguistics
-
[6]
Jesujoba Alabi et al. 2020. Massive vs. curated embeddings for low-resourced languages: The case of yorùbá and twi. In Proceedings of the 12th Language Resources and Evaluation Conference
work page 2020
-
[7]
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm \'a n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440--8451
work page 2020
-
[8]
Danny Hernandez, Tom Brown, Tom Conerly, Nova DasSarma, Dawn Drain, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Tom Henighan, Tristan Hume, Scott Johnston, Ben Mann, Chris Olah, Catherine Olsson, Dario Amodei, Nicholas Joseph, Jared Kaplan, and Sam McCandlish. 2022. https://arxiv.org/abs/2205.10487 Scaling laws and interpretability of learning from...
arXiv 2022
Show all 14 references
-
[9]
Ben Hutchinson. 2024. Modeling the sacred: Considerations when using religious texts in natural language processing. arXiv preprint arXiv:2404.14740
2024 arXiv
-
[10]
Duc Minh Nguyen et al. 2023. Culturax: A cleaned, enormous, and multilingual dataset for large language models in 167 languages. arXiv preprint arXiv:2309.09400
2023 arXiv
-
[11]
Kelechi Ogueji et al. 2021. Small data? no problem! exploring the viability of pretrained multilingual language models for low-resourced languages. arXiv preprint arXiv:2011.03823
2021 arXiv
-
[12]
Adewale Oladipo et al. 2023. Wura: A multilingual dataset of african languages. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics
2023
-
[13]
Pedro Javier Ortiz Su \'a rez, Laurent Romary, and Beno \^i t Sagot. 2020. A monolingual approach to contextualized word embeddings for mid-resource languages. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1703--1714
2020
-
[14]
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mt5: A massively multilingual pre-trained text-to-text transformer. arXiv preprint arXiv:2010.11934
2021 arXiv
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.