REVIEW 4 major objections 5 minor 1 cited by
Data Augmentation With Back translation for Low Resource languages: A case of English and Luganda
T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read Selective back translation lifts English–Luganda BLEU by more than 10 points, the paper reports.
desk verdict Useful data effort and a plausible internal result, but the +10 BLEU claim over prior work is not established and the selection step risks overfitting to the test set. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the paper's 'OurBT' algorithm: starting from a bilingual baseline model, it translates each small monolingual dataset in both directions, retrains a model on each resulting synthetic set, ranks the datasets by the BLEU score of the retrained models, and then retrains the final model on synthetic translations of the top-ranked monolingual sets. Iterating this cycle implements iterative back translation; adding monolingual data in growing portions implements incremental back translation. The mechanism acts as a selection filter: it scores fragmented monolingual corpora by how much they help the translation task, and the paper finds that the best-scoring sets are those with contextual similarity to the training data.
What would settle it
Run the earlier back-translation system [3] on this paper's 'newtest' sentences and compute its BLEU; if it scores within 10 BLEU points of 40.25 (Eng2Lug) or 44.25 (Lug2Eng), the central comparative claim is not supported. The mirror-image check is to run this paper's final model on the earlier work's test set and see whether the gap survives.
Extended reading notes
Core claim
The paper's central discovery claim is that a back-translation routine that selects the best-performing small monolingual datasets produces large gains for English–Luganda: BLEU rises from 29.67 to 40.25 for English-to-Luganda and from 32.92 to 44.25 for Luganda-to-English on the newtest split, more than 10 points above the earlier back-translation scores of 26.7 and 33.2 [3]. The paper also claims that the improvement is not an artifact of BLEU alone, since ChrF2 rises and TER falls in both directions, and sample translations align with reference translations better than a commercial online translator. It attributes the gains to selecting datasets whose context matches the training data rather than to simply adding more monolingual text.
Load-bearing premise
The headline claim assumes that BLEU scores on the newly built 'newtest' set can be compared directly with earlier reported BLEU scores obtained on different test sets; if the test sets differ, the claimed 10-point lead is not established.
Editorial extensions
If this is right
- English–Luganda translation can reach BLEU scores of 40.25 and 44.25 on the newtest split using back translation over small, selected monolingual datasets, a level that would make the pair substantially more usable.
- Dataset selection matters: applying standard back translation to the best dataset combination gives only +2.49 BLEU for Eng2Lug and +0.45 for Lug2Eng, while the selection-then-retrain routine gives +6.27 and +7.05 before iteration.
- Repeating the selection-retrain cycle improves scores in each of three reported iterations, suggesting the routine can be run until convergence rather than fixed at a single pass.
- The publicly released bilingual and monolingual datasets give other researchers a common resource for English–Luganda NMT, addressing the previous lack of public data for this pair.
- Evaluation with ChrF2 and TER gives a fuller picture for a morphologically rich language: TER falls from 69.7 to 65.4 for Eng2Lug and from 64.7 to 60.4 for Lug2Eng, consistent with the BLEU gains.
Reading between the lines
- The 10-point margin over prior work is only meaningful if the test sets are comparable; a head-to-head evaluation on a shared held-out set could confirm or shrink the gap.
- The fact that context-matching datasets win selection suggests back translation here is doing implicit domain adaptation, so a cheaper selector could rank monolingual datasets by domain similarity without retraining a full model per candidate.
- The recipe—small monolingual corpora from varied sources, scored one at a time—should transfer to other Bantu and low-resource languages with fragmented data; testing it on Swahili, Acholi, or Runyankore would show whether the selection step generalises.
- The large gap between the default validation scores and the newtest scores (for example, 53.77 versus 29.67 for Eng2Lug) shows how much test-set choice affects apparent quality; future benchmarks for this language pair should settle on a shared public test split to make comparisons meaningful.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies back translation (BT) for English–Luganda neural machine translation. It assembles bilingual data from four sources and monolingual data for both languages, including web-crawled news text, trains Transformer models, and applies standard, incremental, iterative, and a proposed 'OurBT' variant. OurBT (Algorithm 1) selects the best monolingual dataset combination and model based on BLEU score. On an author-constructed 'newtest' set, iterative BT raises BLEU from 29.67 to 40.25 (Eng2Lug) and from 32.92 to 44.25 (Lug2Eng), with TER decreasing. The abstract and Section 5 claim that this exceeds previous benchmarks by more than 10 BLEU units.
Significance. If the internal comparison is taken at face value, the paper provides a useful demonstration that iterative and incremental back translation can improve a very low-resource language pair, and it has several commendable practices: reporting SacreBLEU, ChrF2, and TER; documenting data sources in detail; and providing human-readable sample translations. The strongest contribution would be the curated and web-crawled datasets, although they are not currently released. The headline comparison with previous benchmarks is not established, and the selection protocol raises evaluation-bias concerns, so the paper's significance as a benchmark claim is currently limited.
major comments (4)
- [Abstract and Section 5] The claim that the authors exceed previous benchmarks by more than 10 BLEU units compares newtest scores (40.25 Eng2Lug, 44.25 Lug2Eng, Tables 6 and 7) with scores of 26.7 and 33.2 from [3], but there is no evidence that these numbers were computed on the same test set, with the same tokenization, or with the same BLEU implementation. The newtest set is new and unreleased, so the comparison is uncontrolled; either replace it with a comparison on a shared, released test set or remove the benchmark claim from the abstract.
- [Algorithm 1 and Section 4.3] The selection step in Algorithm 1 (lines 10–11) chooses monolingual dataset combinations and models 'based on the BLEU score,' and the paper then reports final BLEU scores on the same newtest set used to demonstrate the gains. If the same split was used for selection and final evaluation, the reported improvements are optimistically biased. The authors should state explicitly which split SelectBasedOnBLEU uses; ideally, selection should be performed on a held-out validation set, with the newtest set used only once for final reporting.
- [Section 4.3 and Section 4.1.1] The newtest set is described as 'focused on news from the local context' and is used to evaluate models whose monolingual training data include web-crawled material from the same news sources (Tables 2 and 3). Without a deduplication or overlap analysis, the gains may partly reflect domain match rather than the back-translation method itself. Please report the overlap between newtest and all training and monolingual sources and, if possible, evaluate on an independent out-of-domain test set.
- [Section 4.1 and Tables 6–7] The word 'significant' in the abstract is not supported by significance testing or variance estimates; all results appear to come from a single training run per condition. At minimum, the authors should report multiple seeds with standard deviations, or temper the wording of the claim.
minor comments (5)
- [Throughout] There are several typographical errors: 'la nguages' in the title, 'BLUE' for BLEU in Section 4.3, 'SacreBLE U' for SacreBLEU in Section 3.3, and 'reproducability' in Section 1.
- [Section 4.2] The large jump from 39.05 to 53.77 BLEU after removing the Bible text deserves a sentence of explanation; as written, it is surprising and could indicate an unexpected interaction between the default T/V split and the Bible-domain training data.
- [Section 1 and Section 4.3] The paper states that releasing datasets would aid reproducibility, but no URL or repository is provided for newtest or the crawled corpora; please include them or state clearly that they are not yet available.
- [References] References [29] and [30] appear to describe the same arXiv paper; they should be consolidated to avoid duplication.
- [Additional material] The arXiv source includes a 'sample-franklin.png' figure that is not referenced in the text; it should be removed or cited.
Circularity Check
OurBT selects monolingual datasets and models by BLEU on the same 'newtest' split that is later reported, so the reported gains are partly the selection objective rather than an independent prediction.
-
fitted input called prediction
[Section 3.2.2 (Algorithm 1, lines 10-11); Section 4.3 (Tables 6-7)]
"Based on the BLEU score, OurBT identifies the most relevant datasets ... and models ... // Using the newtest dataset and applying our approach (OurBT) improved the model performance."
Algorithm 1 chooses the best monolingual corpus combination and the best intermediate models 'Based on the BLEU score.' Section 4.3 then reports the BLEU scores of exactly those selected configurations on the newtest split, presenting the gains (6.27 Eng2Lug, 7.05 Lug2Eng) as evidence that OurBT improves translation. If the BLEU used for selection is computed on the same newtest set that is later reported—and the paper does not state that a separate validation split was used—the reported improvement is the maximization objective, not an independent estimate.
full rationale
The concrete circular step is the model/dataset selection on the evaluation metric: the same BLEU objective that picks the best combination is later reported as the improvement. Apart from this, the paper is largely self-contained: back translation is an established external technique, the Transformer and BPE are standard components, and no load-bearing uniqueness theorem or self-citation chain is invoked. The headline comparison to prior work (+10 BLEU over [3]) is a separate validity problem—the newtest set is not shared with [3], so the scores are not directly comparable—but that is a comparability issue, not a circular derivation. Because the central 'OurBT improves BLEU' result is partly the same BLEU objective used in selection, the circularity score is 6 rather than 0-2.
Assumptions & free parameters
free parameters (3)
- Best Luganda monolingual dataset combination for Eng2Lug =
Mozilla Common Voice, Makerere Text and Speech, YouTube news headlines
- Best English monolingual dataset combination for Lug2Eng =
Digital Umuganda, Gamayun MiniKit, Chimpreports news
- Number of iterative BT steps =
3 to 4 iterations until convergence
assumptions (3)
- domain assumption Back translation of monolingual data improves NMT quality for low-resource pairs
- domain assumption BLEU, ChrF2, and TER measured on the newtest split are valid and comparable measures of translation quality
- domain assumption The newtest set is representative of the evaluation conditions under which prior benchmarks were reported
Cite this review
Pith. "Pith review of Data Augmentation With Back translation for Low Resource languages: A case of English and Luganda." pith.science (2026). https://pith.science/paper/53W7IJKY
@misc{pith2026250502463,
author = {Pith},
title = {Pith review of: Data Augmentation With Back translation for Low Resource languages: A case of English and Luganda},
year = {2026},
howpublished = {\url{https://pith.science/paper/53W7IJKY}},
note = {Machine review of arXiv:2505.02463}
}
read the original abstract
In this paper,we explore the application of Back translation (BT) as a semi-supervised technique to enhance Neural Machine Translation(NMT) models for the English-Luganda language pair, specifically addressing the challenges faced by low-resource languages. The purpose of our study is to demonstrate how BT can mitigate the scarcity of bilingual data by generating synthetic data from monolingual corpora. Our methodology involves developing custom NMT models using both publicly available and web-crawled data, and applying Iterative and Incremental Back translation techniques. We strategically select datasets for incremental back translation across multiple small datasets, which is a novel element of our approach. The results of our study show significant improvements, with translation performance for the English-Luganda pair exceeding previous benchmarks by more than 10 BLEU score units across all translation directions. Additionally, our evaluation incorporates comprehensive assessment metrics such as SacreBLEU, ChrF2, and TER, providing a nuanced understanding of translation quality. The conclusion drawn from our research confirms the efficacy of BT when strategically curated datasets are utilized, establishing new performance benchmarks and demonstrating the potential of BT in enhancing NMT models for low-resource languages.
Forward citations
Cited by 1 Pith paper
-
The Saturation Point of Backtranslation in High Quality Low Resource English Gujarati Machine Translation
Backtranslated Gujarati-English data fails to improve a strong MBART50 baseline trained on 50k parallel pairs, slightly reducing BLEU from 43.8 to 43.0.
Reference graph
Works this paper leans on
-
[3]
Benjamin Akera, Jonathan Mukiibi, Lydia Sanyu Naggayi, Claire Babirye, Isaac Owomugisha, Solomon Nsumba, Joyce Na katumba-Nabende, En- gineer Bainomugisha, Ernest Mwebaze, and John Quinn. 2022. Machine translation for african languages: Community crea tion of datasets and models in uganda. In 3rd Workshop on African Natural Language Processing
work page 2022
-
[1]
David Ifeoluwa Adelani, Marek Masiak, Israel Abebe Azim e, Jesujoba Oluwadara Alabi, Atnafu Lambebo Tonja, Christi ne Mwase, Odunayo Ogun- depo, Bonaventure FP Dossou, Akintunde Oladipo, Doreen Nix dorf, et al. 2023. Masakhanews: News topic classification fo r african languages. arXiv preprint arXiv:2304.09972 (2023)
arXiv 2023
-
[2]
Alham Fikri Aji and Kenneth Heafield. 2020. Fully synthet ic data improves neural machine translation with knowledge distillation. arXiv preprint arXiv:2012.15455 (2020)
work page Pith review arXiv 2020
-
[4]
Antonios Anastasopoulos, Alessandro Cattelan, Zi-Yi D ou, Marcello Federico, Christian Federman, Dmitriy Genzel , Francisco Guzmán, Junjie Hu, Macduff Hughes, Philipp Koehn, et al. 2020. TICO-19: the tran slation initiative for COvid-19. arXiv preprint arXiv:2007.01788 (2020)
arXiv 2020
-
[5]
Claire Babirye, Joyce Nakatumba-Nabende, Andrew Katum ba, Ronald Ogwang, Jeremy Tusubira Francis, Jonathan Mukii bi, Medadi Ssentanda, Lilian D Wanzare, and Davis David. 2022. Building text and sp eech datasets for low resourced languages: A case of languag es in east Africa. (2022)
work page 2022
-
[6]
Claire Babirye, Jeremy Tusubira, Jonathan Mukiibi, Joy ce Nakatumba-Nabende, and Andrew Katumba. 2023. Sentiment Tagged Parallel Corpus for Luganda and Swahili. https://doi.org/10.7910/DVN/XSGIKR
-
[7]
Elizabeth Baertlein and Martin Ssekitto. 2014. Luganda Nouns: Inflectional Morphology and Tests. Linguistic Portfolios 3, 1 (2014), 3. 10 Data Augmentation With Back translation for Low Resource lan guages: A case of English and Luganda NLPIR 2024, December 13–15,2024, Okayama, Japan
work page 2014
-
[8]
Ankur Bapna, Isaac Caswell, Julia Kreutzer, Orhan Firat , Daan van Esch, Aditya Siddhant, Mengmeng Niu, Pallavi Baljekar, Xavier Garcia, Wolfgang Macherey, et al. 2022. Building machine translation system s for the next thousand languages. arXiv preprint arXiv:2205.03983 (2022)
arXiv 2022
Show all 36 references
-
[9]
Shweta Chauhan, Shefali Saxena, and Philemon Daniel. 20 22. Improved unsupervised neural machine translation with semantically weighted back translation for morphologically rich and low resource lang uages. Neural Processing Letters 54, 3 (2022), 1707–1726
2022
-
[10]
Yong Cheng and Yong Cheng. 2019. Joint training for pivo t-based neural machine translation. Joint training for neural machine translation (2019), 41–54
2019
-
[11]
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vis hrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised cro ss-lingual representation learning at scale. arXiv preprint arXiv:1911.02116 (2019)
2019 arXiv
-
[12]
Sergey Edunov, Myle Ott, Michael Auli, and David Grangi er. 2018. Understanding back-translation at scale. arXiv preprint arXiv:1808.09381 (2018)
2018 arXiv
-
[13]
Turkic Morph eme
Ayrat Gatiatullin, Lenara Kubedinova, Nikolai Prokop yev, and Abduramanov Ibraim. 2022. Toolset of “Turkic Morph eme” Portal for Creation of Electronic Corpora of Turkic Languages in a Unified Conceptu al Space. In 2022 7th International Conference on Computer Science and E ngi...
2022
-
[14]
Catherine Gitau and Vukosi Marivate. 2023. Textual Aug mentation Techniques Applied to Low Resource Machine Trans lation: Case of Swahili. arXiv preprint arXiv:2306.07414 (2023)
2023 arXiv
-
[15]
Unmesh Gundecha. 2015. Selenium Testing Tools Cookbook. Packt Publishing Ltd
2015
-
[16]
Krupakar Hans and RS Milton. 2016. Improving the perfor mance of neural machine translation involving morphologic ally rich languages. arXiv preprint arXiv:1612.02482 (2016)
2016 arXiv
-
[17]
Cong Duy Vu Hoang, Philipp Koehn, Gholamreza Haffari, an d Trevor Cohn. 2018. Iterative back-translation for neural machine translation. In 2nd Workshop on Neural Machine Translation and Generation . Association for Computational Linguistics, 18–24
2018
-
[18]
Imran Khan Mohd Jais, Amelia Ritahani Ismail, and Syed Q amrun Nisa. 2019. Adam optimization algorithm for wide and d eep neural network. Knowl. Eng. Data Sci. 2, 1 (2019), 41–46
2019
-
[19]
Chang Jin, Shigui Qiu, Nini Xiao, and Hao Jia. 2022. AdMi x: A mixed sample data augmentation method for neural machin e translation. arXiv preprint arXiv:2205.04686 (2022)
2022 arXiv
-
[20]
Vanlalmuansangi Khenglawt, Sahinur Rahman Laskar, Pa rtha Pakray, and Ajoy Kumar Khan. [n. d.]. Addressing data sc arcity issue for English– Mizo neural machine translation using data augmentation an d language model. Journal of Intelligent & Fuzzy Systems Preprint ([n. d.]), 1–11
-
[21]
Richard Kimera, Daniela N Rim, and Heeyoul Choi. 2022. B uilding a Parallel Corpus and Training Translation Models B etween Luganda and English. Information Science Journal 49, 11 (2022), 1009–1016
2022
-
[22]
Annie Lamar and Zeyneb Kaya. 2023. Measuring the Impact of Data Augmentation Methods for Extremely Low-Resource NM T. In Proceedings of the The Sixth Workshop on Technologies for Machine Translat ion of Low-Resource Languages (LoResMT 2023) . 101–109
2023
-
[23]
Maxwell Mojapelo and Jan Buys. 2023. Data augmentation for low resource neural machine translation for sotho-tswa na languages. (2023)
2023
-
[24]
Jonathan Mukiibi, Claire Babirye, Jeremy Tusubira, To bias Bateesa, Eric Peter Wairagala, Chodrine Mutebi, Joyce Nakatumba-Nabende, Andrew Katumba, Ivan Ssenkungu, and Medadi Sentanda. 2023. Lugand a Monolingual Corpus. https://doi.org/10.7910/DVN/EQOWTW
2023 doi
-
[25]
Jonathan Mukiibi, Andrew Katumba, Joyce Nakatumba-Na bende, Ali Hussein, and Josh Meyer. 2022. The makerere radiospeech corpus: A Luganda radio corpus for automatic speech recognition. arXiv preprint arXiv:2206.09790 (2022)
2022 arXiv
-
[26]
Seokjin Oh, Woohwan Jung, et al. 2023. Data augmentatio n for neural machine translation using generative language model. arXiv preprint arXiv:2307.16833 (2023)
2023 arXiv
-
[27]
Daniela N Rim, Richard Kimera, and Heeyoul Choi. 2023. M ini-Batching with Similar-Length Sentences to Quickly Train NMT Models. Information Science Journal 50, 7 (2023), 614–620
2023
-
[28]
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015 . Improving neural machine translation models with monolin gual data. arXiv preprint arXiv:1511.06709 (2015)
2015 arXiv
-
[30]
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016 . Neural Machine Translation of Rare Words with Subword Units. arXiv: 1508.07909 [cs.CL]
2016 arXiv
-
[31]
AM Ssentumbwe, BM Kim, and HA Lee. 2019. English to Lugan da SMT: Ganda Noun Class Prefix Segmentation for Enriched Mac hine Translation [J]. International Journal of Advanced Trends in Computer Scien ce and Engineering 8, 5 (2019), 1861–1868
2019
-
[32]
Amane Sugiyama and Naoki Yoshinaga. 2019. Data augment ation using back-translation for context-aware neural machine translation. In Proceed- ings of the fourth workshop on discourse in machine translat ion (DiscoMT 2019) . 35–44
2019
-
[33]
Herry Sujaini, Samuel Cahyawijaya, and Arif B Putra. 20 23. Analysis of Language Model Role in Improving Machine Tra nslation Accuracy for Extremely Low Resource Languages. Journal of Advances in Information Technology 14, 5 (2023)
2023
-
[34]
Mengtao Sun, Hao Wang, Mark Pasquine, and Ibrahim A. Ham eed. 2021. Machine translation in low-resource languages b y an adversarial neural network. Applied Sciences 11, 22 (2021), 10860
2021
-
[35]
Jörg Tiedemann and Santhosh Thottingal. 2020. OPUS-MT –building open translation services for the world. In Proceedings of the 22nd Annual Conference of the European Association for Machine Transla tion. 479–480
2020
-
[36]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Po losukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017). 11 NLPIR 2024, December 13–15, 2024, Okayama, Japan Rich...
2017
-
[37]
acm-jdslogo.png
Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau , Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, and E douard Grave. 2019. CCNet: Extracting high quality monolingual datasets from w eb crawl data. arXiv preprint arXiv:1911.00359 (2019). 12 This figure "acm-jdslogo.pn...
2019 arXiv
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.