pith. sign in

arxiv: 2606.25365 · v1 · pith:SER5QZS7new · submitted 2026-06-24 · 💻 cs.CL · cs.AI

Neural Machine Translation for Low-Resource Tangkhul--English

Chormi Zimik Vashai , Agniva Maiti This is my paper

Pith reviewed 2026-06-25 21:22 UTC · model grok-4.3

classification 💻 cs.CL cs.AI
keywords neural machine translationlow-resource MTTangkhul languageByT5Tibeto-Burman languagesparallel corpusdomain adaptation
0
0 comments X

The pith

ByT5-large fine-tuned on 38,336 Tangkhul-English pairs reaches 39.97 BLEU on a 3,856-sentence test set.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops neural machine translation systems for Tangkhul, a Tibeto-Burman language with almost no prior NLP resources. It fine-tunes a ByT5-large model on 38,336 parallel sentences drawn mainly from biblical and narrative texts. On a separate test set of 3,856 sentences, the model reaches a BLEU score of 39.97 along with strong scores on chrF++, BERTScore, and COMET. The work also examines specific challenges like handling diacritics in Latin script and the effects of domain bias in the training data. This establishes initial performance benchmarks for future work on this language pair.

Core claim

The primary ByT5-large system, fine-tuned on 38,336 Tangkhul-English parallel sentence pairs, achieves a corpus BLEU score of 39.97, chrF++ of 58.07, BERTScore F1 of 0.8104, and COMET of 0.7302 on a held-out test set of 3,856 sentences. A contrastive mT5-small system is also evaluated on the same data. The approach addresses orthographic challenges from Tangkhul's diacritics and notes the domain bias toward biblical text, stories, and conversational data.

What carries the argument

Fine-tuning of ByT5-large, a byte-level text-to-text transfer transformer, on a small parallel corpus for sequence-to-sequence translation.

If this is right

  • The system provides usable translation quality for domain-specific biblical and narrative content.
  • Data diversification offers a direct path to higher performance.
  • Domain adaptation techniques can reduce the impact of corpus bias.
  • The reported metrics serve as a baseline for other low-resource Tibeto-Burman language pairs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Byte-level models may offer an advantage for languages whose scripts include variable diacritics.
  • The same fine-tuning approach could transfer to other under-resourced languages sharing similar data constraints.
  • Combining the existing corpus with synthetic data from related languages might lift scores without new human translations.

Load-bearing premise

The held-out test set reflects the same distribution as the training data despite noted domain bias in the corpus.

What would settle it

Evaluating the model on a test set drawn from everyday conversational Tangkhul outside biblical and narrative domains would show substantially lower scores if the domain bias is a problem.

Figures

Figures reproduced from arXiv: 2606.25365 by Agniva Maiti, Chormi Zimik Vashai.

Figure 1
Figure 1. Figure 1: Distribution of token counts per sentence in the Tangkhul–English parallel corpus. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparison of subword-level (mT5) versus byte-level (ByT5) representation for Tangkhul words with [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Effect of beam size on BLEU score and relative inference time. A beam size of 4–5 offers the best trade-off [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
read the original abstract

We present a study on low-resource machine translation for the Tangkhul-English (nmf-en) language pair. Tangkhul is a severely under-resourced Tibeto-Burman language spoken primarily in Manipur, India, with virtually no prior natural language processing infrastructure. We describe two systems: (1) a primary system based on ByT5-large fine-tuned on 38,336 Tangkhul-English parallel sentence pairs, and (2) a contrastive system based on mT5-small fine-tuned on the same corpus. Our primary ByT5-large system achieves a corpus BLEU score of 39.97, chrF++ of 58.07, BERTScore F1 of 0.8104, and COMET (wmt22-comet-da) of 0.7302 on a held-out test set of 3,856 sentences. We further discuss the orthographic challenges specific to Tangkhul's Latin-script diacritics, the domain bias of our training corpus (which comprises biblical text, stories, and conversational data), and avenues for future improvement through data diversification and domain adaptation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The manuscript presents the first neural machine translation systems for the low-resource Tangkhul-English (nmf-en) pair. A primary ByT5-large model is fine-tuned on 38,336 parallel sentences drawn from biblical, narrative, and conversational sources; a contrastive mT5-small system is also trained on the same data. On a held-out test set of 3,856 sentences the ByT5-large system reports corpus BLEU 39.97, chrF++ 58.07, BERTScore F1 0.8104 and COMET (wmt22-comet-da) 0.7302. The work additionally discusses orthographic challenges arising from Tangkhul Latin-script diacritics and notes the domain bias of the corpus.

Significance. If the reported scores are reproducible and the test set is representative, the paper supplies the first published baseline for Tangkhul MT and illustrates the suitability of byte-level models for languages whose orthography includes diacritics. The multi-metric evaluation (automatic metrics plus COMET) is a positive feature. The acknowledged domain bias, however, restricts the immediate applicability of the numbers to broader translation scenarios outside the dominant corpus domains.

major comments (1)
  1. [Data section / Experimental setup] The abstract states that the corpus exhibits domain bias toward biblical and narrative text, yet no information is supplied on whether the train/test split was stratified by domain or on the domain composition of the 3,856-sentence test set. Because the central claim rests on the reported metric values as evidence of effective low-resource MT, clarification of the splitting procedure and, ideally, domain-specific scores are required to establish that the test set measures performance beyond the dominant domains. (Data section / Experimental setup)

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback. We address the single major comment below and will revise the manuscript accordingly to improve clarity on the experimental setup.

read point-by-point responses

  1. Referee: [Data section / Experimental setup] The abstract states that the corpus exhibits domain bias toward biblical and narrative text, yet no information is supplied on whether the train/test split was stratified by domain or on the domain composition of the 3,856-sentence test set. Because the central claim rests on the reported metric values as evidence of effective low-resource MT, clarification of the splitting procedure and, ideally, domain-specific scores are required to establish that the test set measures performance beyond the dominant domains. (Data section / Experimental setup)

    Authors: We agree that the current manuscript lacks sufficient detail on the train/test split and domain composition, which limits interpretation of the results given the acknowledged domain bias. In the revised manuscript we will expand the Data section to explicitly describe the splitting procedure (including whether the split was performed randomly or stratified by domain) and report the domain composition of both the training and test sets. We will also add domain-specific metric scores (BLEU, chrF++, BERTScore, COMET) broken down by source type (biblical, narrative, conversational) to the extent the source metadata permits. This will allow readers to assess performance beyond the dominant domains. revision: yes

Circularity Check

0 steps flagged

No circularity: purely empirical MT evaluation on held-out data

full rationale

The paper reports standard fine-tuning of ByT5-large and mT5-small on 38,336 parallel sentences followed by direct metric computation (BLEU, chrF++, BERTScore, COMET) on a 3,856-sentence held-out test set. No equations, derivations, first-principles predictions, or self-citation chains appear. All reported numbers are outputs of off-the-shelf evaluation tools applied to an explicit train/test split; the domain-bias discussion is an explicit limitation statement rather than a hidden assumption that collapses the result. This is self-contained empirical work against external benchmarks.

Axiom & Free-Parameter Ledger

1 free parameters · 2 axioms · 0 invented entities

The central claim rests on the representativeness of the collected parallel corpus and the assumption that standard MT metrics remain informative under domain shift and limited data volume.

free parameters (1)
  • fine-tuning hyperparameters
    Chosen during adaptation of ByT5-large and mT5-small; not enumerated in the abstract.
axioms (2)
  • domain assumption Byte-level pre-trained models can be effectively adapted to new languages via supervised fine-tuning on parallel sentences.
    Invoked by the choice of ByT5-large as the primary system.
  • domain assumption Automatic metrics such as BLEU, chrF++, BERTScore and COMET provide a valid proxy for translation quality in this setting.
    Used to report the headline results.

pith-pipeline@v0.9.1-grok · 5732 in / 1417 out tokens · 32243 ms · 2026-06-25T21:22:37.219868+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

33 extracted references · 2 canonical work pages

  1. [1]

    Tangkhul.https://www.ethnologue.com/18/language/nmf/, 2015

    Ethnologue. Tangkhul.https://www.ethnologue.com/18/language/nmf/, 2015. Subscription required

    2015

  2. [2]

    Gyan Publishing House, 2011

    Khomdan Singh Lisam.Encyclopaedia of Manipur, volume 3. Gyan Publishing House, 2011

    2011

  3. [3]

    Ethnologue: Languages of the World. Myanmar. https://web.archive.org/web/20161010180533/http: //www.ethnologue.com/country/MM/languages, 2016. Archived from the original on 10 October 2016

    work page arXiv 2016

  4. [4]

    Commonwealth Publishers, 1996

    Visier Sanyu.A History of Nagas and Nagaland: Dynamics of Oral Tradition in Village F ormation. Commonwealth Publishers, 1996

    1996

  5. [5]

    A. S. W. Shimray.History of the Tangkhul Nagas. Akansha Publishing House, 2001. 9 Neural Machine Translation for Low-Resource Tangkhul–EnglishPreprint

    2001

  6. [6]

    Vangamla Salle K. S. Manipur: Literature festival strives to promote Tangkhul language. http://www.eastmojo .com/manipur/2023/11/26/manipur-literature-festival-strives-to-promote-tangkhul-languag e/, nov 2023. Retrieved 27 November 2023

    2023

  7. [7]

    William Pettigrew.Tangkhul Primer and Catechism. 1897

  8. [8]

    PhD thesis, Jawaharlal Nehru University, New Delhi, 1997

    Victor Ahum.Tangkhul-Naga Grammar: A Study of Word F ormation. PhD thesis, Jawaharlal Nehru University, New Delhi, 1997

    1997

  9. [9]

    PhD thesis, Assam University, 2019

    Loitongbam Sarankumari Devi.A Descriptive Grammar of Kabonglo: A Dialect of Tangkhul. PhD thesis, Assam University, 2019. hdl:10603/355391

    2019

  10. [10]

    PhD thesis, Assam University, 2019

    Aheibam Linthoingambi Chanu.A Descriptive Grammar of Lairamlo: A Dialect of Tangkhul. PhD thesis, Assam University, 2019. hdl:10603/355393

    2019

  11. [11]

    Attention is all you need

    Ashish Vaswani et al. Attention is all you need. InAdvances in Neural Information Processing Systems, pages 5998–6008, 2017

    2017

  12. [12]

    Transfer learning for low-resource neural machine translation

    Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. Transfer learning for low-resource neural machine translation. InProceedings of EMNLP 2016, pages 1568–1575, 2016

    2016

  13. [13]

    Universal neural machine translation for extremely low resource languages

    Jiatao Gu, Hany Hassan, Jacob Devlin, and Victor OK Li. Universal neural machine translation for extremely low resource languages. InProceedings of NAACL-HLT 2018, pages 344–354, 2018

    2018

  14. [14]

    Improving neural machine translation models with monolin- gual data

    Rico Sennrich, Barry Haddow, and Alexandra Birch. Improving neural machine translation models with monolin- gual data. InProceedings of ACL 2016, pages 86–96, 2016

    2016

  15. [15]

    Unsupervised machine translation using monolingual corpora only

    Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc’Aurelio Ranzato. Unsupervised machine translation using monolingual corpora only. InProceedings of ICLR 2018, 2018

    2018

  16. [16]

    Unsupervised neural machine translation

    Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. Unsupervised neural machine translation. In Proceedings of ICLR 2018, 2018

    2018

  17. [17]

    Khapra, and Pratyush Kumar

    Gowtham Ramesh, Sumanth Doddapaneni, Aravinth Bheemaraj, Mayank Jobanputra, Raghavan AK, Ajitesh Sharma, Sujit Sahoo, Harshita Diddee, Divyanshu Kakwani, Navneet Kumar, Aswin Majumder, Dipesh Raman, Vivek Jain, sachin tiwary, Mohit Yadav, Anoop Kunchukuttan, Pratyush Ramesh, Jay Gala, Sakshi Doshi, Pranshu M M, Vishal Kharde, Srihari V , Shruti Prakhya, A...

    2022

  18. [18]

    Indictrans2: Towards high-quality and accessible machine translation models for all 22 scheduled indian languages.arXiv preprint arXiv:2305.16307, 2023

    Jay Gala, Pranjal A Chitale, Raghavan Ak, Varun Gumma, Sumanth Doddapaneni, Aswanth Kumar, Janki Nawale, Anupama Sujatha, Ratish Puduppully, Vivek Raghavan, et al. Indictrans2: Towards high-quality and accessible machine translation models for all 22 scheduled indian languages.arXiv preprint arXiv:2305.16307, 2023

    work page arXiv 2023

  19. [19]

    ByT5: Towards a token-free future with pre-trained byte-to-byte models.Transactions of the Association for Computational Linguistics, 10:291–306, 2022

    Linting Xue et al. ByT5: Towards a token-free future with pre-trained byte-to-byte models.Transactions of the Association for Computational Linguistics, 10:291–306, 2022

    2022

  20. [20]

    Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of Machine Learning Research, 21(140):1–67, 2020

    Colin Raffel et al. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of Machine Learning Research, 21(140):1–67, 2020

    2020

  21. [21]

    mt5: A massively multilingual pre-trained text-to-text transformer

    Linting Xue et al. mt5: A massively multilingual pre-trained text-to-text transformer. InProceedings of NAACL 2021, pages 483–498, 2021

    2021

  22. [22]

    A few thousand translations go a long way! leveraging pre-trained models for african news translation

    David Ifeoluwa Adelani, Jesujoba Alabi, Angela Fan, Julia Kreutzer, Xiaoyu Shen, Machel Reid, Dana Ruiter, Dietrich Klakow, Peter Nabende, Ernie Chang, et al. A few thousand translations go a long way! leveraging pre-trained models for african news translation. InProceedings of the 2022 Conference of the North American Chapter of the Association for Compu...

    2022

  23. [23]

    Are character-level translations worth the wait? comparing ByT5 and mT5 for machine translation.Transactions of the Association for Computational Linguistics, 12:392–410, 2024

    Lukas Edman, Gabriele Sarti, Antonio Toral, Gertjan van Noord, and Arianna Bisazza. Are character-level translations worth the wait? comparing ByT5 and mT5 for machine translation.Transactions of the Association for Computational Linguistics, 12:392–410, 2024

    2024

  24. [24]

    Creating a massively parallel bible corpus

    Thomas Mayer and Michael Cysouw. Creating a massively parallel bible corpus. InProceedings of LREC 2014, pages 3158–3163, 2014

    2014

  25. [25]

    JW300: A wide-coverage parallel corpus for low-resource languages

    Željko Agi ´c and Ivan Vuli´c. JW300: A wide-coverage parallel corpus for low-resource languages. InProceedings of ACL 2019, pages 3204–3210, 2019

    2019

  26. [26]

    BLEU: A method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. BLEU: A method for automatic evaluation of machine translation. InProceedings of ACL 2002, pages 311–318, 2002. 10 Neural Machine Translation for Low-Resource Tangkhul–EnglishPreprint

    2002

  27. [27]

    A call for clarity in reporting BLEU scores

    Matt Post. A call for clarity in reporting BLEU scores. InProceedings of the Third Conference on Machine Translation (WMT18): Research Papers, pages 186–191, 2018

    2018

  28. [28]

    chrf: Character n-gram f-score for automatic mt evaluation

    Maja Popovi´c. chrf: Character n-gram f-score for automatic mt evaluation. InProceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, 2015

    2015

  29. [29]

    chrf++: Words helping character n-grams

    Maja Popovi´c. chrf++: Words helping character n-grams. InProceedings of the Second Conference on Machine Translation (WMT17), pages 612–618, 2017

    2017

  30. [30]

    BERTScore: Evaluating text generation with BERT

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. BERTScore: Evaluating text generation with BERT. InProceedings of ICLR 2020, 2020

    2020

  31. [31]

    COMET: A neural framework for mt evaluation

    Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. COMET: A neural framework for mt evaluation. In Proceedings of EMNLP 2020, pages 2685–2695, 2020

    2020

  32. [32]

    COMET-22: Unbabel-ist 2022 submission for the metrics shared task

    Ricardo Rei et al. COMET-22: Unbabel-ist 2022 submission for the metrics shared task. InProceedings of WMT22, pages 578–585, 2022

    2022

  33. [33]

    Datasets: A community library for natural language processing

    Quentin Lhoest et al. Datasets: A community library for natural language processing. InProceedings of EMNLP 2021: System Demonstrations, pages 175–184, 2021. 11

    2021