REVIEW 1 major objections 7 minor 43 references
Reviewed by Pith at T0; open to challenge.
T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →
T0 review · glm-5.2
Pivoting through Cyrillic lifts Traditional Mongolian translation
2026-07-08 22:32 UTC pith:37QRABXP
load-bearing objection CoPiT is a solid, well-engineered pipeline for Traditional Mongolian MT with a real linguistic contribution, but the headline comparison to GPT-4.1 is apples-to-oranges and the reference-based evaluation has a structural bias the paper doesn't acknowledge. the 1 major comments →
CoPiT: Cognitive Pivot Translation for Digraphic Low-Resource Mongolian in the Traditional Script
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central mechanism is script-level pivoting: routing translation through a better-resourced orthographic representation of the same language to resolve ambiguity before semantic transfer. The Traditional-to-Cyrillic conversion is factorized into linguistically motivated sub-steps — vowel harmony recovery narrows phonological interpretations, Latin-assisted normalization makes implicit phonological distinctions explicit, Cyrillic normalization produces a canonical intermediate form, and self-reflection enforces sentence-level coherence. The ablation shows that self-reflection is the most load-bearing component: removing it causes the largest performance drop (e.g., English COMET falls from
What carries the argument
CoPiT pipeline: morphological segmentation → vowel harmony recovery → Latin-assisted normalization → Cyrillic normalization → sentence reconstruction with self-reflection → Cyrillic-to-target translation. Trained component-wise on 14,125 word-level lexical pairs and 2,061 sentence-level revision pairs, requiring no sentence-level Traditional-to-target parallel data.
Load-bearing premise
The pipeline depends on the Traditional-to-Cyrillic conversion being accurate enough that conversion errors do not propagate into downstream translation, and the evaluation rests on small datasets (1,031 sentences for reference-based evaluation, 12 sentences for human evaluation) with sometimes low inter-annotator agreement. The comparison that open-source models 'match or outperform GPT-4.1' pits fine-tuned open-source models against zero-shot GPT-4.1, an asymmetry in how
What would settle it
If the Traditional-to-Cyrillic conversion step introduced systematic errors that the downstream translation could not recover from, the pivot would degrade rather than improve translation quality. A direct test would measure conversion error rates and correlate them with end-to-end translation quality degradation.
If this is right
- Script-level pivoting could generalize to other digraphic languages where one script is better-resourced than another (e.g., languages with both Latin and non-Latin orthographies, or classical/modern script pairs).
- The synthetic data generation loop — using the pipeline to create parallel corpora from monolingual Traditional-script sources — offers a self-bootstrapping path for languages where parallel data collection is prohibitively expensive.
- The finding that self-reflection is the most critical component suggests that global sentence-level coherence, not local disambiguation, is the bottleneck in translating from orthographically ambiguous scripts.
- Fine-tuned open-source models matching GPT-4.1 suggests that structured linguistic decomposition can compensate for raw model scale in low-resource settings.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes CoPiT, a cognitively motivated pipeline for translating Mongolian text written in the Traditional script into English, Korean, and Russian. The core idea is to route translation through the Cyrillic script, which is more phonemically transparent and better-resourced. The Traditional-to-Cyrillic conversion is factorized into linguistically motivated steps: morphological segmentation, vowel harmony recovery, Latin-assisted normalization, Cyrillic normalization, and sentence-level self-reflection. Each component is independently fine-tuned using word-level lexical data (14,125 entries) and sentence-level revision pairs (2,061). Experiments span multiple backbone models (Qwen-3 4B/30B, Ministral-3 3B/14B, GPT-4.1) and three target languages, evaluated under reference-based, reference-free, and human evaluation protocols. The authors also demonstrate that CoPiT-generated synthetic parallel data (8,034 pairs) enables reverse-direction translation and improves forward translation. Datasets and code are released publicly.
Significance. The paper addresses a genuine and well-motivated problem: the digraphic resource imbalance in Mongolian NLP, where the Traditional script is both orthographically ambiguous and severely under-resourced. The cognitive motivation—mirroring how fluent readers map Traditional forms to Cyrillic—is a reasonable framing. The pipeline is linguistically grounded, with components targeting distinct sources of underspecification (vowel harmony, phonological normalization, sentence-level coherence). The ablation study (Table 3, Tables 9–11) is thorough, isolating both individual and pairwise component contributions. The release of a multi-script parallel corpus and all code is a concrete contribution to the community. The synthetic data generation loop and its validation through both forward and reverse translation experiments (Table 4, Figure 6) demonstrate practical utility beyond the inference-time pipeline.
major comments (1)
- §4.1, paragraph on Datasets: The reference translations for the reference-based evaluation set (1,031 sentences) are derived from the Cyrillic side of parallel pairs—'English references are translated from the Cyrillic side and subsequently validated by bilingual speakers.' Since CoPiT explicitly routes Traditional→Cyrillic→Target, its outputs are derived from the same Cyrillic representation that generated the references. Direct translation, by contrast, interprets the Traditional script independently and may produce semantically valid translations that diverge from the Cyrillic-grounded reference, particularly for the orthographically ambiguous forms the paper identifies as the core challenge. This creates a systematic structural bias in reference-based evaluation (Table 1) that favors CoPiT by construction. The paper should explicitly acknowledge this bias and clarify that the COMET/B
minor comments (7)
- Table 1: The BLEU-3/4 column header is unusual; standard practice reports BLEU-4 (or BLEU). Clarify what BLEU-3/4 means—is it n-gram order 3 and 4 reported separately?
- Table 2: The GPT-4.1 row for Russian appears to have a formatting issue where the COMETKiwi value (0.429) runs into the Adeq. column.
- §4.2.2: The ablation discussion notes that removing Vowel Harmony Recovery sometimes yields higher COMET (e.g., Qwen-3 4B English: 0.633 without VHR vs. 0.628 with). The paper attributes this to backbone-dependent behavior, but the interaction is not analyzed further. A brief discussion of why VHR can hurt would strengthen the analysis.
- Appendix A.2, Table 5: Fluency inter-annotator agreement for English is α=0.263, which is below the conventional threshold for reliable annotation. The paper discusses this but could note more explicitly that fluency conclusions for English should be treated with caution.
- §3.2, Morphological Segmentation: The suffix dictionary is mentioned but its size and coverage are not specified. Providing the number of suffixes would help readers assess the generality of this approach.
- Figure 2: The fire emoji symbols are unconventional for a system architecture diagram. Consider replacing with standard notation.
- References: The citation 'Tumur-Ochir et al.' (in §2.2) is missing a year in the reference list.
Simulated Author's Rebuttal
The referee raises a valid concern about structural bias in reference-based evaluation: because reference translations are derived from the Cyrillic side of parallel pairs, and CoPiT routes through Cyrillic, CoPiT's outputs may be systematically favored over direct translation outputs that could be semantically valid but diverge from the Cyrillic-grounded reference. We acknowledge this bias and will revise the manuscript accordingly, while also noting that our reference-free evaluation (Table 2) and human evaluation provide independent evidence that is not subject to this concern.
read point-by-point responses
-
Referee: §4.1, Datasets: The reference translations for the reference-based evaluation set (1,031 sentences) are derived from the Cyrillic side of parallel pairs. Since CoPiT explicitly routes Traditional→Cyrillic→Target, its outputs are derived from the same Cyrillic representation that generated the references. Direct translation interprets the Traditional script independently and may produce semantically valid translations that diverge from the Cyrillic-grounded reference, particularly for orthographically ambiguous forms. This creates a systematic structural bias in reference-based evaluation (Table 1) that favors CoPiT by construction. The paper should explicitly acknowledge this bias and clarify that the COMET/BLEU scores in Table 1 should be interpreted with this caveat.
Authors: The referee is correct that there is a structural affinity between CoPiT's intermediate representation and the reference translations, since both are derived from the Cyrillic side of the parallel pairs. We acknowledge this bias and will explicitly state it in the revised manuscript, adding a caveat to the description of the reference-based evaluation set in §4.1 and a note in the discussion of Table 1. Specifically, we will add language clarifying that because English references are translated from the Cyrillic side, CoPiT's Cyrillic pivoting may align more closely with reference translations by construction, and that direct translation outputs that are semantically valid but diverge from the Cyrillic-grounded reading would be penalized under reference-based metrics. We will recommend that readers interpret Table 1 results in conjunction with the reference-free evaluation (Table 2), which uses COMETKiwi and human adequacy/fluency ratings and does not rely on Cyrillic-derived references. The reference-free results show consistent improvements for CoPiT across all backbones and target languages, providing evidence that the gains are not solely an artifact of the evaluation bias. That said, we agree the bias should be transparently acknowledged, and the manuscript will be revised accordingly. revision: yes
Circularity Check
No significant circularity; evaluation-bias concern is real but is a correctness risk, not a derivation-chain circularity.
full rationale
The paper's derivation chain is: (1) Traditional→Cyrillic conversion trained on word-level lexical pairs (14,125 entries) and sentence-level revision pairs (2,061), (2) Cyrillic→Target translation via LLM. The training data for conversion components is independent of the evaluation test sets (1,031 reference-based, 380 reference-free). The skeptic's headline concern—that English references are translated from the Cyrillic side (Section 4.1: 'English references are translated from the Cyrillic side and subsequently validated by bilingual speakers')—is a legitimate evaluation-bias concern, but it is not circularity in the sense of the derivation reducing to its own inputs. The references are human-created from Cyrillic, not generated by CoPiT itself. CoPiT's outputs are not used to construct the evaluation references. The paper also provides reference-free evaluation (Table 2, COMETKiwi) and human evaluation as independent support that does not depend on Cyrillic-derived references. The synthetic data generation loop (Section 4.2.3) is explicitly acknowledged as an 'upper-bound analysis' and is not presented as a first-principles prediction. No self-citation chain is load-bearing: the cognitive motivation cites an external textbook (Altangerel and Togtokh, 2024), and the self-reflection component cites external work (Wang et al., 2024; Chen et al., 2024). The one point is assigned for the minor structural affinity between the Cyrillic-derived references and the Cyrillic-pivoting pipeline, which, while not circular, could inflate reference-based metrics relative to direct translation—a concern the paper does not fully address but which falls under evaluation validity rather than circularity of derivation.
Axiom & Free-Parameter Ledger
free parameters (2)
- Suffix dictionary for morphological segmentation =
curated, size not specified
- LoRA hyperparameters (rank, alpha, dropout) =
rank not specified; lr=1e-4; batch=4; grad_accum=2-4; epochs=2-3
axioms (4)
- domain assumption Fluent Mongolian readers implicitly map Traditional script to Cyrillic when reading
- domain assumption Cyrillic Mongolian is sufficiently better-resourced than Traditional to serve as a reliable intermediate representation
- ad hoc to paper Word-level lexical supervision (14,125 entries) generalizes compositionally to sentence-level disambiguation
- domain assumption BLEU and chrF are adequate metrics for Traditional Mongolian reverse-direction evaluation
read the original abstract
Low-resource languages remain challenging for machine translation, and Mongolian is a representative case. As a digraphic language, Mongolian is written in both Cyrillic and Traditional scripts, which exhibit a severe imbalance in data availability. While the Cyrillic script is relatively well-resourced, the Traditional script remains extremely data-scarce and orthographically ambiguous, leading to substantial performance degradation in direct translation. We propose CoPiT, a cognitively motivated pivot-based translation pipeline that exploits this internal resource hierarchy by routing translation through the Cyrillic script. The pipeline explicitly resolves script-induced ambiguity in the Traditional script before translation, enabling more stable and accurate meaning transfer. Across multiple backbone models and target languages, CoPiT consistently outperforms direct translation, achieving substantial absolute BLEU improvements together with consistent 1.5-1.6x COMET gains. These gains allow strong open-source models to match or outperform GPT-4.1 under comparable evaluation settings. Beyond inference-time improvements, CoPiT enables the construction of synthetic parallel data directly from Traditional-script text, mitigating data scarcity in realistic low-resource scenarios. We release a new multi-script parallel dataset covering Mongolian in both scripts alongside English, Korean, and Russian. All datasets and code are publicly available at https://anonymous.4open.science/r/anonymous_project-76C7.
Figures
Reference graph
Works this paper leans on
-
[1]
Parallel Corpora for Machine Translation in Low-Resource I ndic Languages: A Comprehensive Review
Raja, Rahul and Vats, Arpita. Parallel Corpora for Machine Translation in Low-Resource I ndic Languages: A Comprehensive Review. Proceedings of the Eighth Workshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT 2025). 2025. doi:10.18653/v1/2025.loresmt-1.12
-
[2]
Classical Mongolian: A Textbook for Students, Scholars, and Everyone Interested in Mongolian , author =. 2024 , publisher =
work page 2024
-
[3]
Үндэсний бичиг VI (Traditional Mongolian Script VI) , author =. 2018 , publisher =
work page 2018
-
[4]
Baishya, Diganta and Baruah, Rupam and Bora, Mousoomi and Sarma, Biswajit. Processing Low-Resource Languages: A Review Of Challenges And Strategies For Inclusive NLP And Sustainable Environment_2025. International Journal of Environmental Sciences , year=. doi:10.64252/w55rwj24
-
[5]
W iki M atrix: Mining 135 M Parallel Sentences in 1620 Language Pairs from W ikipedia
Schwenk, Holger and Chaudhary, Vishrav and Sun, Shuo and Gong, Hongyu and Guzm \'a n, Francisco. W iki M atrix: Mining 135 M Parallel Sentences in 1620 Language Pairs from W ikipedia. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 2021. doi:10.18653/v1/2021.eacl-main.115
-
[6]
CCM atrix: Mining Billions of High-Quality Parallel Sentences on the Web
Schwenk, Holger and Wenzek, Guillaume and Edunov, Sergey and Grave, Edouard and Joulin, Armand and Fan, Angela. CCM atrix: Mining Billions of High-Quality Parallel Sentences on the Web. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume ...
-
[7]
Improving Neural Machine Translation Models with Monolingual Data
Sennrich, Rico and Haddow, Barry and Birch, Alexandra. Improving Neural Machine Translation Models with Monolingual Data. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2016. doi:10.18653/v1/P16-1009
-
[8]
Understanding Back-Translation at Scale
Edunov, Sergey and Ott, Myle and Auli, Michael and Grangier, David. Understanding Back-Translation at Scale. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018. doi:10.18653/v1/D18-1045
-
[9]
Scaling Low-Resource MT via Synthetic Data Generation with LLM s
de Gibert, Ona and Attieh, Joseph and Vahtola, Teemu and Aulamo, Mikko and Li, Zihao and V. Scaling Low-Resource MT via Synthetic Data Generation with LLM s. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. doi:10.18653/v1/2025.emnlp-main.1408
-
[10]
Multilingual Data Filtering using Synthetic Data from Large Language Models
Waldendorf, Jonas and Haddow, Barry and Birch, Alexandra and Klimaszewski, Mateusz. Multilingual Data Filtering using Synthetic Data from Large Language Models. Findings of the Association for Computational Linguistics: EMNLP 2025. 2025. doi:10.18653/v1/2025.findings-emnlp.495
-
[11]
Unsupervised Cross-lingual Representation Learning at Scale
Conneau, Alexis and Khandelwal, Kartikay and Goyal, Naman and Chaudhary, Vishrav and Wenzek, Guillaume and Guzm \'a n, Francisco and Grave, Edouard and Ott, Myle and Zettlemoyer, Luke and Stoyanov, Veselin. Unsupervised Cross-lingual Representation Learning at Scale. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. ...
-
[12]
m T 5: A Massively Multilingual Pre-trained Text-to-Text Transformer
Xue, Linting and Constant, Noah and Roberts, Adam and Kale, Mihir and Al-Rfou, Rami and Siddhant, Aditya and Barua, Aditya and Raffel, Colin. m T 5: A Massively Multilingual Pre-trained Text-to-Text Transformer. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2...
-
[13]
Liu, Yihong and Ma, Chunlan and Ye, Haotian and Schuetze, Hinrich. T ransli C o: A Contrastive Learning Framework to Address the Script Barrier in Multilingual Pretrained Language Models. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024. doi:10.18653/v1/2024.acl-long.136
-
[14]
Proceedings of the 5th Workshop on Multilingual Representation Learning (MRL 2025) , pages=
Exploring the role of transliteration in in-context learning for low-resource languages written in non-latin scripts , author=. Proceedings of the 5th Workshop on Multilingual Representation Learning (MRL 2025) , pages=
work page 2025
-
[15]
Chaudhary, Aditi and Zhou, Chunting and Levin, Lori and Neubig, Graham and Mortensen, David R. and Carbonell, Jaime. Adapting Word Embeddings to New Languages with Morphological and Phonological Subword Representations. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018. doi:10.18653/v1/D18-1366
-
[16]
Low-resource neural machine translation with morphological modeling
Low-resource neural machine translation with morphological modeling , author=. arXiv preprint arXiv:2404.02392 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[17]
International Conference on Intelligent Multilingual Information Processing , pages=
MM-Eval: A Hierarchical Benchmark for Modern Mongolian Evaluation in LLMs , author=. International Conference on Intelligent Multilingual Information Processing , pages=. 2024 , organization=
work page 2024
-
[18]
Evaluating Large Language Models in Mongolian , author =
-
[19]
2024 International Joint Conference on Neural Networks (IJCNN) , pages=
Pre-training Language Model for Mongolian with Agglutinative Linguistic Knowledge Injection , author=. 2024 International Joint Conference on Neural Networks (IJCNN) , pages=. 2024 , organization=
work page 2024
-
[20]
Incorporating Inner-word and Out-word Features for M ongolian Morphological Segmentation
Liu, Na and Su, Xiangdong and Zhang, Haoran and Gao, Guanglai and Bao, Feilong. Incorporating Inner-word and Out-word Features for M ongolian Morphological Segmentation. Proceedings of the 28th International Conference on Computational Linguistics. 2020. doi:10.18653/v1/2020.coling-main.408
-
[21]
Dependent syntactic analysis of Mongolian based on semi-supervised self-training , year=
Zhang, Zhonghao and Ma, Jiajia and Liu, Na and Wu, Nier and Ji, Yatu and Liu, Guiping , booktitle=. Dependent syntactic analysis of Mongolian based on semi-supervised self-training , year=
-
[22]
The Study of Comparison and Conversion about Traditional Mongolian and Cyrillic Mongolian , year=
Li, Hao and Sarina, Bao , booktitle=. The Study of Comparison and Conversion about Traditional Mongolian and Cyrillic Mongolian , year=
-
[23]
2023 IEEE 9th International Conference on Cloud Computing and Intelligent Systems (CCIS) , pages=
Traditional Mongolian-to-Cyrillic Mongolian Conversion Method Based on the Combination of Rules and Transformer , author=. 2023 IEEE 9th International Conference on Cloud Computing and Intelligent Systems (CCIS) , pages=. 2023 , organization=
work page 2023
-
[24]
Journal of Institute of Mathematics and Digital Technology , author=
Cyrillic Mongolian-to-traditional Mongolian conversion method based on the transformer , volume=. Journal of Institute of Mathematics and Digital Technology , author=. 2024 , month=. doi:10.5564/jimdt.v6i1.3599 , abstractNote=
-
[25]
International Conference on Neural Information Processing , pages=
A Deep Investigation of RNN and Self-attention for the Cyrillic-Traditional Mongolian Bidirectional Conversion , author=. International Conference on Neural Information Processing , pages=. 2022 , organization=
work page 2022
-
[26]
Journal of Institute of Mathematics and Digital Technology , volume=
Joint NMT models for text conversion between traditional Mongolian script and cyrillic Mongolian: a comparative study , author=. Journal of Institute of Mathematics and Digital Technology , volume=
- [27]
-
[28]
Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=
work page internal anchor Pith review Pith/arXiv arXiv
- [29]
- [30]
-
[31]
Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[32]
Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities , author=. arXiv preprint arXiv:2507.06261 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[33]
Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages=
Bleu: a method for automatic evaluation of machine translation , author=. Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages=
-
[34]
Proceedings of the tenth workshop on statistical machine translation , pages=
chrF: character n-gram F-score for automatic MT evaluation , author=. Proceedings of the tenth workshop on statistical machine translation , pages=
-
[35]
Proceedings of the second conference on machine translation , pages=
chrF++: words helping character n-grams , author=. Proceedings of the second conference on machine translation , pages=
-
[36]
Proceedings of the Seventh Conference on Machine Translation (WMT) , pages=
COMET-22: Unbabel-IST 2022 submission for the metrics shared task , author=. Proceedings of the Seventh Conference on Machine Translation (WMT) , pages=
work page 2022
-
[37]
Proceedings of the Eighth Conference on Machine Translation , pages=
Scaling up cometkiwi: Unbabel-ist 2023 submission for the quality estimation shared task , author=. Proceedings of the Eighth Conference on Machine Translation , pages=
work page 2023
-
[38]
Edward J Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen-Zhu and Yuanzhi Li and Shean Wang and Lu Wang and Weizhu Chen , booktitle=. Lo. 2022 , url=
work page 2022
-
[39]
TasTe: Teaching Large Language Models to Translate through Self-Reflection
Taste: Teaching large language models to translate through self-reflection , author=. arXiv preprint arXiv:2406.08434 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[40]
DUAL-REFLECT: Enhancing large language models for reflective translation through dual learning feedback mechanisms , author=. arXiv preprint arXiv:2406.07232 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[41]
International conference on machine learning , pages=
Similarity of neural network representations revisited , author=. International conference on machine learning , pages=. 2019 , organization=
work page 2019
-
[42]
Revisiting Low-Resource Neural Machine Translation: A Case Study
Revisiting low-resource neural machine translation: A case study , author=. arXiv preprint arXiv:1905.11901 , year=
work page internal anchor Pith review Pith/arXiv arXiv 1905
-
[43]
Content Analysis: An Introduction to Its Methodology , author =. 2019 , edition =. doi:10.4135/9781071878781 , url =
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.