Pith. sign in

REVIEW 3 major objections 7 minor 35 references

Vuyko Mistral: Adapting LLMs for Low-Resource Dialectal Translation

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuned 7B–8B open models beat zero-shot GPT-4o on every reported metric for standard-to-Hutsul translation, with Mistral at 74.35 BLEU and 3.60 dialect quality versus GPT-4o's 56.64 and 3.22.

desk verdict Useful new Hutsul-Ukrainian resource, but the headline GPT-4o comparison is not established because the test set draws on the same novel that dominates training and synthetic data. read the letter →

arxiv 2506.07617 v1 pith:BGSKOX2J submitted 2025-06-09 cs.CL

Roman Kyslyi , Yuliia Maksymiuk , Ihor Pysmennyi This is my paper
classification cs.CL
keywords Hutsuldialectdialectalmachinetranslationlow-resourceNLPretrieval-augmentedgenerationsyntheticparalleldataLoRAfine-tuningUkrainianLLM-basedevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a small, openly available language model fine-tuned on a modest amount of Hutsul–Ukrainian parallel data can translate standard Ukrainian into the Hutsul dialect better than a large commercial model prompted zero-shot. To do this, it builds the first Hutsul–Ukrainian parallel corpus, a dialect dictionary, and a retrieval-augmented pipeline that manufactures 52,142 additional training pairs from grammar rules and retrieved examples. On a 1,900-sentence test set, the best Mistral model reaches a BLEU of 74.35 and a dialect-quality rating of 3.60, versus GPT-4o's 56.64 and 3.22. The authors argue this shows specialized fine-tuning is a practical route for low-resource dialect translation where large models alone fall short.

What carries the argument

The load-bearing mechanism is the advanced RAG synthetic-data pipeline: GPT-4o first extracts structured Hutsul grammar rules from the dialectal novel and auxiliary sources, then for each standard Ukrainian sentence from UberText retrieves the top-3 semantically similar Hutsul sentences from an indexed corpus, and is prompted to produce a dialect translation; a sequence-similarity and alignment-based filter (U-src, U-tgt, X) removes low-quality pairs. This mechanism expands the 9,852 manually aligned pairs into about 62,000 training pairs. The models are then adapted by LoRA fine-tuning, and evaluation combines BLEU, chrF++, and TER with GPT-4o ratings of fluency, adequacy, and dialectal quality.

What would settle it

Translate a set of Hutsul texts written outside the novel that anchors the corpus — for example, blog posts, transcribed Carpathian speech, or folk tales from another region — with the best fine-tuned model and GPT-4o, and have native Hutsul speakers rate the outputs. If GPT-4o matches or exceeds the fine-tuned model on these out-of-corpus texts, the central claim of general superiority fails.

Watch

Extended reading notes

Core claim

The central claim is that fine-tuned 7B–8B open-source models outperform GPT-4o on standard-to-Hutsul translation across every reported automatic and LLM-judged metric. The best configuration, Mistral-7B fine-tuned on the combined manual and synthetically augmented corpus, scores 74.35 BLEU, 81.89 chrF++, and a dialect quality rating of 3.60, while GPT-4o scores 56.64, 65.90, and 3.22. The paper also finds that synthetic data from the RAG pipeline substantially improves automatic metrics, and that even manual-only fine-tuning beats the commercial baseline, with adequacy staying near 4.7 for all models while dialectal quality is the most data-sensitive axis.

Load-bearing premise

The paper assumes that its 1,900-sentence test set, taken entirely from the same novel that supplied training, synthetic generation, and retrieval, fairly measures Hutsul translation ability; if that novel is not representative of the dialect, the reported advantage over GPT-4o may not generalize.

Editorial extensions

If this is right

  • Fine-tuned local 7B models can beat a large commercial zero-shot model on a low-resource dialect, so usable dialect translation does not require API access or large-scale compute.
  • The RAG-based synthetic augmentation expands the training set to about 62,000 pairs and yields large jumps in BLEU and chrF++, from 62.36 to 74.35 BLEU for Mistral.
  • The released corpus, dictionary, and models give the Hutsul dialect its first computational resources, and the same recipe is claimed to be adaptable to other Ukrainian dialects.
  • Because synthetic data mainly boosts surface-level metrics, dialectal quality gains are more modest, suggesting that authentic retraining data remains important.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the test set comes from the same novel that anchors the training, grammar-rule extraction, and retrieval index, the reported margin over GPT-4o is best read as in-distribution performance; a cross-corpus test set would show how far the models generalize to other Hutsul writing.
  • The pipeline's recipe — extract rules, retrieve similar examples, generate synthetic pairs, filter by alignment — is a candidate template for other low-resource dialect pairs, and could be tested by applying it to Boyko or Lemko Ukrainian.
  • If GPT-4o's judgments are biased toward standard Ukrainian forms, then valid dialectal variants may be underrated; collecting native-speaker ratings would recalibrate the dialect-quality scores.
  • The alignment-based filtering thresholds (U-src < 0.1, U-tgt < 0.1, X < 0.2) could serve as a cheap generic filter for synthetic dialect data, though the thresholds would likely need retuning per dialect.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper introduces the first parallel corpus for Hutsul dialect and standard Ukrainian, consisting of 9,852 manually aligned sentence pairs and a 7,320-entry dialect-to-standard dictionary, together with a retrieval-augmented generation (RAG) pipeline that expands the corpus with 52,142 synthetic pairs. Using LoRA, the authors fine-tune Mistral-7B-Instruct v0.3 and LLaMA-3.1 8B Instruct on either the manual corpus alone or the manual-plus-synthetic corpus, and compare them against GPT-4o prompted with RAG context and dictionary entries on a 1,900-sentence held-out test set, using BLEU, chrF++, TER, and a GPT-4o-based fluency/adequacy/dialect-quality judge. The paper's central claim is that these 7B–8B fine-tuned models outperform GPT-4o on standard-to-Hutsul translation across both automatic and LLM-based metrics, with the best configuration (Mistral on manual-plus-synthetic) reaching 74.35 BLEU and a 3.60 dialect-quality rating.

Significance. The resource contributions are genuinely valuable: the first parallel Hutsul–Ukrainian corpus, a substantial dictionary, a reproducible RAG-based synthetic generation pipeline with alignment-based filtering, and a public release of data, models, and code are all strengths that will benefit future work on this underexplored dialect. However, the headline comparative claim—that small open models outperform GPT-4o on Hutsul translation—is not established by the present evaluation design. Training data, synthetic generation, and the test set all draw on the same literary source, and the primary evaluation metric is an unvalidated GPT-4o judge; the reported margins are therefore consistent with in-distribution memorization and judge bias rather than general translation capability. The paper's durable value lies in its resources and pipeline, while the comparative results require a redesigned evaluation before they can support the stated conclusion.

major comments (3)
  1. [§3.4, §3.1, §3.3, Table 2] The test set of 1,900 sentences is, by the paper's own description in §3.4, composed only of human-annotated sentence pairs from 'Dido Yvanchik', while §3.1 states that a significant portion of the manual training corpus is also based on that novel, and §3.3 (steps 1 and 2) describes the same novel as the primary retrieval corpus and the source of linguistic examples for synthetic generation. The fine-tuned models are therefore both trained and synthetically augmented on the very text distribution from which the test sentences are drawn, whereas the GPT-4o baseline has no such training exposure. Under this design, the Table 2 margins (74.35 vs 56.64 BLEU; 3.60 vs 3.22 dialect rating) can be explained by in-distribution memorization or style replication, so the abstract's claim that fine-tuned models 'outperform zero-shot baselines such as GPT-4o across both automatic and LLM-evaluated metrics' is not supported as a statement about general Hutsul translation ability.
  2. [§5.1, §3.3, §5.2, Table 2] Section 5.1 declares the GPT-4o-based fluency, adequacy, and dialect scores to be the paper's primary evaluation metrics, yet no human validation of this judge is provided, and the judge is the same model family (GPT-4o) that generated the synthetic training data in §3.3 and that serves as the comparison baseline in §5.2; the judgment prompt in §5.1 also supplies the reference translation to the judge. The reported dialect-quality differences in Table 2 (3.22–3.60 on a 1–5 scale) are small, and no variance, confidence intervals, or significance tests are reported for either the automatic or the LLM-based metrics. The Limitations section itself concedes that automatic metrics 'may overestimate linguistic validity' and that GPT-4o's preferences may align with standard Ukrainian; because the LLM scores carry the primary comparative claim, the unvalidated-judge concern is load-bearing rather than ancillary.
  3. [§5.4, Table 2] The qualitative example in §5.4 contradicts the ranking in Table 2: Mistral trained on manual data only receives Dialect=4 and Adequacy=5 in the example, while Mistral trained on manual-plus-synthetic receives Dialect=3, the reverse of the Table 2 ordering (3.35 vs 3.60 aggregate dialect scores), and the per-example BLEU scores (7.77–34.39) are far below the Table 2 averages (56.64–74.35). The example-level reversal of dialect scores is inconsistent with the aggregate ordering, and the large gap between example and aggregate BLEU shows that the reference-based metrics are highly unstable at the sentence level; this reinforces the need for variance reporting before any cross-model conclusion can be drawn.
minor comments (7)
  1. [Abstract, §5.2] The abstract calls GPT-4o a 'zero-shot baseline' while the Introduction and §5.2 describe the comparison as few-shot prompting with RAG context and dictionary entries; these descriptions should be reconciled, since §5.2 shows the baseline is not zero-shot.
  2. [§3.3] The filtering thresholds in §3.3 (SequenceMatcher similarity 0.45; U-src<0.1, U-tgt<0.1, X<0.2) are presented as empirical choices without sensitivity analysis; given that Table 1 shows the filtered synthetic data has very different alignment statistics from the original corpus, the robustness of downstream results to these thresholds should be assessed.
  3. [Table 2, §5.1] Table 2 reports no confidence intervals, bootstrap estimates, or significance tests; for automatic metrics computed over 1,900 sentences this information is standard practice and would clarify whether the dialect-rating differences between systems are meaningful.
  4. [§3.2] The dictionary of 7,320 pairs described in §3.2 is never used in the fine-tuning or evaluation description, so its role in the pipeline should be stated explicitly or removed from the contribution list.
  5. [References] The references contain several formatting defects, including 'and 1 others' placeholders, 'Ziweietal.Liu.2023', and 'HugoTouvron,ThibautLavril,AlpYurtsever,and1others', which should be corrected.
  6. [§5.1] Section 5.1 is internally inconsistent: it first calls the LLM-based adequacy and dialect scores the 'primary evaluation metrics' and then states that automatic metrics are 'used in a supporting role', which reverses the hierarchy described two sentences earlier.
  7. [§3.3, §5.1] Typos and usage errors include 'To insure translation quality' (§5.1), 'This process have created some data alignment challenges' (§3.3), and 'enlarged' (§3.3); these should be cleaned up.

Circularity Check

1 steps flagged · score 6.0 of 10

Test and training data share the same single novel source, so the headline comparison with GPT-4o is an in-domain fit rather than an independent Hutsul translation prediction.

  1. other [Section 3.4 (Data Splits and Availability); Section 3.3 (Synthetic Data via Advanced RAG); Section 3.1 (Parallel Corpus Collection)]
    "Test and validation sets contain only human-annotated sentence pairs from "Дiдо Иванчiк" (Dido Yvanchik). ... The foundation of this pipeline was the dialectal novel "Дiдо Иванчiк" (Dido Yvanchik), which served as both the primary corpus for retrieval and the source of linguistic examples."

    The test set used to support the paper's central claim is drawn exclusively from the same novel that is the core of the manual training corpus (Section 3.1: "A significant portion of the dataset is based on the novel...") and that anchors the RAG pipeline generating 52,142 synthetic training pairs. Fine-tuned models are therefore evaluated on the text distribution they were trained to reproduce, so high BLEU/chrF++ and dialect-quality scores can be achieved by memorizing the novel's linguistic inventory. GPT-4o, by contrast, receives no such training. The claimed general standard-to-Hutsul translation advantage is thus not an independent prediction but a within-corpus fit to the shared source text.

full rationale

The only substantial circularity is in the evaluation design, not in the formal derivation or in citation practice. The paper contains no load-bearing self-citations and no definitional equivalence between fitted parameters and reported predictions. However, the central result—that small fine-tuned models outperform GPT-4o "across both automatic and LLM-evaluated metrics"—is measured on a test set consisting solely of human-annotated pairs from the novel Дiдо Иванчiк, while the same novel is the primary source of the manual training data and the retrieval/grammar base for the synthetic training data. The test therefore shares its distribution with the training input, making the headline comparison an in-domain memorization result rather than evidence of general Hutsul translation ability. The paper's Limitations section concedes that synthetic data lacks modern-domain coverage and that automatic metrics "may overestimate linguistic validity," but it does not acknowledge the shared-source test-set problem. This partial circularity warrants a score of 6: the comparison is not logically forced, but the central predictive claim reduces, in large part, to fitting the statistics of a single book.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the quality and independence of the test set. That independence is absent: the training, synthetic generation, and test all draw from the same single novel. The filtering thresholds for synthetic data are free parameters chosen by hand, and the evaluation relies on an LLM judge that is the same model family used to generate the synthetic data.

free parameters (4)
  • SequenceMatcher similarity threshold = 0.45
    Pairs below this character-similarity score are dropped from the synthetic set; the value is chosen empirically, Section 3.3.
  • Maximum unaligned source fraction (U-src) = <0.1
    Empirically defined threshold for filtering synthetic pairs, Section 3.3.
  • Maximum unaligned target fraction (U-tgt) = <0.1
    Empirically defined threshold for filtering synthetic pairs, Section 3.3.
  • Maximum crossing alignment fraction (X) = <0.2
    Empirically defined threshold for filtering synthetic pairs, Section 3.3.
assumptions (5)
  • domain assumption GPT-4o can generate fluent, correct Hutsul text when given transformation rules and retrieved examples
    The entire synthetic data pipeline assumes this, Section 3.3.
  • domain assumption The novel 'Дідо Иванчік' and its standard translation are representative of Hutsul and suffice as the basis for training and testing
    Used as the manual corpus, the grammar-rule source, the RAG index, and the only test source, Sections 3.1, 3.3, 3.4.
  • domain assumption Alignment statistics (U-src, U-tgt, X) are valid quality filters for synthetic sentence pairs
    Filtering thresholds are set from these statistics without validating that kept pairs are better, Section 3.3.
  • domain assumption GPT-4o zero-shot ratings of fluency, adequacy, and dialect quality are reliable substitutes for human evaluation
    Used as the primary evaluation; the authors acknowledge the limitation but proceed, Section 5.1.
  • ad hoc to paper The held-out test set is an unbiased sample of Hutsul translation
    Section 3.4 states test and validation contain only pairs from 'Дідо Иванчік', the same novel used to build training and synthetic data, so the assumption is not satisfied.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Vuyko Mistral: Adapting LLMs for Low-Resource Dialectal Translation." pith.science (2026). https://pith.science/paper/BGSKOX2J

@misc{pith2026250607617,
  author       = {Pith},
  title        = {Pith review of: Vuyko Mistral: Adapting LLMs for Low-Resource Dialectal Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BGSKOX2J}},
  note         = {Machine review of arXiv:2506.07617}
}
read the original abstract

In this paper we introduce the first effort to adapt large language models (LLMs) to the Ukrainian dialect (in our case Hutsul), a low-resource and morphologically complex dialect spoken in the Carpathian Highlands. We created a parallel corpus of 9852 dialect-to-standard Ukrainian sentence pairs and a dictionary of 7320 dialectal word mappings. We also addressed data shortage by proposing an advanced Retrieval-Augmented Generation (RAG) pipeline to generate synthetic parallel translation pairs, expanding the corpus with 52142 examples. We have fine-tuned multiple open-source LLMs using LoRA and evaluated them on a standard-to-dialect translation task, also comparing with few-shot GPT-4o translation. In the absence of human annotators, we adopt a multi-metric evaluation strategy combining BLEU, chrF++, TER, and LLM-based judgment (GPT-4o). The results show that even small(7B) finetuned models outperform zero-shot baselines such as GPT-4o across both automatic and LLM-evaluated metrics. All data, models, and code are publicly released at: https://github.com/woters/vuyko-hutsul

Figures

Figures reproduced from arXiv: 2506.07617 by the authors.

Figure 1
Figure 1. Map of Ukrainian dialects. The Hutsul dialect [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Overview of the synthetic data generation [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 2
Figure 2. Overview of the rules generation pipeline [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 27 canonical work pages

  1. [1]

    No \"e mi Aepli, Sarah Ebling, and Rico Sennrich. 2023. A benchmark for evaluating machine translation metrics on dialects without standard orthography. arXiv preprint arXiv:2311.16865

    work page Pith review arXiv 2023

  2. [2]

    Mistral AI. 2023. Introducing mistral 7b and mixtral. https://mistral.ai/news/mistral-7b/

    work page 2023

  3. [3]

    Rogier Blokland, Trond Trosterud, and Jack Rueter. 2024. https://aclanthology.org/2024.vardial-1.11 Morphological variants in north s 'a mi dialects . In Proceedings of the Tenth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial)

    work page 2024

  4. [4]

    Dmytro Chaplynskyi. 2023. https://aclanthology.org/2023.unlp-1.1 Introducing U ber T ext 2.0: A corpus of modern U krainian at scale . In Proceedings of the Second Ukrainian Natural Language Processing Workshop, pages 1--10, Dubrovnik, Croatia. Association for Computational Linguistics

    work page 2023

  5. [5]

    it's how you do things that matters

    Ned Cooper, Courtney Heldreth, and Ben Hutchinson. 2024. https://aclanthology.org/2024.eacl-short.19 “it's how you do things that matters”: Attending to process to better serve indigenous communities with language technologies . In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2024), page...

    work page 2024

  6. [6]

    G 'a bor Cs 'a ki and 1 others. 2023. https://aclanthology.org/2024.acl-long.777 Tokenizer retrofitting for morphologically rich languages . In Findings of the Association for Computational Linguistics: ACL 2023

    work page 2023

  7. [7]

    Chris Dyer, Victor Chahuneau, and Noah A. Smith. 2013. https://aclanthology.org/N13-1073/ A simple, fast, and effective reparameterization of IBM model 2 . In Proceedings of the 2013 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies , pages 644--648, Atlanta, Georgia. Association for Co...

    2013

  8. [8]

    Xavier Garcia and 1 others. 2024. https://arxiv.org/abs/2402.07836 Don't hallucinate, retrieve! a survey on retrieval-augmented text generation . arXiv preprint arXiv:2402.07836

    work page Pith review arXiv 2024

Show all 35 references
  1. [9]

    hutsul dialectal vocabulary in ukrainian belletristic language

    Vasyl Greshchuk. 2016. https://jgrst.donnu.edu.ua/article/view/3030 Models of word formation in hutsul dialects (based on the dictionary “hutsul dialectal vocabulary in ukrainian belletristic language”) . Gramatychni Studii, 6:272--286

    2016

  2. [10]

    Aditya Gudibande and 1 others. 2023. https://arxiv.org/abs/2305.11864 Synthetic data scaling for low-resource nlp . arXiv preprint arXiv:2305.11864

    2023 arXiv

  3. [11]

    Wolfgang Held and Dietrich Klakow. 2024. https://aclanthology.org/2024.kallm-1.7 Tada: Task-agnostic dialect adapters for multilingual transformers . In Proceedings of the First Workshop on Domain Adaptation for NLP

    2024

  4. [12]

    Nora Hollenstein, C 'e drick Fairon, and Julie Snyers. 2020. https://aclanthology.org/2020.lrec-1.353 German’s many voices: A corpus of regional variation in german . In Proceedings of the 12th Language Resources and Evaluation Conference

    2020

  5. [13]

    Edward J Hu and 1 others. 2021. https://arxiv.org/abs/2106.09685 Lora: Low-rank adaptation of large language models . arXiv preprint arXiv:2106.09685

    2021 arXiv

  6. [14]

    Torodd Kinn and Tor A. farli. 2024. https://aclanthology.org/2024.vardial-1.19 Exploring parallel machine translation for norwegian nynorsk and bokmål . In Proceedings of the Tenth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial)

    2024

  7. [15]

    Christo Kirov, Ryan Cotterell, and 1 others. 2023. https://aclanthology.org/2023.sigmorphon-1.12 Sigmorphon 2023 shared task: Morphological inflection in context . In Proceedings of the 20th SIGMORPHON Workshop

    2023

  8. [16]

    Artur Kiulian, Anton Polishko, Mykola Khandoga, Oryna Chubych, Jack Connor, Raghav Ravishankar, and Adarsh Arunkumar Shirawalmath. 2024. https://arxiv.org/abs/2404.09138 From bytes to borsch: Fine-tuning gemma and mistral for the ukrainian language representation . arXiv prepr...

    2024 arXiv

  9. [17]

    Zihao Li, Yucheng Shi, Zirui Liu, Fan Yang, Ninghao Liu, and Mengnan Du. 2024. https://arxiv.org/abs/2404.11553 Language ranker: A metric for quantifying llm performance across high and low-resource languages . arXiv preprint arXiv:2404.11553

    2024 arXiv

  10. [18]

    Zi Lin, Joel Tetreault, and 1 others. 2021. https://aclanthology.org/2021.emnlp-main.621 Multi-value: A multilingual, multi-dialect, and multi-task benchmark for language understanding . In Proceedings of EMNLP 2021

    2021

  11. [19]

    Xiang Liu and 1 others. 2024. https://arxiv.org/abs/2409.11404 Dada: Dynamic adapter aggregation for dialectal adaptation . arXiv preprint arXiv:2409.11404

    2024 arXiv

  12. [20]

    Ziwei et al. Liu. 2023. Gpt-4 as an automatic grader: An evaluation of zero-shot and few-shot prompting for text scoring tasks. arXiv preprint arXiv:2304.02329

    2023 arXiv

  13. [21]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311--318

    2002

  14. [22]

    Maja Popović. 2015. chrf: character n-gram f-score for automatic mt evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392--395

    2015

  15. [23]

    Alan Ramponi and Barbara Plank. 2021. https://aclanthology.org/2021.eacl-main.211 Neural multi-dialect language models for zero-shot cross-dialect transfer . In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume

    2021

  16. [24]

    Yves Scherrer. 2023. https://aclanthology.org/2024.vardial-1.17 Character alignment for dialect standardization: A comparative evaluation . In Proceedings of the 1st Workshop on NLP for Less-Resourced Languages

    2023

  17. [25]

    Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas, pages 223--231

    2006

  18. [26]

    Shahbaz Syed, Ahmad Dawar Hakimi, Khalid Al-Khatib, and Martin Potthast. 2023. https://aclanthology.org/2023.findings-emnlp.481 Quantifying the dialect gap and its correlates across languages . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5196--5210

    2023

  19. [27]

    Hugo Touvron, Thibaut Lavril, Alp Yurtsever, and 1 others. 2024. https://arxiv.org/abs/2404.14219 Llama 3: Open foundation and instruction models . arXiv preprint arXiv:2404.14219

    2024 arXiv

  20. [28]

    Peter Trudgill. 2003. https://aclanthology.org/W03-0320 Dialect contact and new-dialect formation: The inevitability of colonial englishes . In Proceedings of the 14th International Congress of Phonetic Sciences, pages 2193--2196

    2003

  21. [29]

    Yizhong et al. Wang. 2023. Llm-eval: Unified, automatic and robust evaluation of large language models with gpt-4. arXiv preprint arXiv:2305.03045

    2023 arXiv

  22. [30]

    Shijie Wu, Yuxuan Li, Cheng Li, Hao Zhu, and 1 others. 2023. https://openreview.net/pdf?id=Jtsk15wzv4 Benchmarking public large language models in low-resource settings . In Proceedings of the 2023 EMNLP

    2023

  23. [31]

    Marcos Zampieri, Tommi Jauhiainen, Nikola Ljubešić, Noëmi Aepli, Simon Clematide, and Jörg Tiedemann. 2024. https://aclanthology.org/2024.vardial-1.0 Overview of the vardial evaluation campaign 2024 . In Proceedings of the Tenth Workshop on NLP for Similar Languages, Varieties...

    2024

  24. [32]

    Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Ahmed Ali, and Stephan Vogel. 2017. https://aclanthology.org/W17-1212 Arabic dialect identification for the dsl 2017 shared task . In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial)

    2017

  25. [33]

    Tianyu Zhong, Ziqi Yang, Zhen Liu, Rui Zhang, Yiheng Liu, Hanqi Sun, Yujia Pan, Yiming Li, and Yifan Zhou. 2024. https://arxiv.org/abs/2412.04497 Opportunities and challenges of large language models for low-resource languages in humanities research . arXiv preprint arXiv:2412.04497

    2024 arXiv

  26. [34]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  27. [35]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.