Pith. sign in

REVIEW 4 major objections 3 minor 47 references

Lost in Variation? Evaluating NLI Performance in Basque and Spanish Geographical Variants

T0 review · 4 major / 3 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Including geographic dialect variants in NLI causes a statistically significant accuracy drop in language models — most sharply in Basque — and the drop reflects linguistic variation, not lexical overlap.

desk verdict XNLIvar is a genuinely useful new dialectal NLI resource for Basque and Spanish, but the paper's chi-square significance claims don't survive contact with its own table. read the letter →

arxiv 2506.15239 v2 pith:S2XEMCXQ submitted 2025-06-18 cs.CL

Jaione Bengoetxea , Itziar Gonzalez-Dios , Rodrigo Agerri This is my paper · ORCID
classification cs.CL
keywords naturallanguageinferencevariationdialectsBasqueSpanishcross-lingualtransferLLMevaluationXNLIvar
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to test whether current language models really understand regional language as it is actually spoken, rather than only the standardized written form. To do this, it introduces XNLIvar, a manually curated natural language inference dataset containing Basque sentences in Western, Central, and Navarrese dialects and Spanish sentences in Cuban, Ecuadorian, Andalusian, Tenerife, and Uruguayan variants, all built from the same native-language source sentences. Across encoder-only and generative models, the paper reports that accuracy drops consistently when dialectal variants replace standard-language test sentences, with the sharpest drop in Basque, and a chi-square test on parallel subsets puts the difference at $p < .001$ for both languages. The authors argue the drop is caused by linguistic variation itself: lexical overlap between premise and hypothesis stays the same, while accuracy still falls. A sympathetic reader would care because it gives a concrete, reusable benchmark for measuring whether language technology is inclusive of dialect speakers, and because it suggests standard benchmarks overstate model competence for real regional language use.

What carries the argument

The load-bearing object is XNLIvar, a parallel NLI evaluation set built by manually adapting the native Basque XNLIeunative sentences into three Basque dialectal variants and five Spanish variants, keeping every other property of the test identical. Its power comes from controlled comparison: the same source sentences appear in standard and variant form, so any accuracy difference is attributable to variation. The analysis machinery has three parts: Levenshtein distance quantifies how far each dialect lies from the standard; a "No repetitions" subset removes multiply-adapted sentences so the standard and variant sets are exactly parallel; and a chi-square test of independence measures whether the accuracy difference is statistically significant. An additional ablation removing Western or Central Basque duplicates isolates the influence of specific dialect groups. Together these pieces separate linguistic variation from lexical overlap and from dataset size.

What would settle it

Collect a fresh validation set by having a different, larger panel of native speakers, ideally including non-linguists, naturally retell or rewrite the same standard sentences in their own regional speech, then rerun the best XLM-RoBERTa and Gemma/Llama configurations; if accuracy no longer drops relative to the standard set, the reported effect is an artifact of the original annotator pool and adaptation style.

Watch

Extended reading notes

Core claim

The paper's central claim is that language models lose ground on natural language inference the moment test inputs shift from a standardized language to authentic geographical variants, and that this loss is a property of linguistic variation rather than a side effect of lexical similarity between premise and hypothesis. Starting from XNLIeunative, a native Basque NLI dataset, the authors created parallel dialectal versions in Basque and Spanish, then evaluated the same models in model-transfer, translate-train, and translate-test settings and with zero-shot, few-shot, Chain-of-Thought, and QA-prompted generative configurations. Accuracy falls on the variant sets in nearly every configuration, and the fall is statistically significant ($p < .001$) once duplicated dialectal instances are removed; for Basque the drop reaches about two to four percentage points for the best discriminative models and about six to ten points for generative models. Error analysis ties the largest Basque loss to the Western dialect, the most distant from the standard form according to Basque dialectology, while Spanish losses concentrate in variants written with non-standard orthography. The authors therefore conclude that current models under-process dialectal variation, and that XNLIvar can serve as a benchmark for measuring and eventually correcting this gap.

Load-bearing premise

The dialectal adaptations in XNLIvar are authentic, representative examples of the named Basque and Spanish geographical variants, despite being produced by only 12 Basque and 6 Spanish speakers, all with linguistics or NLP backgrounds.

Editorial extensions

If this is right

  • Standard-only NLI benchmarks overestimate how well current models understand real regional language use, because the same native sentences in dialect form score consistently lower.
  • The Western Basque dialect is the hardest of the well-represented Basque variants for encoder-only models, matching its greater linguistic distance from standard Basque.
  • For Spanish variants, accuracy losses concentrate in adaptations with non-standard orthography, so the bottleneck is orthographic rather than lexical or grammatical.
  • The statistically significant $p < .001$ difference on the parallel "No repetitions" subsets shows the drop is not an artifact of extra duplicated instances.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the effect survives a broader speaker sample, dialectal variation should become a first-class evaluation axis for NLP benchmarks in any language with geographic variants; the paper only claims the effect for Basque and Spanish, but nothing in the mechanism is language-specific.
  • Because the Spanish variants were produced by machine-translating Basque source sentences before manual adaptation, the reported Spanish drop may understate sensitivity to variation found in Spanish-native texts; a native-source Spanish control would be a sharper test.
  • The orthography finding suggests a testable preprocessing fix the paper does not run: normalize non-standard Spanish orthography (for example, turning "mihma" into "misma") and check whether the accuracy gap closes.
  • The released speaker metadata may let future work connect model errors to sociolinguistic variables such as age and gender, since the annotator profiles are recorded.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper presents XNLIvar, a manually curated parallel NLI dataset that adds Basque and Spanish geographic variants to the XNLIeunative resource. It evaluates encoder-only models (XLM-RoBERTa, mDeBERTa) in model-transfer, translate-train, and translate-test settings, and decoder-only LLMs (Llama-3.1, Gemma 2) in several prompting formulations. The main empirical claim is that accuracy drops when dialectal variants are included, especially in Basque, and that this drop reflects linguistic variation rather than lexical overlap; Western Basque is singled out as particularly challenging. The authors also report chi-square tests in Section 6 claiming highly significant differences on parallel 'No repetitions' subsets.

Significance. If the empirical claims held up, this would be a valuable contribution: the dataset is, to my knowledge, the first manually curated variation-inclusive NLI benchmark for these language varieties, and the evaluation covers a sensible range of model families and training regimes. The public release of data and code is commendable. However, the headline statistical evidence is internally inconsistent: the reported p-values cannot be reproduced from the accuracies in Table 6. There are also unaddressed confounds related to label preservation and the selection of the 'No repetitions' subset. The resource itself is potentially useful, but the central claim about the cause and significance of the observed drop is not currently established by the data as presented.

major comments (4)
  1. [Section 6, Table 6 (No repetitions)] The chi-square tests reported for the 'No repetitions' splits are not reproducible from the table. For Basque, accuracies of 75.63% (≈470/621) and 71.77% (≈446/621) yield a 2x2 chi-square statistic of about 2.39 (p≈0.12), not p<.001. For Spanish, accuracies of 74.61% (≈463/621) and 73.00% (≈453/621) yield χ²≈0.42 (p≈0.52). The generative Spanish comparison (77.29 vs 77.13 on 621 items) is trivially non-significant. The claim in Section 5.1 that 'language models perform worse when variants are included' and the Section 6 conclusion that the decline is 'highly statistically significant' therefore rest on a test that is either misapplied or misreported. Please provide the actual contingency tables for any statistical test (e.g., a McNemar test for paired data, if appropriate) and correct the claims accordingly.
  2. [Section 3.1, Appendix A] The adaptation guidelines instruct informants to rewrite sentences in their dialect, but the paper does not report any verification that the dialectal reformulations preserve the original NLI label (entailment/contradiction/neutral). If a rewrite changes the semantic relation between premise and hypothesis, the gold label becomes noisy, and the observed accuracy drop might reflect label corruption rather than difficulty with dialectal variation. Please either add a label-preservation validation step (e.g., a second annotation or automatic check) or explicitly discuss this as a limitation.
  3. [Section 6, Table 6 (construction of No repetitions)] For premises with multiple dialectal rewrites, the procedure for selecting which variant is retained in the 'No repetitions' split is not described. This selection can bias the accuracy estimate and hence the comparison with XNLIeunative. Please specify the selection rule and, if feasible, assess robustness by repeating the analysis over different selections or by reporting the full distribution of accuracies across retained variants.
  4. [Section 5.1, Table 13; Section 7] The claim that encoder-only models 'particularly struggle with Western Basque' is not strongly supported by the reported numbers. In the translate-train setting (Table 13), the lowest accuracy is for Navarrese (71.43), not Western (73.75), and the Western-Central difference is only about 0.7 points. The ablation in Table 6 shows a 0.44-point difference between removing Western and removing Central repeated instances, with no significance test. Please qualify the claim or provide statistical evidence (e.g., a paired test across the retained premise set) that Western examples are significantly harder than Central ones.
minor comments (3)
  1. [Table 4] Several cells in the PDF rendering appear to run together (e.g., '83.4275.6373.21'); please ensure the table is formatted with clear separation between values.
  2. [Figure 1] The dual-axis plots with Levenshtein distance and accuracy on the same figure may be visually misleading; consider normalizing the two quantities or using separate panels to make the relationship easier to assess.
  3. [Section 3.1] Minor naming consistency: the paper alternates between 'XNLIes var' and 'XNLIesvar'; please use the latter uniformly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the empirical claim is a direct measurement on independently constructed data splits.

full rationale

The central claim that NLI accuracy drops when geographical variants are included is an empirical measurement: models are fine-tuned on standard MNLI/translated training data and evaluated on standard versus variation-inclusive test sets. No parameter is fitted to the comparison, no equation defines the variant accuracy in terms of the standard accuracy, and no uniqueness theorem or prior result by the same authors is invoked to force the conclusion. The seed dataset XNLIeunative (Heredia et al., 2024) is an external, publicly available resource, and the new variant corpora are human adaptations of those premises; using a prior public dataset as the testbed is data reuse, not a self-citation that carries the argument. Section 5.1's observed drops and Section 6's chi-square tests are reported from the paper's own splits, and the statistical inconsistency flagged by the skeptic (Table 6 counts do not evidently support p<.001) is a correctness/reproducibility concern, not a case of prediction-by-construction or a definitional equivalence. The Limitations passage about 12 Basque and 6 Spanish annotators is an external-validity caveat about representativeness and does not indicate that the derivation reduces to its own inputs. No circular step can be quoted, so no circularity is identified.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claims rest on assumptions about data authenticity, label preservation under paraphrase, the adequacy of Levenshtein distance as a proxy for linguistic distance and lexical overlap, and the representativeness of the chosen models. There are no fitted numeric constants in the paper.

assumptions (4)
  • domain assumption The NLI label (entailment/contradiction/neutral) is preserved when standard sentences are rewritten into dialectal variants.
    Annotators were instructed to adapt sentences to dialectal forms (Appendix A), but there is no explicit check that the logical relation between premise and hypothesis was preserved after adaptation. If the dialectal rewrite changes meaning, the gold labels in XNLIvar would be wrong, and the accuracy drop could partly reflect label noise.
  • domain assumption XNLIeunative is a valid source of native Basque NLI data for generating dialectal variants, and the automatic translation of this data into Spanish followed by manual review produces authentic Spanish prompts.
    The Spanish variant dataset is derived by machine translation of Basque native sentences, not from original Spanish text. Manual review was performed, but the provenance could introduce translationese or non-native Spanish patterns, affecting the Spanish results.
  • domain assumption Character-level Levenshtein distance is an adequate proxy for both dialectal distance from standard and lexical overlap between premise and hypothesis.
    Section 6 uses Levenshtein distance to conclude that lexical overlap is not the cause of the accuracy drop and that Western Basque is the most distant dialect. For agglutinative, morphologically rich Basque, character edit distance may not capture morphological or syntactic variation.
  • domain assumption The models tested (XLM-RoBERTa, mDeBERTa, Llama-3.1, Gemma-2) are representative of current language technologies for Basque and Spanish.
    The paper generalizes from two encoder-only and four generative model families. If other model classes behave differently, the conclusion that language models struggle with variation would be overstated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Lost in Variation? Evaluating NLI Performance in Basque and Spanish Geographical Variants." pith.science (2026). https://pith.science/paper/S2XEMCXQ

@misc{pith2026250615239,
  author       = {Pith},
  title        = {Pith review of: Lost in Variation? Evaluating NLI Performance in Basque and Spanish Geographical Variants},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S2XEMCXQ}},
  note         = {Machine review of arXiv:2506.15239}
}
read the original abstract

In this paper, we evaluate the capacity of current language technologies to understand Basque and Spanish language varieties. We use Natural Language Inference (NLI) as a pivot task and introduce a novel, manually-curated parallel dataset in Basque and Spanish, along with their respective variants. Our empirical analysis of crosslingual and in-context learning experiments using encoder-only and decoder-based Large Language Models (LLMs) shows a performance drop when handling linguistic variation, especially in Basque. Error analysis suggests that this decline is not due to lexical overlap, but rather to the linguistic variation itself. Further ablation experiments indicate that encoder-only models particularly struggle with Western Basque, which aligns with linguistic theory that identifies peripheral dialects (e.g., Western) as more distant from the standard. All data and code are publicly available.

Figures

Figures reproduced from arXiv: 2506.15239 by the authors.

Figure 1
Figure 1. Standard to dialectal Levenshtein distance vs accuracy of best discriminative models. [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Levenshtein distance from premise to hypothesis and accuracy of discriminative models. [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Geographical variant label representation in XNLIvar [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Spanish accuracy results and its correlation to types of linguistic adaptations. [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 23 canonical work pages

  1. [1]

    No \"e mi Aepli and Rico Sennrich. 2022. https://doi.org/10.18653/v1/2022.findings-acl.321 Improving zero-shot cross-lingual transfer between closely related languages by injecting character-level noise . In Findings of the Association for Computational Linguistics: ACL 2022, pages 4074--4083, Dublin, Ireland. Association for Computational Linguistics

    doi:10.18653/v1/2022.findings-acl.321 2022

  2. [2]

    Rodrigo Agerri, I \ n aki San Vicente, Jon Ander Campos, Ander Barrena, Xabier Saralegi, Aitor Soroa, and Eneko Agirre. 2020. https://aclanthology.org/2020.lrec-1.588/ Give your text representation models some love: the case for B asque . In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4781--4788, Marseille, France. Europ...

    work page 2020

  3. [3]

    Md Mahfuz Ibn Alam, Sina Ahmadi, and Antonios Anastasopoulos. 2024. https://aclanthology.org/2024.findings-eacl.125/ CODET : A benchmark for contrastive dialectal evaluation of machine translation . In Findings of the Association for Computational Linguistics: EACL 2024, pages 1790--1859, St. Julian ' s, Malta. Association for Computational Linguistics

    work page 2024

  4. [4]

    Tahani Alshareef and Muazzam Ahmed Siddiqui. 2020. https://doi.org/10.1109/ACIT50332.2020.9300059 A seq2seq neural network based conversational agent for gulf arabic dialect . In 2020 21st International Arab Conference on Information Technology (ACIT), pages 1--7

    arXiv 2020

  5. [5]

    Ekaterina Artemova and Barbara Plank. 2023. https://aclanthology.org/2023.nodalida-1.39/ Low-resource bilingual dialect lexicon induction with large language models . In Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa), pages 371--385, T \'o rshavn, Faroe Islands. University of Tartu Library

    work page 2023

  6. [6]

    Mikel Artetxe, Itziar Aldabe, Rodrigo Agerri, Olatz Perez-de Vi \ n aspre, and Aitor Soroa. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.499 Does corpus quality really matter for low-resource languages? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7383--7390, Abu Dhabi, United Arab Emirates. Associa...

    doi:10.18653/v1/2022.emnlp-main.499 2022

  7. [7]

    Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.618 Translation artifacts in cross-lingual transfer learning . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7674--7684, Online. Association for Computational Linguistics

    doi:10.18653/v1/2020.emnlp-main.618 2020

  8. [8]

    Ari Ball-Burack, Michelle Seng Ah Lee, Jennifer Cobbe, and Jatinder Singh. 2021. https://doi.org/10.1145/3442188.3445875 Differential tweetment: Mitigating racial dialect bias in harmful tweet detection . In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT '21, page 116–128, New York, NY, USA. Association for Com...

    arXiv 2021

Show all 47 references
  1. [9]

    Galo Castillo-l \'o pez, Arij Riabi, and Djam \'e Seddah. 2023. https://doi.org/10.18653/v1/2023.vardial-1.1 Analyzing zero-shot transfer scenarios across S panish variants for hate speech detection . In Tenth Workshop on NLP for Similar Languages, Varieties and Dialects (VarD...

    2023 doi

  2. [10]

    Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm \'a n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. https://doi.org/10.18653/v1/2020.acl-main.747 Unsupervised cross-lingual representation learning ...

    2020 doi

  3. [11]

    Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. https://doi.org/10.18653/v1/D18-1269 XNLI : Evaluating cross-lingual sentence representations . In Proceedings of the 2018 Conference on Empirical Methods ...

    2018 doi

  4. [12]

    Eugenio Coseriu. 1956. La geograf \' a ling \"u \' stica , volume 11. Universidad de la Rep \'u blica, Facultad de Humanidades y Ciencias

    1956

  5. [13]

    Javier De la Rosa, \'A lvaro Cu \'e llar, and J \"o rg Lehmann. 2024. The modernifa project: orthographic modernization of spanish golden age dramas with language models. Anuario Lope de Vega Texto literatura cultura, 30:410--425

    2024

  6. [14]

    Dorottya Demszky, Devyani Sharma, Jonathan Clark, Vinodkumar Prabhakaran, and Jacob Eisenstein. 2021. https://doi.org/10.18653/v1/2021.naacl-main.184 Learning to recognize dialect features . In Proceedings of the 2021 Conference of the North American Chapter of the Association...

    2021 doi

  7. [15]

    Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aur \'e lien Rodriguez, (...), and Zhiwei Zhao

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony S. Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston ...

    2024 arXiv

  8. [16]

    Cristina Espa \ n a-Bonet and Alberto Barr \'o n-Cede \ n o. 2024. https://doi.org/10.18653/v1/2024.naacl-long.204 Elote, choclo and mazorca: on the varieties of S panish . In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational...

    2024 doi

  9. [17]

    Ainara Estarrona, Izaskun Etxeberria, Ricardo Etxepare, Manuel Padilla-Moyano, and Ander Soraluze. 2020. https://aclanthology.org/2020.vardial-1.8/ Dealing with dialectal variation in the construction of the B asque historical corpus . In Proceedings of the 7th Workshop on NLP...

    2020

  10. [18]

    Ainara Estarrona, Izaskun Etxeberria, Manuel Padilla-Moyano, and Ander Soraluze. 2023. Measuring language distance for historical texts in basque. Procesamiento del Lenguaje Natural, 70:53--61

    2023

  11. [19]

    Julen Etxaniz, Oscar Sainz, Naiara Miguel, Itziar Aldabe, German Rigau, Eneko Agirre, Aitor Ormazabal, Mikel Artetxe, and Aitor Soroa. 2024. Latxa: An open language model and evaluation suite for B asque. In Proceedings of the 62nd Annual Meeting of the Association for Computa...

    2024

  12. [20]

    Fahin Faisal, Orevaoghene Ahia, Aarohi Srivastava, Kabir Ahuja, David Chiang, Yulia Tsvetkov, and Antonios Anastasopoulos. 2024. https://api.semanticscholar.org/CorpusID:268513057 Dialectbench: A nlp benchmark for dialects, varieties, and closely-related languages . ArXiv, abs...

    2024 arXiv

  13. [21]

    Blanca Calvo Figueras, Eneko Sagarzazu, Julen Etxaniz, Jeremy Barnes, Pablo Gamallo, Iria De Dios Flores, and Rodrigo Agerri. 2025. Truth knows no language: Evaluating truthfulness beyond english. In Proceedings of the 63rd Annual Meeting of the Association for Computational L...

    2025

  14. [22]

    Iker Garc \' a-Ferrero, Rodrigo Agerri, and German Rigau. 2022. Model and data transfer for cross-lingual sequence labelling in zero-resource settings. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 6403--6416

    2022

  15. [23]

    Hany Hassan, Mostafa Elaraby, and Ahmed Y. Tawfik. 2017. https://aclanthology.org/2017.iwslt-1.12/ Synthetic data for neural machine translation of spoken-dialects . In Proceedings of the 14th International Conference on Spoken Language Translation, pages 82--89, Tokyo, Japan....

    2017

  16. [24]

    Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2021. https://arxiv.org/abs/2111.09543 Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing . CoRR, abs/2111.09543

    2021 arXiv

  17. [25]

    Maite Heredia, Julen Etxaniz, Muitze Zulaika, Xabier Saralegi, Jeremy Barnes, and Aitor Soroa. 2024. https://doi.org/10.18653/v1/2024.naacl-long.234 XNLI eu: a dataset for cross-lingual NLI in B asque . In Proceedings of the 2024 Conference of the North American Chapter of the...

    2024 doi

  18. [26]

    Dirk Hovy and Diyi Yang. 2021. https://doi.org/10.18653/v1/2021.naacl-main.49 The importance of modeling social factors of language: Theory and practice . In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Huma...

    2021 doi

  19. [27]

    Aditya Joshi, Raj Dabre, Diptesh Kanojia, Zhuang Li, Haolan Zhan, Gholamreza Haffari, and Doris Dippold. 2024. https://arxiv.org/abs/2401.05632 Natural language processing for dialects of a language: A survey . Preprint, arXiv:2401.05632

    2024 arXiv

  20. [28]

    Olli Kuparinen, Aleksandra Mileti \'c , and Yves Scherrer. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.923 Dialect-to-standard normalization: A large-scale multilingual evaluation . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13814...

    2023 doi

  21. [29]

    William Labov. 2006. The Social Stratification of English in New York City, 2 edition. Cambridge University Press

    2006

  22. [30]

    Emmy Liu, Chenxuan Cui, Kenneth Zheng, and Graham Neubig. 2022. Testing the ability of language models to interpret figurative language. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technol...

    2022

  23. [31]

    Lopetegui, Arij Riabi, and Djam \'e Seddah

    Javier A. Lopetegui, Arij Riabi, and Djam \'e Seddah. 2025. https://aclanthology.org/2025.vardial-1.13/ Common ground, diverse roots: The difficulty of classifying common examples in S panish varieties . In Proceedings of the 12th Workshop on NLP for Similar Languages, Varieti...

    2025

  24. [32]

    Gemma Team Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, L. Sifre, Morgane Rivi \`e re, Mihir Kale, J Christopher Love, Pouya Dehghani Tafti, L'eonard Hussenot, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ros, Am...

    2024 arXiv

  25. [33]

    Julio de Urquijo

    Luis Mitxelena. 1981. Lengua com \'u n y dialectos vascos. Anuario del Seminario de Filolog \' a Vasca" Julio de Urquijo" , 15:289--313

    1981

  26. [34]

    Aakanksha Naik, Abhilasha Ravichander, Norman Sadeh, Carolyn Rose, and Graham Neubig. 2018. Stress test evaluation for natural language inference. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2340--2353

    2018

  27. [35]

    Alan Ramponi and Camilla Casula. 2023. https://doi.org/10.18653/v1/2023.vardial-1.19 D iatop I t: A corpus of social media posts for the study of diatopic language variation in I taly . In Tenth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2023), page...

    2023 doi

  28. [36]

    Elisa Sanchez-Bayona and Rodrigo Agerri. 2024. Meta4xnli: A crosslingual parallel corpus for metaphor detection and interpretation. arXiv, 2404.07053

    2024 arXiv

  29. [37]

    Kevin Stowe, Prasetya Utama, and Iryna Gurevych. 2022. IMPLI : Investigating NLI models ' performance on figurative language. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5375--5388

    2022

  30. [38]

    Larraitz Uria and Ricardo Etxepare. 2012. Hizkeren arteko aldakortasun sintaktikoa aztertzeko metodologiaren nondik norakoak: Basyque aplikazioa. Lapurdum. Euskal ikerketen aldizkaria| Revue d' \'e tudes basques| Revista de estudios vascos| Basque studies review , (16):117--135

    2012

  31. [39]

    Francisco Valentini, Viviana Cotik, Dami \'a n Ariel Furman, Ivan Bercovich, Edgar Altszyler, and Juan Manuel P'erez. 2024. https://api.semanticscholar.org/CorpusID:272550762 Messirve: A large-scale spanish information retrieval dataset . ArXiv, abs/2409.05994

    2024

  32. [40]

    Vered Volansky, Noam Ordan, and Shuly Wintner. 2013. https://doi.org/10.1093/llc/fqt031 On the features of translationese . Digital Scholarship in the Humanities, 30(1):98--118

    2013 doi

  33. [41]

    Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. https://doi.org/10.18653/v1/N18-1101 A broad-coverage challenge corpus for sentence understanding through inference . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computation...

    2018 doi

  34. [42]

    Marcos Zampieri, Preslav Nakov, and Yves Scherrer. 2020. https://doi.org/10.1017/S1351324920000492 Natural language processing for similar languages, varieties, and dialects: A survey . Natural Language Engineering, 26(6):595–612

    2020 doi

  35. [43]

    Marcos Zampieri, Kai North, Tommi Jauhiainen, Mariano Felice, Neha Kumari, Nishant Nair, and Yash Mahesh Bangera. 2024. https://aclanthology.org/2024.lrec-main.882/ Language variety identification with true labels . In Proceedings of the 2024 Joint International Conference on ...

    2024

  36. [44]

    Koldo Zuazu. 2008. Euskalkiak. Euskararen dialektoak. Elkar

    2008

  37. [45]

    Koldo Zuazu. 2015. The unification of the basque language. http://euskalkiak.eus/en/euskararen_batasuna.php. Accessed: 09-06-2025

    2015

  38. [46]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  39. [47]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.