Pith. sign in

REVIEW 2 major objections 5 minor 117 references

Different Perturbations, Different Mechanisms: Understanding Continued Pre-training for Zero-Shot Dialect Robustness

T0 review · 2 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Character-noised continued pre-training reliably improves zero-shot dialect robustness in multilingual LLMs, and similar downstream scores can arise from distinct mechanisms.

desk verdict The practical finding is solid—character-level noise is a strong default for robustness-focused CPT—but the 'distinct mechanisms' claim is statistically underspecified and needs a closer look before it can be trusted. read the letter →

arxiv 2608.05510 v1 pith:SVCSTTVC submitted 2026-08-06 cs.CL

Aarohi Srivastava , David Chiang This is my paper · ORCID
classification cs.CL
keywords dialectrobustnesscontinuedpre-trainingperturbation-basedCPTcharacter-levelnoisezero-shotmultilinguallanguagemodelsrepresentationalignmentpredictionrepair
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that perturbation-based continued pre-training (CPT) is a dependable way to make multilingual LLMs more robust to dialects they have never been trained on, and that different perturbation recipes achieve that robustness through different internal routes. Across nine German, Italian, and Arabic tasks and six training conditions, 43 of 45 dialect evaluations improve over the base model, with an average gain of 3.0 points on dialect data and a small net gain on standard varieties. Character-level noise is the strongest single strategy, but the more general point is that similar benchmark numbers can hide different mechanisms: clean and token-level perturbations mostly improve language-model fit, while character-level and phonological perturbations mostly increase the similarity between standard and dialect representations. The paper therefore reframes perturbation-based CPT as a family of adaptation strategies, not a single intervention, and gives practitioners a principled basis for choosing among them.

What carries the argument

The machinery is a controlled comparison of six CPT conditions with calibrated perturbation strength, plus two mechanistic probes. Bits per character (BPC) measures how well each checkpoint predicts natural text; standard–dialect cosine similarity measures how similarly the model represents paired standard and dialect sentences. The paper also tracks prediction transitions between BASE and each CPT model, defining the net repair rate as corrected BASE errors minus newly introduced errors, and correlates each mechanistic variable with downstream dialect gains to attribute credit. This combination separates perturbation type from perturbation severity and links benchmark improvements to specific behavioral and representational changes.

What would settle it

Recompute Figure 2's correlations using per-seed or per-dialect observations rather than three seed-averaged settings and test whether CHAR's downstream gains still track standard–dialect cosine similarity while CLEAN's track bits per character; the paper reports p-values but never states the sample size, and Tables 13 and 14 report the mechanistic variables for only three parallel settings. A behavioral control would run CHAR on a dialect pair whose divergence is lexical rather than orthographic: the mechanism account predicts a smaller gain.

Watch

Extended reading notes

Core claim

The central claim is that perturbation-based continued pre-training, especially character-noised CPT, reliably improves zero-shot dialect robustness in decoder-only LLMs while largely preserving standard-variety performance, and that methods with nearly equal downstream accuracy can be robust for different reasons. The authors compare BASE, CLEAN, BPE dropout, subword replacement, character noise, and a phonological noiser on German, Italian, and Arabic, training linear probes on frozen representations to isolate the effect of CPT. CHAR achieves the best dialect-average score in six of nine language–task settings and the highest net repair rates, while CLEAN, BPE-DROP, and SUB-REP correlate with improved language-model fit and CHAR and PHON correlate with increased standard–dialect cosine similarity. They conclude that robustness from synthetic surface variation is not one phenomenon, and that languages with larger standard–dialect tokenization gaps, such as German and Italian, offer more room for these interventions than Arabic.

Load-bearing premise

The distinct-mechanisms conclusion depends on correlations computed over only three language settings (German intent, Italian intent, Arabic topic); with so few data points, a different selection of dialects could change which mechanism looks responsible.

Editorial extensions

If this is right

  • Character-level noise is the strongest general-purpose CPT choice for zero-shot dialect robustness, winning six of nine language–task settings in this study.
  • Similar benchmark scores are not evidence of the same intervention effect: clean and token-level CPT correlate with better language-model fit, while character and phonological CPT correlate with higher standard–dialect representation similarity.
  • CPT gains come primarily from repairing baseline errors rather than introducing new ones, with CHAR showing the largest net repair rates.
  • Languages with larger standard–dialect tokenization gaps, such as German and Italian, offer more room for perturbation-based CPT than Arabic, whose dialect surface forms are closer to the standard.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference, not the paper's: pairing a fit-improving CPT (clean or token-level) with an alignment-improving CPT (character or phonological noise) may combine the two mechanisms; the paper does not test combined strategies.
  • My inference: because all downstream evaluations freeze the LLM and train linear probes, the mechanism split describes representation quality for discriminative tasks; the paper itself limits its conclusions to those settings, so open-ended generation remains an untested extension.
  • My inference: the orthographic-divergence account predicts character noise should transfer best to other spelling-heavy dialect pairs, for example Scots or Latin-script dialectal Arabic, and worse where dialect divergence is mostly lexical or syntactic.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper presents a systematic comparison of six continued pre-training (CPT) conditions—BASE, CLEAN, BPE-DROP, SUB-REP, CHAR, and PHON—applied to Llama-2-7B and, in a subset, Qwen2-7B, evaluating zero-shot dialect robustness on German, Italian, and Arabic tasks. The authors report that perturbation-based CPT, especially character-level noise (CHAR), consistently improves dialect performance while largely preserving standard-variety accuracy. They further claim that methods with similar downstream performance achieve robustness through distinct mechanisms, evidenced by different correlations between downstream gains and two mechanistic variables: bits-per-character (BPC) improvement and standard–dialect cosine similarity increase. The paper includes auxiliary analyses of prediction repair, tokenization statistics, and cross-linguistic differences.

Significance. If the mechanism claim is adequately supported, the paper would be a valuable contribution to the growing literature on dialect robustness: it moves beyond simple benchmark comparisons to ask how different synthetic perturbations induce robustness, and it offers practical guidance for selecting CPT strategies. Strengths include the controlled comparison of multiple perturbation families on a fixed architecture and training setup, calibration of perturbation strengths to match tokenization statistics, evaluation across three typologically distinct languages and multiple tasks, and a second model family as a generalization check. The prediction-repair analysis provides a useful additional lens. However, the central novelty—the 'different mechanisms' claim—currently rests on correlational evidence whose statistical basis is not transparent or reproducible, which limits the paper's significance until that evidence is properly reported and verified.

major comments (2)
  1. [§5.1 and Contribution 2] The paper claims that '43 of 45 language–task–method comparisons' outperform BASE, but Table 3 does not support this count. A careful count from Table 3 yields 42 comparisons above BASE: there are two ties (German NER CLEAN at 42.7 = 42.7, Italian NER PHON at 51.6 = 51.6) and one below-base result (Arabic NER PHON at 50.1 vs. BASE 50.3). The qualitative conclusion that CPT reliably improves dialect robustness remains intact, but the stated numerical result is inaccurate and must be corrected.
  2. [§5.3, Figure 2, Tables 13 and 14] The 'distinct mechanisms' claim—the paper's central novelty—relies on Pearson correlations reported in Figure 2, but the manuscript never states the number of observations entering these correlations. Tables 13 and 14 report BPC and cosine similarity for only three language–setting aggregates (German intent, Italian intent, Arabic topic). If the correlations are computed over these three points, the reported p-values (e.g., r=0.810, p=.003 for CHAR) are arithmetically impossible under the standard t-test with n=3 (df=1, p would be approximately 0.40). The p-values are roughly consistent with n≈11 for CHAR and PHON and n≈33 for CLEAN/BPE/SUB-REP, suggesting the analysis might have been performed at the per-dialect or per-seed level, but the paper does not say so, nor does it report the per-dialect BPC/cosine values needed to reproduce the correlations. Without a clear statement of the unit of analysis, the number of observations, and the underlying data, the 'different mechanisms' claim is not reproducible from the manuscript as written.
minor comments (5)
  1. [Table 13 caption] The caption refers to 'NOISY' but the corresponding CPT condition is called 'CHAR' throughout the text; please reconcile the terminology.
  2. [Tables 3 and throughout] The column header 'BPE' is used in Table 3 but the method is introduced as 'BPE-DROP' in Section 3.1; use a single consistent abbreviation.
  3. [Abstract] The phrase 'nine German, Italian, and Arabic dialect tasks' is ambiguous: there are three languages and nine tasks total. Reword to 'nine dialect tasks across German, Italian, and Arabic' for clarity.
  4. [§5.3 and Figure 2] For each reported correlation, please include the sample size (n) either in the text or in the figure caption, and consider showing the scatter plots so that the relationship is visually verifiable.
  5. [Table 14 caption] The caption states values are 'macro-averaged over dialect varieties,' but for Italian intent detection there is only one dialect variety (Neapolitan); clarify whether 'macro' is meaningful for that row.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical comparison and correlational analysis; no fitted input is renamed as a prediction and no derivation reduces to its own inputs.

full rationale

This paper is an empirical benchmarking and mechanism-correlation study; it contains no derivation chain that could collapse into its inputs. The perturbation conditions (CHAR, BPE-DROP, SUB-REP, PHON, CLEAN) are defined independently and downstream dialect robustness is measured on held-out test sets with classifiers trained only on standard-variety data. Perturbation severities are calibrated to match tokenization statistics or a fixed replacement probability, which controls a nuisance variable rather than forcing any downstream outcome, improvement, or mechanism ranking. The mechanism claims in Figure 2 correlate measured downstream improvement over BASE with measured BPC improvement and measured standard–dialect cosine-similarity increase; none of these variables is fitted to the others, so there is no reduction by construction. The only self-citations (Srivastava and Chiang 2023a,b) supply prior character-noise recipes and hyperparameters, and they are not invoked as load-bearing evidence for the central claims about which method performs best or why. The skeptic's concern about unreported sample sizes and p-values in Figure 2, even if valid, is a statistical-reporting and evidentiary issue, not a circularity: an unsupported or underpowered correlation is still not equivalent to its own input by definition. No passage in the manuscript exhibits a fitted parameter renamed as a prediction, a result forced by a self-citation chain, or a definition that presupposes the target claim. A non-finding is therefore appropriate.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on experimental design choices (perturbation calibration, LoRA hyperparameters) and domain assumptions about the validity of linear-probe evaluation and mechanism proxies. No new theoretical entities are introduced.

free parameters (6)
  • BPE dropout max probability = DE 0.23, IT 0.24, AR 0.24
    Calibrated per language to match token inflation of CHAR; chosen by hand, not from literature.
  • DialUp phonological theta max = DE 0.45, IT 0.30, AR 0.25
    Calibrated per language to match tokenization changes of CHAR.
  • SUB-REP replacement probability = 0.05 fixed
    Chosen to produce comparable perturbation level to CHAR; not derived.
  • CHAR word-level corruption rate = sampled p~U(0,1), average 0.25 observed
    Sampling distribution chosen following Kojima et al. (2025).
  • LoRA rank and alpha = r=16, alpha=16
    Chosen based on Kojima et al. (2025).
  • Training steps = 2500
    Selected by loss plateau, not downstream performance.
assumptions (4)
  • domain assumption mC4 primarily contains standard-variety web text
    Section 3 states dialectal content in mC4 is incidental, so synthetic perturbations are the main source of variation. If mC4 contains substantial dialectal text, the control conditions are confounded.
  • domain assumption Linear probes on frozen final-layer representations isolate the effect of CPT
    Section 4: evaluation uses logistic regression on hidden representations; assumes this captures robustness-relevant behavior better than prompt-based evaluation.
  • domain assumption BPC and standard-dialect cosine similarity are valid mechanistic measures
    Section 5.3 uses these as proxies for language model fit and representational alignment.
  • domain assumption Calibration to tokenization statistics makes perturbation severities comparable
    Section 3.2: matching token inflation is assumed to equate perturbation severity across strategies.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Different Perturbations, Different Mechanisms: Understanding Continued Pre-training for Zero-Shot Dialect Robustness." pith.science (2026). https://pith.science/paper/SVCSTTVC

@misc{pith2026260805510,
  author       = {Pith},
  title        = {Pith review of: Different Perturbations, Different Mechanisms: Understanding Continued Pre-training for Zero-Shot Dialect Robustness},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SVCSTTVC}},
  note         = {Machine review of arXiv:2608.05510}
}
read the original abstract

Dialectal variation remains a major challenge for multilingual language models. Perturbation-based continued pre-training (CPT) has emerged as a promising approach to improving robustness, yet existing work largely evaluates individual perturbation strategies in isolation and provides limited insight into why they work. We present a systematic study of perturbation-based CPT for multilingual dialect robustness in LLMs, comparing six training conditions across nine German, Italian, and Arabic dialect tasks. Perturbation-based CPT, especially character-noised CPT, consistently improves zero-shot dialect robustness while largely preserving standard variety performance. More importantly, we show that methods with similar downstream performance induce distinct mechanisms of robustness, exhibiting different patterns of language model adaptation, representational alignment, and prediction repair. Our results provide a more complete understanding of how synthetic surface variation improves robustness and offer practical guidance for selecting CPT strategies in multilingual and dialectal settings.

Figures

Figures reproduced from arXiv: 2608.05510 by the authors.

Figure 1
Figure 1. Average improvement over BASE across all language–task settings. Continued pre-training consistently improves dialect performance, with CHAR producing the largest gains in most settings while leaving standard variety performance largely unchanged. Qwen To assess whether these conclusions gen￾eralize beyond a single model family, we repeat a subset of the experiments on Qwen2-7B3 using BASE, CLEAN, and CHAR ( [PITH_… view at source ↗
Figure 2
Figure 2. Associations between mechanistic changes and downstream improvement. Each cell reports the Pearson correlation between improvement over BASE and either bits-per-character (BPC) improvement or in￾creased standard–dialect representation similarity. Sig￾nificant correlations are indicated by asterisk. larity, followed closely by PHON. The contrast is particularly pronounced for German, while Ara￾bic begins from a much … view at source ↗
Figure 3
Figure 3. Prediction transitions relative to BASE. Net repair rate is the difference between wrong to correct transitions (base to post-CPT) and correct to wrong. Across all settings except slot filling, all CPT modes repair substantially more errors than introduced. representational similarity (r = 0.703, p = .016) but not language model fit (r = 0.077, p = .677). These analyses show that perturbation-based CPT does not indu… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Language-level relationship between tok￾enization mismatch and CPT gains. Mean increase in tokens per word from standard to dialect text com￾pared with the mean downstream improvement from CPT on the corresponding parallel-data task. German and Italian exhibit larger t…
Figure 5
Figure 5. Figure 5: Dialect-level relationship between tokeniza￾tion mismatch and CPT gains. tokenization differences cause robustness gains. In￾stead, they suggest that languages with greater surface-form divergence between standard and dialect varieties provide more opportunity for pert…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

117 extracted references · 64 canonical work pages

  1. [1]

    Predicting the target word of game-playing conversations using a low-rank dialect adapter for decoder models , author=. Proc. NAACL HLT , pages=. 2025 , url=

    2025

  2. [2]

    Evaluating large language models along dimensions of language variation: A systematik invesdigatiom uv cross-lingual generalization , author=. Proc. EMNLP , pages=. 2024 , url=

    2024

  3. [3]

    arXiv preprint arXiv:2302.13971 , year=

    Llama: Open and efficient foundation language models , author=. arXiv preprint arXiv:2302.13971 , year=

    arXiv

  4. [4]

    Bafna, Niyati and Chang, Emily and Robinson, Nathaniel Romney and Mortensen, David R and Murray, Kenton and Yarowsky, David and Sirin, Hale , booktitle=. Dial. 2025 , url=

    2025

  5. [5]

    Painter, Jordan and Srirag, Dipankar and Kappiyath, Adarsh and Kanojia, Diptesh and Joshi, Aditya and Yin, Lu , journal=. Dia. 2026 , url=

    2026

  6. [6]

    Improving Dialect Robustness in Large Language Models via

    Maheshwari, Sanjh and Rajpoot, Aniket Singh and Cocarascu, Oana and Mamta , booktitle=. Improving Dialect Robustness in Large Language Models via

  7. [7]

    Kyslyi, Roman and Maksymiuk, Yuliia and Pysmennyi, Ihor , booktitle=

  8. [8]

    Ondrejov. Can. Proc. VarDial , pages=

Show all 117 references
  1. [9]

    Cross-dialectal transfer and zero-shot learning for

    Vidal-Gor. Cross-dialectal transfer and zero-shot learning for. Proceedings of the 4th International Conference on Natural Language Processing for Digital Humanities , pages=

  2. [10]

    Adapters for altering

    Han, HyoJung and Eriguchi, Akiko and Xu, Haoran and Hoang, Hieu and Carpuat, Marine and Khayrallah, Huda , booktitle=. Adapters for altering

  3. [11]

    Testing the Boundaries of LLM s: Dialectal and Language-Variety Tasks

    Faisal, Fahim and Anastasopoulos, Antonios. Testing the Boundaries of LLM s: Dialectal and Language-Variety Tasks. Proc. VarDial. 2025

    2025

  4. [12]

    Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke , booktitle=

  5. [13]

    ASTD : A rabic Sentiment Tweets Dataset

    Nabil, Mahmoud and Aly, Mohamed and Atiya, Amir. ASTD : A rabic Sentiment Tweets Dataset. Proc. EMNLP. 2015. doi:10.18653/v1/D15-1299

    2015 doi

  6. [14]

    Introducing A large T unisian A rabizi Dialectal Dataset for Sentiment Analysis

    Fourati, Chayma and Haddad, Hatem and Messaoudi, Abir and BenHajhmida, Moez and Ben Elhaj Mabrouk, Aymen and Naski, Malek. Introducing A large T unisian A rabizi Dialectal Dataset for Sentiment Analysis. Proceedings of the Sixth Arabic Natural Language Processing Workshop. 2021

    2021

  7. [15]

    Sentiment Classifier: Logistic Regression for

    Al Omari, Marwan and Al-Hajj, Moustafa and Hammami, Nacereddine and Sabra, Amani , booktitle=. Sentiment Classifier: Logistic Regression for. 2019 , volume=

    2019

  8. [16]

    and Shaalan, Khaled

    Alomari, Khaled Mohammad and ElSherif, Hatem M. and Shaalan, Khaled. A rabic Tweets Sentimental Analysis Using Machine Learning. Advances in Artificial Intelligence: From Theory to Practice. 2017

    2017

  9. [17]

    MAC : An Open and Free M oroccan A rabic Corpus for Sentiment Analysis

    Garouani, Moncef and Kharroubi, Jamal. MAC : An Open and Free M oroccan A rabic Corpus for Sentiment Analysis. Innovations in Smart Cities Applications Volume 5. 2022. doi:10.1007/978-3-030-94191-8_68

    2022 doi

  10. [18]

    Customer Sentiments Toward S audi Banks During the C ovid-19 Pandemic

    Alqahtani, Dhuha and Alzahrani, Lama and Bahareth, Maram and Alshameri, Nora and Al-Khalifa, Hend and Aldhubayi, Luluh. Customer Sentiments Toward S audi Banks During the C ovid-19 Pandemic. Proceedings of the 5th International Conference on Natural Language and Speech Process...

    2022

  11. [19]

    Sentiment Analysis of

    Abdelli, Adel and Guerrouf, Fayçal and Tibermacine, Okba and Abdelli, Belkacem , booktitle=. Sentiment Analysis of. 2019 , volume=

    2019

  12. [20]

    No language left behind: Scaling human-centered machine translation , author=

  13. [21]

    and Mao, Yanke and Gao, Haonan and Lee, En-Shiun Annie

    Adelani, David Ifeoluwa and Liu, Hannah and Shen, Xiaoyu and Vassilyev, Nikita and Alabi, Jesujoba O. and Mao, Yanke and Gao, Haonan and Lee, En-Shiun Annie. SIB -200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects. Proc...

    2024 doi

  14. [22]

    Massively Multilingual Transfer for NER

    Rahimi, Afshin and Li, Yuan and Cohn, Trevor. Massively Multilingual Transfer for NER. Proc. ACL. 2019

    2019

  15. [23]

    Cross-lingual Name Tagging and Linking for 282 Languages

    Pan, Xiaoman and Zhang, Boliang and May, Jonathan and Nothman, Joel and Knight, Kevin and Ji, Heng. Cross-lingual Name Tagging and Linking for 282 Languages. Proc. ACL. 2017. doi:10.18653/v1/P17-1178

    2017 doi

  16. [24]

    Slot and Intent Detection Resources for B avarian and L ithuanian: Assessing Translations vs Natural Queries to Digital Assistants

    Winkler, Miriam and Juozapaityte, Virginija and van der Goot, Rob and Plank, Barbara. Slot and Intent Detection Resources for B avarian and L ithuanian: Assessing Translations vs Natural Queries to Digital Assistants. Proc. LREC-COLING. 2024

    2024

  17. [25]

    Findings of the V ar D ial Evaluation Campaign 2023

    Aepli, No. Findings of the V ar D ial Evaluation Campaign 2023. Proc. VarDial. 2023. doi:10.18653/v1/2023.vardial-1.25

    2023 doi

  18. [26]

    From Masked Language Modeling to Translation: Non- E nglish Auxiliary Tasks Improve Zero-shot Spoken Language Understanding

    van der Goot, Rob and Sharaf, Ibrahim and Imankulova, Aizhan and. From Masked Language Modeling to Translation: Non- E nglish Auxiliary Tasks Improve Zero-shot Spoken Language Understanding. Proc. NAACL HLT. 2021. doi:10.18653/v1/2021.naacl-main.197

    2021 doi

  19. [27]

    Transactions of the Association for Computational Linguistics , volume=

    Continual Pre-training on Character-level Noisy Texts Makes Decoder-based Language Models Robust Few-shot Learners , author=. Transactions of the Association for Computational Linguistics , volume=. doi:10.1162/tacl.a.21

    doi

  20. [28]

    Liu , title =

    Colin Raffel and Noam Shazeer and Adam Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi Zhou and Wei Li and Peter J. Liu , title =. Journal of Machine Learning Research , year =

  21. [29]

    ACM Transactions on Information Systems (TOIS) , volume=

    Challenges in building intelligent open-domain dialog systems , author=. ACM Transactions on Information Systems (TOIS) , volume=. 2020 , publisher=

    2020

  22. [30]

    Demographic Dialectal Variation in Social Media: A Case Study of A frican- A merican E nglish

    Blodgett, Su Lin and Green, Lisa and O ' Connor, Brendan. Demographic Dialectal Variation in Social Media: A Case Study of A frican- A merican E nglish. Proc. EMNLP. 2016

    2016

  23. [31]

    ACM Transactions on Asian and Low-Resource Language Information Processing , volume=

    Impact of tokenization on language models: An analysis for turkish , author=. ACM Transactions on Asian and Low-Resource Language Information Processing , volume=. 2023 , publisher=

    2023

  24. [32]

    Tokenization Falling Short:

    Chai, Yekun and Fang, Yewei and Peng, Qiwei and Li, Xuhong , booktitle=. Tokenization Falling Short:. 2024 , url=

    2024

  25. [33]

    arXiv preprint arXiv:2412.09871 , year=

    Byte Latent Transformer: Patches Scale Better Than Tokens , author=. arXiv preprint arXiv:2412.09871 , year=

    arXiv

  26. [34]

    van der Goot, Rob and Jensen, Anette and Schledermann, Emil Allerslev and Kildeberg, Mikkel Wildner and Larsen, Nicolaj and Zhang, Mike and Bassignana, Elisa , booktitle=. Mor. 2024 , organization=

    2024

  27. [35]

    Van Der Goot, Rob and Ramponi, Alan and Zubiaga, Arkaitz and Plank, Barbara and Muller, Benjamin and Roncal, I. Proc. W-NUT , pages=. 2021 , organization=

    2021

  28. [36]

    2023 , url=

    Srivastava, Aarohi and Chiang, David , booktitle=. 2023 , url=

    2023

  29. [37]

    Reese, Samuel and Boleda, Gemma and Cuadros, Montse and Padr. Proc. LREC , year=

  30. [38]

    2020 , url=

    El Boukkouri, Hicham and Ferret, Olivier and Lavergne, Thomas and Noji, Hiroshi and Zweigenbaum, Pierre and Tsujii, Jun’ichi , booktitle=. 2020 , url=

    2020

  31. [39]

    2002 , url=

    Loper, Edward and Bird, Steven , booktitle=. 2002 , url=

    2002

  32. [40]

    Micro-syntactic Variation in

    Zanuttini, Raffaella and Horn, Laurence , year=. Micro-syntactic Variation in

  33. [41]

    Journal of Sociolinguistics , volume=

    Systematic patterning in phonologically-motivated orthographic variation , author=. Journal of Sociolinguistics , volume=. 2015 , publisher=

    2015

  34. [42]

    Batsuren, Khuyagbaatar and Bella, G. Proc. SIGMORPHON , pages=. 2021 , url=

    2021

  35. [43]

    , journal=

    Venezky, Richard L. , journal=. 1967 , xpublisher=

    1967

  36. [44]

    Consonant clusters in

    Hultz. Consonant clusters in. American Speech , volume=. 1965 , xpublisher=

    1965

  37. [45]

    Condorelli, Marco and Rutkowska, Hanna , year=. The

  38. [46]

    Morphological and Syntactic Variation and Change in

    Serrano, Mar. Morphological and Syntactic Variation and Change in. Oxford Research Encyclopedia of Linguistics , year=

  39. [47]

    Describing and Modeling Variation in Grammar , pages=

    Morphological variation: A declarative approach , author=. Describing and Modeling Variation in Grammar , pages=. 2009 , publisher=

    2009

  40. [48]

    Asian Journal of Research in Social Sciences and Humanities , volume=

    On the nature of linguistic variation and its types , author=. Asian Journal of Research in Social Sciences and Humanities , volume=. 2021 , publisher=

    2021

  41. [49]

    Ilbury, Christian , journal=. ``. 2020 , publisher=

    2020

  42. [50]

    Foundations of Language , pages=

    Leaped and leapt: A theoretical account of linguistic variation , author=. Foundations of Language , pages=. 1976 , xpublisher=

    1976

  43. [51]

    2024 , publisher=

    Lexical Variation and Change: A Distributional Semantic Approach , author=. 2024 , publisher=

    2024

  44. [52]

    1994 , publisher=

    The structure of lexical variation: Meaning, naming, and context , author=. 1994 , publisher=

    1994

  45. [53]

    The Handbook of Speech Perception , pages=

    Perception of dialect variation , author=. The Handbook of Speech Perception , pages=. 2021 , publisher=

    2021

  46. [54]

    Aarts, F. G. A. M. , year=. Linguistic variation in

  47. [55]

    2019 , url =

    Language models are unsupervised multitask learners , author=. 2019 , url =

    2019

  48. [56]

    Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Review , author=. Proc. ACL , year=

  49. [57]

    On the Cross-lingual Transferability of Monolingual Representations , author=. Proc. ACL , pages=. 2020 , url=

    2020

  50. [58]

    How to Do a Vocab Swap?

    Jain, Neel and Kirchenbauer, John and Geiping, Jonas and Goldstein, Tom , year=. How to Do a Vocab Swap?

  51. [59]

    Deshpande, Ameet and Talukdar, Partha and Narasimhan, Karthik , booktitle=. When is. 2022 , url=

    2022

  52. [60]

    2022 , url=

    Hiraoka, Tatsuya , booktitle=. 2022 , url=

    2022

  53. [61]

    2023 , url=

    Held, William and Ziems, Caleb and Yang, Diyi , booktitle=. 2023 , url=

    2023

  54. [62]

    ACM Computing Surveys , year=

    Natural language processing for dialects of a language: A survey , author=. ACM Computing Surveys , year=

  55. [63]

    2024 , url =

    Faisal, Fahim and Ahia, Orevaoghene and Srivastava, Aarohi and Ahuja, Kabir and Chiang, David and Tsvetkov, Yulia and Anastasopoulos, Antonios , booktitle =. 2024 , url =

    2024

  56. [64]

    Findings of the ACL: EMNLP , pages=

    Quantifying the Dialect Gap and its Correlates Across Languages , author=. Findings of the ACL: EMNLP , pages=. 2023 , url=

    2023

  57. [65]

    Decoupled Weight Decay Regularization , author=. Proc. ICLR , year=

  58. [66]

    Attention is All you Need , url =

    Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser,. Attention is All you Need , url =. Advances in Neural Information Processing Systems , xpublisher =

  59. [67]

    Pfeiffer, Jonas and Vuli. Proc. EMNLP , pages=. 2020 , url=

    2020

  60. [68]

    Oolong: Investigating What Makes Transfer Learning Hard with Controlled Studies , author=. Proc. EMNLP , year=

  61. [69]

    Improving Robustness of Machine Translation with Synthetic Noise , author=. Proc. NAACL HLT , pages=. 2019 , url=

    2019

  62. [70]

    2018 , url=

    Michel, Paul and Neubig, Graham , booktitle=. 2018 , url=

    2018

  63. [71]

    Noising and denoising natural language: Diverse backtranslation for grammar correction , author=. Proc. NAACL HLT , pages=. 2018 , url=

    2018

  64. [72]

    and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu and others , booktitle=

    Hu, Edward J. and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu and others , booktitle=. 2021 , url=

    2021

  65. [73]

    Stress Test Evaluation for Natural Language Inference , author =. Proc. COLING , xaddress =. 2018 , url =

    2018

  66. [74]

    2018 , url=

    Wang, Alex and Singh, Amanpreet and Michael, Julian and Hill, Felix and Levy, Omer and Bowman, Samuel , booktitle=. 2018 , url=

    2018

  67. [75]

    2019 , url=

    Tenney, Ian and Das, Dipanjan and Pavlick, Ellie , booktitle=. 2019 , url=

    2019

  68. [76]

    Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation , author=. Proc. W-NUT , pages=. 2019 , url=

    2019

  69. [77]

    and Laham, Darrell and Rehder, Bob and Schreiner, M

    Landauer, Thomas K. and Laham, Darrell and Rehder, Bob and Schreiner, M. E. , booktitle=. How well can passage meaning be derived without using word order? A comparison of

  70. [78]

    Gaman, Mihaela and Chifu, Adrian-Gabriel and Domingues, William and Ionescu, Radu Tudor , journal=

  71. [79]

    Recursive deep models for semantic compositionality over a sentiment treebank , author=. Proc. EMNLP , pages=

  72. [80]

    Cross-lingual Transfer Learning for Multilingual Task Oriented Dialog , author=. Proc. NAACL HLT , pages=

  73. [81]

    Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces , author=. Proc. Workshop on Privacy in ML and AI , year=

  74. [82]

    Overview of

    Garci. Overview of. Proc. IberLEF , pages=. 2020 , url =

    2020

  75. [83]

    Compilation of Large

    Jos. Compilation of Large

  76. [84]

    Ca. Proc. PML4DC , year=

  77. [85]

    Does Manipulating Tokenization Aid Cross-Lingual Transfer?

    Blaschke, Verena and Sch. Does Manipulating Tokenization Aid Cross-Lingual Transfer?. Proc. VarDial , pages=

  78. [86]

    Fine-Tuning

    Srivastava, Aarohi and Chiang, David , booktitle=. Fine-Tuning

  79. [87]

    Automatically constructing a corpus of sentential paraphrases , author=. Proc. IWP , year=

  80. [88]

    Ziems, Caleb and Held, William and Yang, Jingfeng and Dhamala, Jwala and Gupta, Rahul and Yang, Diyi , booktitle=. Multi-. 2023 , url=

    2023

  81. [89]

    Findings of the ACL , pages=

    Improving Zero-Shot Cross-lingual Transfer Between Closely Related Languages by Injecting Character-Level Noise , author=. Findings of the ACL , pages=. 2022 , url=

    2022

  82. [90]

    Enhancing BERT for lexical normalization , author=. Proc. W-NUT , year=

  83. [91]

    2015 , url=

    Supranovich, Dmitry and Patsepnia, Viachaslau , booktitle=. 2015 , url=

    2015

  84. [92]

    ACM Transactions on Intelligent Systems and Technology (TIST) , volume=

    Lexical normalization for social media text , author=. ACM Transactions on Intelligent Systems and Technology (TIST) , volume=. 2013 , publisher=. doi:https://doi.org/10.1145/2414425.2414430

    2013

  85. [93]

    Graph-based

    Demir, Seniz and Topcu, Berkay , journal=. Graph-based. 2022 , publisher=. doi:https://doi.org/10.1016/j.jestch.2022.101192

    2022

  86. [94]

    Easy-to-use combination of

    Benamar, Alexandra and Bothua, Meryl and Grouin, Cyril and Vilnat, Anne , booktitle=. Easy-to-use combination of

  87. [95]

    Lee, Sangah and Shin, Hyopil , booktitle=. The. 2021 , url=

    2021

  88. [96]

    Dealing with textual noise for robust and effective

    Chen, Xuanang and He, Ben and Hui, Kai and Sun, Le and Sun, Yingfei , journal=. Dealing with textual noise for robust and effective. 2023 , publisher=. doi:https://doi.org/10.1016/j.ipm.2022.103135

    2023

  89. [97]

    2021 , url=

    Soper, Elizabeth and Fujimoto, Stanley and Yu, Yen-Yun , booktitle=. 2021 , url=

    2021

  90. [98]

    Transactions of the Association for Computational Linguistics , volume=

    Byt5: Towards a token-free future with pre-trained byte-to-byte models , author=. Transactions of the Association for Computational Linguistics , volume=. 2022 , publisher=. doi:10.1162/tacl_a_00461

    2022 doi

  91. [99]

    2022 , publisher=

    Clark, Jonathan H and Garrette, Dan and Turc, Iulia and Wieting, John , journal=. 2022 , publisher=

    2022

  92. [100]

    CharBERT: Character-aware Pre-trained Language Model , author=. Proc. COLING , pages=

  93. [101]

    2019 , url=

    Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina , booktitle=. 2019 , url=

    2019

  94. [102]

    Advances in neural information processing systems , volume=

    Attention is all you need , author=. Advances in neural information processing systems , volume=. 2017 , url =

    2017

  95. [103]

    Multi-view Subword Regularization , author=. Proc. NAACL HLT , pages=. doi:10.18653/v1/2021.naacl-main.40

    2021 doi

  96. [104]

    Superbizarre Is Not Superb: Derivational Morphology Improves

    Hofmann, Valentin and Pierrehumbert, Janet and Sch. Superbizarre Is Not Superb: Derivational Morphology Improves. Proc. ACL-IJCNLP , year=

  97. [105]

    Charformer: Fast Character Transformers via Gradient-based Subword Tokenization , author=. Proc. ICLR , year=

  98. [106]

    Models In a Spelling Bee: Language Models Implicitly Learn the Character Composition of Tokens , author=. Proc. NAACL HLT , pages=

  99. [107]

    Stress Test Evaluation of

    Aspillaga, Carlos and Carvallo, Andr. Stress Test Evaluation of. Proc. LREC , pages=

  100. [108]

    Proceedings of the 23rd Annual Conference of the European Association for Machine Translation , pages=

    A taxonomy and study of critical errors in machine translation , author=. Proceedings of the 23rd Annual Conference of the European Association for Machine Translation , pages=

  101. [109]

    On the Robustness of Language Encoders against Grammatical Errors , author=. Proc. ACL , year=

  102. [110]

    Noisy text data:

    Kumar, Ankit and Makhija, Piyush and Gupta, Anuj , booktitle=. Noisy text data:

  103. [111]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman , title =. 1972 , volume =

    1972

  104. [112]

    Publications Manual , year =

  105. [113]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year =. Alternation , journal =

  106. [114]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

  107. [115]

    1997 , publisher =

    Dan Gusfield , title =. 1997 , publisher =

    1997

  108. [116]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    2015

  109. [117]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.