pith. sign in

arxiv: 1906.11483 · v1 · pith:4UKLVDVCnew · submitted 2019-06-27 · 💻 cs.CL

Morphological Irregularity Correlates with Frequency

Shijie Wu , Ryan Cotterell , Timothy J. O'Donnell This is my paper

Pith reviewed 2026-05-25 15:15 UTC · model grok-4.3

classification 💻 cs.CL
keywords morphological irregularityword frequencyneural transduction modelinformation-theoretic measurelinguistic paradigmscross-linguistic analysismorphology
0
0 comments X

The pith

Analyses of 28 languages show higher-frequency words are more likely to be morphologically irregular.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper defines morphological irregularity through an information-theoretic measure of how predictable a word form is given others in its paradigm. Using a neural transduction model, it estimates this quantity across forms in 28 languages and conducts analyses that reveal a correlation with frequency. Higher-frequency items tend to be irregular, and irregular items tend to be high-frequency. The pattern strengthens when measured at the level of entire paradigms rather than isolated forms. A sympathetic reader would care because the result supplies broad empirical backing for longstanding linguistic ideas about how usage shapes structure and how abstract stems organize inflected words.

Core claim

The central claim is that morphological irregularity correlates with frequency: higher frequency items are more likely to be irregular and irregular items are more likely to be highly frequent. This holds across the 28 languages examined, is more robust when forms are grouped into whole paradigms, and supplies the first confirmation of this breadth for proposals from the linguistics literature.

What carries the argument

An information-theoretic measure of irregularity based on the predictability of forms, estimated by a neural transduction model.

Load-bearing premise

The neural model's estimate of irregularity matches the linguistic notion of irregularity instead of reflecting model-specific artifacts.

What would settle it

A new cross-linguistic dataset in which linguist-assigned irregularity ratings show no correlation with the model's scores, or in which frequency and irregularity are uncorrelated.

Figures

Figures reproduced from arXiv: 1906.11483 by Ryan Cotterell, Shijie Wu, Timothy J. O'Donnell.

Figure 1
Figure 1. Figure 1: Lemma paradigm tree 4 Modeling Morphological Inflection Our goal is to estimate P(w | `, σ,L−`) from data. We do this by using a structured probabilistic model of string transduction which we call pθ. In the following sections, we describe this model, how we handle syncretism in the model, our training (holdout and test) scheme, and our estimates of the degree of irregularity ι. 4.1 A Lemma-Based Model In … view at source ↗
Figure 2
Figure 2. Figure 2: Average degree of irregularity ι across lan￾guages. semi-regular patterns of inflection. Our approach however makes this impossible by strictly assign￾ing all forms from each lexeme to either train or test. It is important to ask, therefore, how well does our model predict the forms of heldout lexemes given this stricture? The results are displayed in [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Correlations between irregularity and fre [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Correlations between irregularity and fre [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
read the original abstract

We present a study of morphological irregularity. Following recent work, we define an information-theoretic measure of irregularity based on the predictability of forms in a language. Using a neural transduction model, we estimate this quantity for the forms in 28 languages. We first present several validatory and exploratory analyses of irregularity. We then show that our analyses provide evidence for a correlation between irregularity and frequency: higher frequency items are more likely to be irregular and irregular items are more likely be highly frequent. To our knowledge, this result is the first of its breadth and confirms longstanding proposals from the linguistics literature. The correlation is more robust when aggregated at the level of whole paradigms--providing support for models of linguistic structure in which inflected forms are unified by abstract underlying stems or lexemes. Code is available at https://github.com/shijie-wu/neural-transducer.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper defines an information-theoretic measure of morphological irregularity based on predictability from a neural transduction model, estimates it for forms across 28 languages, performs validatory and exploratory analyses, and reports a correlation with frequency: higher-frequency items tend to be more irregular and irregular items tend to be more frequent. The correlation strengthens when aggregated over paradigms, which the authors interpret as support for abstract stem/lexeme representations. Code is released.

Significance. If the central correlation survives controls for frequency-dependent artifacts in the neural model, the result would supply the broadest cross-linguistic empirical support yet for classic linguistic claims linking irregularity and frequency. The paradigm-level finding and the public code release are clear strengths that aid reproducibility and allow direct testing of the measure.

major comments (2)
  1. [Methods (neural model training and irregularity estimation)] Methods (neural transduction model and irregularity estimation): because the model is trained on the same frequency distribution later used for the correlation, any frequency-dependent optimization (better memorization or lower cross-entropy on high-count items) can directly affect the estimated irregularity score. Without frequency-balanced training, frequency-stratified held-out evaluation, or external validation against hand-labeled regular/irregular classes, the reported correlation risks being partly mechanical rather than linguistic.
  2. [Results (correlation analyses)] Results (paradigm-level aggregation): the claim that the correlation is 'more robust when aggregated at the level of whole paradigms' is central to the linguistic interpretation, yet the manuscript provides no explicit definition of how paradigms are constructed, no statistical comparison of form-level vs. paradigm-level effect sizes, and no error analysis showing that the improvement is not driven by a few high-frequency irregular paradigms.
minor comments (1)
  1. [Abstract] Abstract: 'irregular items are more likely be highly frequent' contains a missing 'to'.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. We address each major point below, providing the strongest honest defense of the manuscript while acknowledging where revisions are warranted.

read point-by-point responses

  1. Referee: Methods (neural transduction model and irregularity estimation): because the model is trained on the same frequency distribution later used for the correlation, any frequency-dependent optimization (better memorization or lower cross-entropy on high-count items) can directly affect the estimated irregularity score. Without frequency-balanced training, frequency-stratified held-out evaluation, or external validation against hand-labeled regular/irregular classes, the reported correlation risks being partly mechanical rather than linguistic.

    Authors: We acknowledge the potential for frequency-dependent effects in model training. However, any such bias would improve predictability (reduce estimated irregularity) for high-frequency items, which works directly against the observed positive correlation between frequency and irregularity. The reported result is therefore conservative with respect to this artifact. The manuscript already includes validatory analyses comparing the measure against known morphological patterns in several languages; we will add an explicit discussion of this point in the revision. revision: partial

  2. Referee: Results (paradigm-level aggregation): the claim that the correlation is 'more robust when aggregated at the level of whole paradigms' is central to the linguistic interpretation, yet the manuscript provides no explicit definition of how paradigms are constructed, no statistical comparison of form-level vs. paradigm-level effect sizes, and no error analysis showing that the improvement is not driven by a few high-frequency irregular paradigms.

    Authors: We agree these details should be clarified. In the revised manuscript we will (i) explicitly define paradigm construction (forms grouped by shared lemma), (ii) report a direct statistical comparison of effect sizes between form-level and paradigm-level analyses, and (iii) include a robustness check or error analysis confirming the stronger paradigm-level correlation is not driven by a small number of high-frequency paradigms. These additions will be straightforward to implement from the existing data and code. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical correlation with externally estimated irregularity measure

full rationale

The paper defines irregularity via an information-theoretic predictability measure estimated by a neural transduction model, then correlates the resulting scores with frequency counts across 28 languages. This chain does not reduce the reported correlation to a self-definition, a fitted parameter renamed as a prediction, or a load-bearing self-citation; the model supplies an independent proxy for predictability that is not constructed from the frequency variable itself. No uniqueness theorems, ansatzes smuggled via citation, or renamings of known results appear in the derivation. The study is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Based solely on abstract; the central claim rests on the assumption that neural-model predictability is a valid proxy for linguistic irregularity and that the 28-language sample is representative. No free parameters or invented entities are named in the abstract.

axioms (1)
  • domain assumption Neural transduction model output probabilities provide a faithful estimate of morphological predictability.
    Invoked when defining the information-theoretic irregularity measure from model predictions.

pith-pipeline@v0.9.0 · 5672 in / 1167 out tokens · 21799 ms · 2026-05-25T15:15:59.110411+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 41 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

    work page

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

    work page

  3. [3]

    Farrell Ackerman and Robert Malouf. 2013. Morphological organization: T he low conditional entropy conjecture. Language, 89(3):429--464

    work page 2013

  4. [4]

    Adam Albright and Bruce Hayes. 2003. Rules vs. analogy in E nglish past tenses: A computational/experimental study. Cognition, 90(2):119--161

    work page 2003

  5. [5]

    Harald Baayen

    R. Harald Baayen. 2001. Word Frequency Distributions. Springer, Berlin, Germany

    work page 2001

  6. [6]

    Matthew Baerman, Dunstan Brown, and Greville G. Corbett. 2015. Understanding and measuring morphological complexity: An introduction. Oxford University Press

    work page 2015

  7. [7]

    Corbett, and D

    Matthew Baerman, Greville G. Corbett, and D. P. Brown. 2010. Defective Paradigms: Missing forms and what they tell us. Oxford University Press, Oxford, England

    work page 2010

  8. [8]

    Jean Berko. 1958. The child's learning of E nglish morphology. Word, 14:150--177

    work page 1958

  9. [9]

    Joan L. Bybee. 1985. Morphology: A Study of the Relation between Meaning and Form . John Benjamins, Amsterdam

    work page 1985

  10. [10]

    Joan L. Bybee. 1991. Natural morphology: T he organization of paradigms and language acquisition. In Thom Huebner and Charles A. Ferguson, editors, Cross Currents in Second Language Acquisition and Linguistic Theory. John Benjamins Publishing Company

    work page 1991

  11. [11]

    Chitashvili and R

    Revas J. Chitashvili and R. Harald Baayen. 1993. Word frequency distributions. Quantitative Text Analysis, pages 54--135

    work page 1993

  12. [12]

    Ryan Cotterell, Christo Kirov, Mans Hulden, and Jason Eisner. 2018 a . On the complexity and typology of inflectional morphological systems. Transaction of the Association for Computational Linguistics ( TACL )

    work page 2018

  13. [13]

    Mielke, and Jason Eisner

    Ryan Cotterell, Christo Kirov, Sebastian J. Mielke, and Jason Eisner. 2018 b . https://doi.org/10.18653/v1/N18-2087 Unsupervised disambiguation of syncretism in inflected lexicons . In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pag...

    work page doi:10.18653/v1/n18-2087 2018

  14. [14]

    e raldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sandra K \"u bler, David Yarowsky, Jason Eisner, and Mans Hulden

    Ryan Cotterell, Christo Kirov, John Sylak-Glassman, G \. e raldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sandra K \"u bler, David Yarowsky, Jason Eisner, and Mans Hulden. 2017 a . https://doi.org/10.18653/v1/K17-2001 CoNLL-SIGMORPHON 2017 shared task: U niversal morphological reinflection in 52 languages . In Proceedings of the CoNLL S...

    work page doi:10.18653/v1/k17-2001 2017

  15. [15]

    Ryan Cotterell, John Sylak-Glassman, and Christo Kirov. 2017 b . Neural graphical models over strings for principal parts morphological paradigm completion. In Proceedings of the 15th Conference of the E uropean Chapter of the Association for Computational Linguistics ( EACL2017 )

    work page 2017

  16. [16]

    Viviana Fratini, Joana Acha, and Itziar Laka. 2014. Frequency and morphological irregularity are independent variables. E vidence from a corpus study of S panish verbs. Corpus Linguistics and Linguistic Theory, 10(2):289 --314

    work page 2014

  17. [17]

    Andrew Gelman and Jennifer Hill. 2007. Data Analysis using Regression and Multilevel/Hierarchical Models. Cambridge University Press, Cambridge

    work page 2007

  18. [18]

    Martin Haspelmath and Andrea D. Sims. 2010. Understanding Morphology. Hodder Education

    work page 2010

  19. [19]

    Jennifer Hay. 2003. Causes and Consequences of Word Structure. Routledge, New York, NY

    work page 2003

  20. [20]

    Borja Herce. 2016. Why frequency and morphological irregularity are not independent variables in S panish: A response to F ratini et al. (2014). Corpus Linguistics and Linguistic Theory, 12(2)

    work page 2016

  21. [21]

    Charles F. Hockett. 1954. Two models of grammatical description. Word, 10:210--231

    work page 1954

  22. [22]

    Ferenc Kiefer. 2000. Regularity. In Morphologie: Ein internationales Handbuch zur Flexion und Wortbildung/Morphology: An international Handbook on Inflection and Word-Formation. Walter d e Gruyter, Berlin

    work page 2000

  23. [23]

    Christo Kirov, Ryan Cotterell, John Sylak-Glassman, G \'e raldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sebastian Mielke, Arya D McCarthy, Sandra K \"u bler, et al. 2018. Unimorph 2.0: U niversal morphology. arXiv preprint arXiv:1810.11101

    work page arXiv 2018

  24. [24]

    Kyle Mahowald, Isabelle Dautriche, Edward Gibson, and Steven Thomas Piantadosi. 2018. Word forms are structured for efficient use. Cognitive Science, 42(8):3116--3134

    work page 2018

  25. [25]

    Marcus, Steven Pinker, Michael T

    Gary F. Marcus, Steven Pinker, Michael T. Ullman, Michelle Hollander, T. John Rosen, and Fei Xu. 1992. Overregularization in Language Acquisition. Monographs of the society for research in child development. University of Chicago Press, Chicago, IL

    work page 1992

  26. [26]

    McClelland and Karalyn Patterson

    James L. McClelland and Karalyn Patterson. 2002 a . Rules or connections in past-tense inflections: W hat does the evidence rule out? Trends in Cognitive Sciences, 6(11):465--472

    work page 2002

  27. [27]

    McClelland and Karalyn Patterson

    James L. McClelland and Karalyn Patterson. 2002 b . ` W ords or R ules' cannot exploit the regularity in exceptions. Trends in Cognitive Sciences, 6(11):464--465

    work page 2002

  28. [28]

    O'Donnell

    Timothy J. O'Donnell. 2015. Productivity and Reuse in Language: A Theory of Linguistic Computation and Storage . The MIT Press, Cambridge, Massachusetts

    work page 2015

  29. [29]

    Steven Pinker. 1999. Words and Rules. HarperCollins, New York, NY

    work page 1999

  30. [30]

    Steven Pinker and Alan Prince. 1988. On language and connectionism: A nalysis of a parallel distributed processing model of language acquisition. Cognition, 28:73--193

    work page 1988

  31. [31]

    Steven Pinker and Michael T. Ullman. 2002 a . Combination and structure, not gradedness, is the issue. Trends in Cognitive Sciences, 6(11):472--474

    work page 2002

  32. [32]

    Steven Pinker and Michael T. Ullman. 2002 b . The past and future of the past tense debate. Trends in Cognitive Sciences, 6(11):456--463

    work page 2002

  33. [33]

    Sandeep Prasada and Steven Pinker. 1993. Generalisation of regular and irregular morphological patterns. Language and Cognitive Processes, 8(1):1--56

    work page 1993

  34. [34]

    Lawrence R. Rabiner. 1989. A tutorial on hidden M arkov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257--286

    work page 1989

  35. [35]

    Rumelhart and James L

    David E. Rumelhart and James L. McClelland. 1986. On learning the past tenses of E nglish verbs. In Parallel Distributed Processing: Explorations in the Microstructure of Cognition., volume 2, pages 216--271. Bradford Books/MIT Press, Cambridge, MA

    work page 1986

  36. [36]

    Dorothy Siegel. 1974. Topics in English Morphology. Ph.D. thesis, Massachusetts Institute of Technology

    work page 1974

  37. [37]

    Thomas Stolz, Hitomi Otsuka, Aina Urdze, and Johan v an d er Auwera. 2012. Introduction: I rregularity --- glimpses of a ubiquitous phenomenon. In Thomas Stolz, Hitomi Otsuka, Aina Urdze, and Johan v an d er Auwera, editors, Irregularity in Morphology (and Beyond), pages 7--38. Akademie Verlag, Berlin, Germany

    work page 2012

  38. [38]

    Gregory T. Stump. 2001. Inflection. In Handbook of Morphology. Blackwell, Oxford, England

    work page 2001

  39. [39]

    Shijie Wu and Ryan Cotterell. 2019. Exact hard monotonic attention for character-level transduction. arXiv preprint arXiv:1905.06319

    work page arXiv 2019

  40. [40]

    Charles D. Yang. 2002. Knowledge and Learning in Natural Language. Oxford linguistics. Oxford University Press, New York

    work page 2002

  41. [41]

    Charles D. Yang. 2016. The Price of Productivity: H ow Children Learn to Break the Rules of Language . The MIT Press, Cambridge, Massachusetts

    work page 2016