Pith. sign in

REVIEW 3 major objections 4 minor 43 references

The Natural Selection of Words: Finding the Features of Fitness

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A synonym's chance of dominating its meaning can be predicted fifty years ahead at roughly double the accuracy of random guessing.

desk verdict Predicting which synonym will win a WordNet synset fifty years out works modestly but robustly; the released dataset is the most durable contribution. read the letter →

arxiv 1908.07013 v1 pith:PBSDI6SB submitted 2019-08-19 cs.CL cs.LG

Peter D. Turney , Saif M. Mohammad This is my paper · ORCID
classification cs.CLcs.LG
keywords synsetwordfrequencylanguagechangenaturalselectionnaiveBayesGoogleBooksNgramCorpusNetlexicalreplacement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's central claim is that competition among synonyms is partly predictable: the word that will dominate a meaning fifty years later can be forecast from cheap surface features and past frequency trends. The authors assemble 4,000 WordNet synsets—sets of interchangeable words—covering 9,000 single-sense English words, with frequencies from the Google Books Ngram Corpus sampled at fifty-year intervals from 1800 to 2000. A supervised classifier predicts the future leader of each synset with F-scores of 38.5–43.3%, versus 17.3–24.8% for random guessing, and it also reproduces known linguistic pressures such as -ize overtaking -ise and -ity overtaking -ness. The payoff is that language change is not just noise: it has a measurable selective direction, and that direction can be learned from data.

What carries the argument

The machinery is the synset treated as an evolutionary unit: each word's token count is its population, and the highest-frequency word is the leader. To make the problem a classification, the paper aligns three time points (past, present, future) separated by fifty years, with future data hidden at test time. Each word is converted to a feature vector with eight components: normalized length, syllable count, unique letter trigrams, the fraction of trigrams shared with synset rivals, number of categorial variations, relative growth from past to present, linear extrapolation of relative frequency, and present age. The classifier is naive Bayes with Gaussian class-conditional densities; synset-level predictions are made by picking the word with the highest estimated win probability. The mapping between the two corpora is made by restricting to single-sense WordNet words, so that a part-of-speech-tagged frequency can be assigned unambiguously to one sense.

What would settle it

Check a sample of the 4,000 synsets against historical dictionaries or sense-annotated texts to see whether synset membership actually stayed fixed from 1800 to 2000. If a material fraction of synsets gained or lost words, rerun the classifier on only the historically stable synsets; if the F-score advantage over random guessing vanishes, the predictability result depends on the stability assumption.

Watch

Extended reading notes

Core claim

The central claim is that a synset—a set of synonymous words in WordNet—has a leader (its highest-frequency member), and that leadership is governed by a predictable selective process. Using frequencies from the Google Books Ngram Corpus sampled every fifty years from 1800 to 2000, a naive Bayes classifier built on word length, syllable count, distinctive trigrams, relative growth, linear extrapolation, and word age predicts the leader fifty years later with F-scores of 38.5–43.3%, compared with 17.3–24.8% for random guessing. The paper further claims that the learned models are interpretable: winners tend to be shorter but still distinctive, older, and growing faster than their rivals, and the trigram analysis reproduces documented trends such as -ize replacing -ise and -ity gaining on -ness. The conclusion is that lexical replacement in this period is not random drift but is shaped by persistent selective pressures.

Load-bearing premise

The load-bearing premise is that WordNet's synonym sets describe the same meanings in 1800 as they do today; if words entered or left a synset over those two centuries, the frequency counts, leader labels, and the whole prediction task would be unreliable.

Editorial extensions

If this is right

  • A fifty-year-ahead prediction of a synset's leader is feasible at F-scores of 38.5–43.3%, roughly double the 17.3–24.8% of random guessing, on test periods ending in 1950 and 2000.
  • Linear extrapolation of relative frequency is the strongest single feature, implying that the direction of selection is steady enough over decades to be projected.
  • Leadership change is frequent but declining: 42% of synsets changed leader at least once between 1800 and 2000, and the percentage changing in a fifty-year window falls over time.
  • The model's trigram analysis supports known morphological pressures: -ize is replacing -ise, -ity is gaining on -ness, and short words win only when they keep enough distinctive characters.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested extension is to apply the same feature set to non-book corpora, such as newspapers or transcribed speech; if the predictive advantage shrinks there, the signal may be specific to the written, bookish register of the Google Books sample.
  • The paper's evolutionary framing suggests a transferable prediction: within any competitive set with measurable growth rates and distinctiveness, such as brand shares in a product category, the same class of features should forecast future leaders if cultural selection is the underlying mechanism.
  • A further implication of the coupling between meaning-change and word-change, which the paper leaves untested, is that a word's loss of leadership could be anticipated by tracking its rising ambiguity in historical text; the authors note this is enabled only by full sense-tagging of the corpus.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces a dataset that combines WordNet synsets with Google Books Ngram Corpus frequencies, restricted to monosemous words, and tracks the relative frequencies of synonyms from 1800 to 2000. The central task is to predict, fifty years ahead, which member of a synset will become the most frequent, phrased as a binary winner/loser classification at the word level and aggregated to synset-level leader prediction. The authors use a naive Bayes classifier with features based on word length, syllable count, trigram uniqueness, shared trigrams, categorial variations, relative growth, linear extrapolation, and present age. They report F-scores of 38.5--43.3% for identifying leadership changes, compared with 17.3--24.8% for a random baseline, and interpret the learned models as evidence for suffix rivalry, economy-versus-clarity tradeoffs, and decreasing language change over time. The paper also releases the dataset and processing software.

Significance. If the empirical claim holds, this is a useful demonstration that lexical replacement within a fixed meaning is at least partially predictable, and the released dataset provides a reusable resource for diachronic lexical studies. The main experiment is an out-of-sample evaluation with proper temporal separation: predictive features are computed only from past and present data, and the target labels come from fifty years later. The paper also gives credit for consistent reporting of precision, recall, and F-score against a random baseline, and its feature-level analysis reproduces known linguistic trends such as the -ise/-ize shift and the -ity/-ness rivalry. The significance is tempered by the assumption of stable WordNet synsets over two centuries, which is load-bearing for the meaning of the leader labels, and by data-filtering choices that restrict the scope of the claim.

major comments (3)
  1. [Potential Limitations of GBNC and WordNet; Step 3; Feature 8] The paper explicitly assumes that WordNet synsets are stable from 1800 to 2000, but this assumption is load-bearing for the central prediction task. Historical GBNC token counts for a word are attributed to a modern synset even if the word's sense changed or the word entered the synset later; if early-period leaders are anachronistic assignments, the F-score may partly reflect predicting sense shifts or word-entry artifacts rather than competition within a fixed meaning. The authors offer only qualitative assurances and no quantitative verification. Because Feature 8 (Present age) is computed from the first nonzero GBNC year for each word, the released data directly supports a test: report the distribution of word birth years, the fraction of synset members absent at the start of each period, and the sensitivity of leader labels and of the reported F-scores to restricting synsets to words present throughout the interval.
  2. [Building datasets, Step 4] Step 4 removes any synset with a tie for first place in the present or future and removes synsets containing words with zero frequency in the present. This filtering excludes scientifically important outcomes of selection, including co-leadership and word death, and it changes the target distribution: the reported F-score is conditional on a unique winner at both endpoints and on no dead words at the present. The paper states without supporting analysis that including dead words would 'artificially inflate' the algorithm's score; this is an empirical claim that should be tested. Please report how many synsets are discarded by each criterion and provide a robustness check with less restrictive filtering, such as treating dead words as legitimate losers and handling ties by random or conservative assignment.
  3. [Experiments with NBCP, Table 6] The primary 50-year-cycle evaluation rests on only two test sets, Test1 and Test2, and the reported Wilson confidence intervals treat the synsets in each test set as independent samples. This does not account for temporal autocorrelation or for the small number of temporal instantiations, so the intervals do not directly support a general claim about predictability across time. The varying-cycle experiments add more test periods, but those periods are not independent of one another either. Please add a time-based analysis, for example a bootstrap that resamples time periods, or at least clearly state that the confidence intervals are within-test-set intervals and discuss the limited temporal generality of the 50-year result.
minor comments (4)
  1. [Feature vectors for words, Feature 3] The description of Feature 3 says 'we want to focus on the features that are unique to the winner,' but the construction actually computes trigrams unique to each word within its synset and does not use the target class. This wording could be misread as target leakage and should be rephrased.
  2. [Interpretation of the learned models, Table 12] The top trigrams in Table 12 are selected after sorting by the largest class-mean differences, and then each difference is tested for significance with a t-test. This selection procedure creates a multiple-comparisons problem, so the statement that all differences are significant is not justified as reported; adjusted p-values or a validation-based selection procedure would be more appropriate.
  3. [Experiments with NBCP, Tables 6 and 9] The random baseline is a reasonable reference, but the trivial 'always predict the present leader' baseline, which has zero F-score by construction, is too weak to be informative on its own. Reporting a persistence baseline that predicts no change would help calibrate how much of the advantage over random is due to the task's stability prior.
  4. [General] There are several typographical errors: 'signficantly' in the Table 8 caption, 'contribtion' in the text preceding Table 8, and 'hight' in the Future Work section. These should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

Out-of-sample prediction with no constructional circularity; acknowledged assumptions are validity limitations, not circular reductions.

full rationale

The paper's central claim—that synset leadership changes are predictable—is evaluated with a genuinely out-of-sample protocol. Feature vectors (Features 1-8) are functions of word form, syllable count, CatVar-derived categorial variations, and GBNC frequencies from the past and present periods only; the target class is defined by the future period. Training uses Train1/Train2 shifted fifty years before the corresponding test sets, so no future information enters feature construction (Section 'Building datasets', Step 4; Section 'Feature vectors for words'). The linear-extrapolation feature (Feature 7) is a fixed formula f3 = 2f2 - f1 computed from past and present relative frequencies, not a fitted prediction, and its large ablation impact (Table 7) is an empirical finding rather than a constructional equivalence. The assumption that WordNet synsets are stable over 1800-2000 is explicitly acknowledged as an assumption and is a correctness/validity limitation, not a circular reduction: labels are not derived from features and features are not derived from labels. The only self-citation, reference [15], points to released data and software and does not justify any load-bearing premise. No fitted input is renamed as a prediction, no uniqueness theorem is imported from the authors' prior work, and no known result is merely relabeled. The comparison against random guessing is an external baseline, not a restatement of the model output. Therefore no significant circularity is present.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the stability of WordNet synsets, the representativeness of GBNC relative frequencies, and the generalizability from monosemous to polysemous words. No new entities are introduced.

free parameters (2)
  • fifty-year sampling cycle = 50 years
    Hand-chosen interval between past, present, and future periods; the paper experiments with 30 to 60 years and the F-scores vary.
  • eleven-year smoothing interval = 1800 +/- 5, i.e., 11 years
    Frequency counts are summed over an 11-year window to reduce noise; the choice affects which synsets are considered changed.
assumptions (3)
  • domain assumption WordNet synsets are stable from 1800 to 2000.
    Assumed in dataset construction; if words entered or left synsets, the frequency and leader labels would be inaccurate.
  • domain assumption Relative frequencies within a synset in GBNC are not biased by corpus composition.
    The paper argues scientific text bias affects all words in a synset similarly, but this is an assumption.
  • domain assumption Evolutionary mechanisms for monosemous words generalize to polysemous words.
    Used to claim broader conclusions despite restricting the dataset to monosemous words.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Natural Selection of Words: Finding the Features of Fitness." pith.science (2026). https://pith.science/paper/PBSDI6SB

@misc{pith2026190807013,
  author       = {Pith},
  title        = {Pith review of: The Natural Selection of Words: Finding the Features of Fitness},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PBSDI6SB}},
  note         = {Machine review of arXiv:1908.07013}
}
read the original abstract

We introduce a dataset for studying the evolution of words, constructed from WordNet and the Google Books Ngram Corpus. The dataset tracks the evolution of 4,000 synonym sets (synsets), containing 9,000 English words, from 1800 AD to 2000 AD. We present a supervised learning algorithm that is able to predict the future leader of a synset: the word in the synset that will have the highest frequency. The algorithm uses features based on a word's length, the characters in the word, and the historical frequencies of the word. It can predict change of leadership (including the identity of the new leader) fifty years in the future, with an F-score considerably above random guessing. Analysis of the learned models provides insight into the causes of change in the leader of a synset. The algorithm confirms observations linguists have made, such as the trend to replace the -ise suffix with -ize, the rivalry between the -ity and -ness suffixes, and the struggle between economy (shorter words are easier to remember and to write) and clarity (longer words are more distinctive and less likely to be confused with one another). The results indicate that integration of the Google Books Ngram Corpus with WordNet has significant potential for improving our understanding of how language evolves.

Figures

Figures reproduced from arXiv: 1908.07013 by the authors.

Figure 1
Figure 1. The normalized frequencies of the rapturous–ecstatic synset from 1800 AD to 2000 AD. The sum of the five frequencies for any given year is 1.0. The data has not been smoothed, in order to show the level of noise in the trends. This synset is typical with respect to the shapes of the curves and the level of noise in the trends, but it is atypical in that it contains more words than most of the synsets. Most of the sy… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 43 canonical work pages

  1. [1]

    Quantitative analysis of culture using millions of digitized books

    Michel JB, Shen YK, Aiden A, Veres A, Gray M, Pickett J, et al. Quantitative analysis of culture using millions of digitized books. Science. 2011;331:176–182

    work page 2011

  2. [2]

    Google books ngram corpus; 2012

    Google. Google books ngram corpus; 2012. Available from: http://storage.googleapis.com/books/ngrams/books/datasetsv2.html

    work page 2012

  3. [3]

    Google books ngram viewer; 2013

    Google. Google books ngram viewer; 2013. Available from: https://books.google.com/ngrams/

    work page 2013

  4. [4]

    Wordnet: An Electronic Lexical Database

    Fellbaum C, editor. Wordnet: An Electronic Lexical Database. Cambridge, MA: MIT Press; 1998

    work page 1998

  5. [5]

    WordNet: A lexical database for English; 2007

    WordNet. WordNet: A lexical database for English; 2007. Available from: https://wordnet.princeton.edu/

    work page 2007

  6. [6]

    EuroWordNet: A multilingual database for information retrieval

    Vossen P. EuroWordNet: A multilingual database for information retrieval. In: Proceedings of the DELOS workshop on Cross-language Information Retrieval; 1997

    work page 1997

  7. [7]

    EuroWordNet: Building a multilingual database with wordnets for several European languages; 2001

    Vossen P. EuroWordNet: Building a multilingual database with wordnets for several European languages; 2001. Available from: http://projects.illc.uva.nl/EuroWordNet/

    work page 2001

  8. [8]

    Concepts and methods in evolutionary biology

    Brandon R. Concepts and methods in evolutionary biology. Cambridge, UK: Cambridge University Press; 1996

    work page 1996

Show all 43 references
  1. [9]

    Conditions for evolution by natural selection

    Godfrey-Smith P. Conditions for evolution by natural selection. The Journal of Philosophy. 2007;104:489–516

    2007

  2. [10]

    Word epoch disambiguation: Finding how words change over time

    Mihalcea R, Nastase V. Word epoch disambiguation: Finding how words change over time. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics; 2012. p. 259–263. August 21, 2019 19/21

    2012

  3. [11]

    An automatic approach to identify word sense changes in text media across timescales

    Mitra S, Mitra R, Maity S, Riedl M, Biemann C, Goyal P, et al. An automatic approach to identify word sense changes in text media across timescales. Natural Language Engineering. 2015;21(5):773–798

    2015

  4. [12]

    A computational evaluation of two laws of semantic change

    Xu Y, Kemp C. A computational evaluation of two laws of semantic change. In: Proceedings of the 37th Annual Meeting of the Cognitive Science Society; 2015. p. 2703–2708

    2015

  5. [13]

    Diachronic word embeddings reveal statistical laws of semantic change

    Hamilton W, Leskovec J, Jurafsky D. Diachronic word embeddings reveal statistical laws of semantic change. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics; 2016. p. 1489–1501

    2016

  6. [14]

    On the Origin of Species

    Darwin C. On the Origin of Species. London, UK: Penguin; 1968

    1968

  7. [15]

    The natural selection of words: Software and guide to resources; 2019

    Turney P, Mohammad S. The natural selection of words: Software and guide to resources; 2019. Available from: https://github.com/pdturney/natural-selection-of-words

    2019

  8. [16]

    The descent of words: Evolutionary thinking 1780-1880

    van Wyhe J. The descent of words: Evolutionary thinking 1780-1880. Endeavour. 2005;29(3):94–100

    2005

  9. [17]

    The pleasures and perils of Darwinizing culture (with phylogenies)

    Gray R, Greenhill S, Ross R. The pleasures and perils of Darwinizing culture (with phylogenies). Biological Theory. 2007;2(4):360–375

    2007

  10. [18]

    Human language as a culturally transmitted replicator

    Pagel M. Human language as a culturally transmitted replicator. Nature Reviews Genetics. 2009;10:405–415

    2009

  11. [19]

    Cultural Evolution

    Mesoudi A. Cultural Evolution. Chicago, IL: University of Chicago Press; 2011

    2011

  12. [20]

    The Descent of Man

    Darwin C. The Descent of Man. London, UK: Gibson Square; 2003

    2003

  13. [21]

    The life and death of words

    Bolinger D. The life and death of words. The American Scholar. 1953;22(3):323–335

    1953

  14. [22]

    What’s in a Word? Studies in Phonosemantics

    Magnus M. What’s in a Word? Studies in Phonosemantics. Norwegian University of Science and Technology. Trondheim, Norway; 2001

    2001

  15. [23]

    Languages cool as they expand

    Petersen A, Tenenbaum J, Havlin S, Stanley E, Perc M. Languages cool as they expand. Scientific Reports. 2012;2(943)

    2012

  16. [24]

    Detecting evolutionary forces in language change

    Newberry M, Ahern C, Clark R, Plotkin J. Detecting evolutionary forces in language change. Nature. 2017;551:223–226

    2017

  17. [25]

    Internal and external dynamics in language: Evidence from verb regularity in a historical corpus of English

    Cuskley C, Pugliese M, Castellano C, Colaiori F, Loreto V, Tria F. Internal and external dynamics in language: Evidence from verb regularity in a historical corpus of English. PLOS ONE. 2014;9(8). doi:10.1371/journal.pone.0102882

    2014 doi

  18. [26]

    Extracting information from S-curves of language change

    Ghanbarnejad F, Gerlach M, Miotto J, Altmann E. Extracting information from S-curves of language change. Journal of The Royal Society Interface. 2014;101. doi:10.1098/rsif.2014.1044

    2014

  19. [27]

    The dynamics of norm change in the cultural evolution of language

    Amato R, Lacasa L, D´ ıaz-Guilera A, Baronchelli A. The dynamics of norm change in the cultural evolution of language. Proceedings of the National Academy of Sciences. 2018;115(33):8260–8265. doi:10.1073/pnas.1721059115

    2018 doi

  20. [28]

    Characterizing the Google Books Corpus: Strong limits to inferences of socio-cultural and linguistic evolution

    Pechenick E, Danforth C, Dodds P. Characterizing the Google Books Corpus: Strong limits to inferences of socio-cultural and linguistic evolution. PLOS ONE. 2015;doi:10.1371/journal.pone.0137041. August 21, 2019 20/21

    2015 doi

  21. [29]

    Evolution of the most common English words and phrases over the centuries

    Perc M. Evolution of the most common English words and phrases over the centuries. Journal of the Royal Society Interface. 2012;doi:10.1098/rsif.2012.0491

    2012

  22. [30]

    Lingua::EN::Syllable; 2016

    Bowers N, Fast G. Lingua::EN::Syllable; 2016. Available from: https://metacpan.org/pod/Lingua::EN::Syllable

    2016

  23. [31]

    A categorial variation database for English

    Habash N, Dorr B. A categorial variation database for English. In: Proceedings of HLT-NAACL 2003; 2003. p. 17–23

    2003

  24. [32]

    Catvar2.1: Categorial variation database for English; 2013

    Habash N, Dorr B. Catvar2.1: Categorial variation database for English; 2013. Available from: https://clipdemos.umiacs.umd.edu/catvar/download.html

    2013

  25. [33]

    A comprehensive phylogeny of beetles reveals the evolutionary origins of a superradiation

    Hunt T, Bergsten J, Levkanicova Z, Papadopoulou A, John OS, Wild R, et al. A comprehensive phylogeny of beetles reveals the evolutionary origins of a superradiation. Science. 2007;318(5858):1913–1916

    2007

  26. [34]

    Estimating continuous distributions in Bayesian classifiers

    John G, Langley P. Estimating continuous distributions in Bayesian classifiers. In: Eleventh Conference on Uncertainty in Artificial Intelligence. San Mateo: Morgan Kaufmann; 1995. p. 338–345

    1995

  27. [35]

    Data Mining: Practical Machine Learning Tools and Techniques

    Witten I, Frank E, Hall M, Pal C. Data Mining: Practical Machine Learning Tools and Techniques. San Francisco, CA: Morgan Kaufmann; 2016

    2016

  28. [36]

    Weka 3: Data mining software in Java; 2015

    Weka. Weka 3: Data mining software in Java; 2015. Available from: https://www.cs.waikato.ac.nz/ml/weka/

    2015

  29. [37]

    Information Retrieval

    van Rijsbergen C. Information Retrieval. London, UK: Butterworths; 1979

    1979

  30. [38]

    Evaluating and optimizing autonomous text classification systems

    Lewis D. Evaluating and optimizing autonomous text classification systems. In: Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM; 1995. p. 246–254

    1995

  31. [39]

    Two-sided confidence intervals for the single proportion

    Newcombe R. Two-sided confidence intervals for the single proportion. Statistics in Medicine. 1998;17(8):857–872

    1998

  32. [40]

    Analogy in suffix rivalry: The case of English -ity and -ness

    Arndt-Lappe S. Analogy in suffix rivalry: The case of English -ity and -ness. English Language and Linguistics. 2014;18(3):497–548

    2014

  33. [41]

    Oxford spelling; 2018

    Wikipedia. Oxford spelling; 2018. Available from: https://en.wikipedia.org/wiki/Oxford_spelling

    2018

  34. [42]

    Analogy and Structure

    Skousen R. Analogy and Structure. New York, NY: Springer; 1992

    1992

  35. [43]

    Improved Default Sense Selection for Word Sense Disambiguation

    Hawker T, Honnibal M. Improved Default Sense Selection for Word Sense Disambiguation. In: Proceedings of the 2006 Australasian Language Technology Workshop; 2006. p. 11–17. August 21, 2019 21/21

    2006

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.