{"id":"f82b2fb7-a834-40ef-9036-752029b4cec8","arxiv_id":"1908.07013","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A Naive Bayes classifier using word form and historical frequency features predicts the future leader of a synonym set fifty years ahead with F-scores of 38.5 to 43.3 percent, compared with 17.3 to 24.8 percent for random guessing.","lead":"This paper builds a dataset of 4,000 synonym groups with word frequencies from 1800 to 2000 and shows a machine learning model can predict which synonym will become most common fifty years later, doing better than random guessing. It also uses the model's internal patterns to suggest why some words win, such as a preference for shorter, more distinctive forms.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"WordNet synset stability is the load-bearing assumption; a birth-year check on the released data would test it directly.","rationale":"The reader's weakest assumption is synset stability, and I agree that this is the most load-bearing issue. A predictive result on mislabeled data would not establish the paper's biological analogy. The paper's own limitation section states the assumption and offers only anecdotal support ('Inspection of WordNet shows that it contains many archaic words'), so the concern is explicitly grounded in the manuscript. I considered the alternative concern that the random baseline is weak, but the paper's Table 8 shows linear extrapolation alone nearly reproduces the full-system F-score, so the core empirical finding is robust to that objection; the label-validity issue is more fundamental. The proposed test is feasible because the authors already compute birth years for Feature 8, and the dataset is released. If the test shows many late-born words, the central claim would need substantial qualification; if it shows few, the conditional verdict can be lifted. Thus the reader's CONDITIONAL verdict is appropriate and no change is needed.","tokens_in":16892,"tokens_out":15402,"duration_ms":167924,"concrete_test":"Using the released dataset, compute for each synset the maximum over member words of the GBNC first-occurrence year (the same quantity used for Feature 8). Then, first, report the fraction of synsets containing at least one member whose first occurrence is after 1800; second, rerun the Table 6 NBCP evaluation after removing all such synsets. If the F-score drops materially or the percentage of changed synsets changes sharply, the stability assumption is doing real work and the conditional verdict stands; if the excluded set is tiny and results are unchanged, the concern is settled.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that synset leadership changes are predictable rests on the labels being historically meaningful. The dataset maps every word in a modern WordNet synset to Google Books Ngram frequencies from 1800-2000, and the paper assumes the synset is stable over that interval: 'In this study, we assume that WordNet synsets are stable over the period of time we consider, 1800 to 2000.' If a word had a different sense in 1800 or entered the synset later, its historical token counts are not counts of the same meaning, so the leader calculation mixes different concepts and the 'competition' is not among words expressing a fixed meaning. The paper acknowledges this at the limitation section and asserts that WordNet covers archaic words, but it provides no systematic verification. The released dataset itself contains the means to test one aspect: the 'present age' feature (Feature 8) is computed from the first nonzero GBNC year, so any member word with first occurrence after 1800 was absent from the active competition set at the beginning of the period. If this is common, early-period leader labels are anachronistic, and the predictive F-score may reflect predicting sense-shift or word-entry artifacts rather than the natural selection of words within a stable meaning.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a dataset that combines WordNet synsets with Google Books Ngram Corpus frequencies, restricted to monosemous words, and tracks the relative frequencies of synonyms from 1800 to 2000. The central task is to predict, fifty years ahead, which member of a synset will become the most frequent, phrased as a binary winner/loser classification at the word level and aggregated to synset-level leader prediction. The authors use a naive Bayes classifier with features based on word length, syllable count, trigram uniqueness, shared trigrams, categorial variations, relative growth, linear extrapolation, and present age. They report F-scores of 38.5--43.3% for identifying leadership changes, compared with 17.3--24.8% for a random baseline, and interpret the learned models as evidence for suffix rivalry, economy-versus-clarity tradeoffs, and decreasing language change over time. The paper also releases the dataset and processing software.","tokens_in":17130,"tokens_out":6401,"duration_ms":64648,"significance":"If the empirical claim holds, this is a useful demonstration that lexical replacement within a fixed meaning is at least partially predictable, and the released dataset provides a reusable resource for diachronic lexical studies. The main experiment is an out-of-sample evaluation with proper temporal separation: predictive features are computed only from past and present data, and the target labels come from fifty years later. The paper also gives credit for consistent reporting of precision, recall, and F-score against a random baseline, and its feature-level analysis reproduces known linguistic trends such as the -ise/-ize shift and the -ity/-ness rivalry. The significance is tempered by the assumption of stable WordNet synsets over two centuries, which is load-bearing for the meaning of the leader labels, and by data-filtering choices that restrict the scope of the claim.","major_comments":[{"comment":"The paper explicitly assumes that WordNet synsets are stable from 1800 to 2000, but this assumption is load-bearing for the central prediction task. Historical GBNC token counts for a word are attributed to a modern synset even if the word's sense changed or the word entered the synset later; if early-period leaders are anachronistic assignments, the F-score may partly reflect predicting sense shifts or word-entry artifacts rather than competition within a fixed meaning. The authors offer only qualitative assurances and no quantitative verification. Because Feature 8 (Present age) is computed from the first nonzero GBNC year for each word, the released data directly supports a test: report the distribution of word birth years, the fraction of synset members absent at the start of each period, and the sensitivity of leader labels and of the reported F-scores to restricting synsets to words present throughout the interval.","section":"Potential Limitations of GBNC and WordNet; Step 3; Feature 8"},{"comment":"Step 4 removes any synset with a tie for first place in the present or future and removes synsets containing words with zero frequency in the present. This filtering excludes scientifically important outcomes of selection, including co-leadership and word death, and it changes the target distribution: the reported F-score is conditional on a unique winner at both endpoints and on no dead words at the present. The paper states without supporting analysis that including dead words would 'artificially inflate' the algorithm's score; this is an empirical claim that should be tested. Please report how many synsets are discarded by each criterion and provide a robustness check with less restrictive filtering, such as treating dead words as legitimate losers and handling ties by random or conservative assignment.","section":"Building datasets, Step 4"},{"comment":"The primary 50-year-cycle evaluation rests on only two test sets, Test1 and Test2, and the reported Wilson confidence intervals treat the synsets in each test set as independent samples. This does not account for temporal autocorrelation or for the small number of temporal instantiations, so the intervals do not directly support a general claim about predictability across time. The varying-cycle experiments add more test periods, but those periods are not independent of one another either. Please add a time-based analysis, for example a bootstrap that resamples time periods, or at least clearly state that the confidence intervals are within-test-set intervals and discuss the limited temporal generality of the 50-year result.","section":"Experiments with NBCP, Table 6"}],"minor_comments":[{"comment":"The description of Feature 3 says 'we want to focus on the features that are unique to the winner,' but the construction actually computes trigrams unique to each word within its synset and does not use the target class. This wording could be misread as target leakage and should be rephrased.","section":"Feature vectors for words, Feature 3"},{"comment":"The top trigrams in Table 12 are selected after sorting by the largest class-mean differences, and then each difference is tested for significance with a t-test. This selection procedure creates a multiple-comparisons problem, so the statement that all differences are significant is not justified as reported; adjusted p-values or a validation-based selection procedure would be more appropriate.","section":"Interpretation of the learned models, Table 12"},{"comment":"The random baseline is a reasonable reference, but the trivial 'always predict the present leader' baseline, which has zero F-score by construction, is too weak to be informative on its own. Reporting a persistence baseline that predicts no change would help calibrate how much of the advantage over random is due to the task's stability prior.","section":"Experiments with NBCP, Tables 6 and 9"},{"comment":"There are several typographical errors: 'signficantly' in the Table 8 caption, 'contribtion' in the text preceding Table 8, and 'hight' in the Future Work section. These should be corrected.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The release of the dataset and software is a real strength, and the temporal holdout design rules out the most obvious circularity concern. The main reason for major revision is that the synset-stability assumption is acknowledged but never quantitatively checked, and the released data contains exactly the information needed to perform such a check via Feature 8. I would not recommend rejection because the issue is fixable within the manuscript's scope, but the central claim about the predictability of leadership change should not be accepted until the anachronism risk is quantified."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. The headline result is real: a naive Bayes classifier using past and present frequencies can predict synset leadership change fifty years ahead with an F-score around 40%, versus roughly 20% for the random baseline, and the evaluation is a genuine temporal holdout. The more lasting contribution is the dataset itself: 4,000 monosemous-word synsets aligned with Google Books Ngram frequencies from 1800 to 2000, released with software.\n\nWhat is genuinely new here is the framing. Most diachronic word work studies the same word changing meaning; this paper studies the same meaning changing its preferred word. The task is well operationalized, and the feature analysis is honest. Linear extrapolation is the dominant feature, which is interesting in itself: it suggests the selective pressure on words is fairly constant over decades. The suffix effects the model recovers (ise losing to ize, ness losing to ity) match known linguistics, which is good confirmatory evidence that the setup is capturing something real.\n\nThe soft spots are proportionate to how soft they are. The load-bearing assumption is WordNet synset stability over 1800–2000. The authors state it explicitly in the limitations section and argue that WordNet covers archaic words well, but they do not verify it. The released dataset contains the means for a direct check: Feature 8 (present age) records the first nonzero GBNC year for each word, so one can count how many member words were absent in 1800. If that number is large, early leader labels are anachronistic and some of the predictive signal may be about word entry rather than selection within a stable set. This is a testable concern, not a demonstrated flaw.\n\nTwo smaller issues. The main fifty-year result rests on only two test sets (Test1 and Test2), though the cycle-length experiments add some confidence. And the interpretation of 3,660 unique trigrams is multiple-testing-heavy: the top dozen are all significant by individual t-tests, but there is no multiple-comparison correction. Since this analysis is interpretive rather than load-bearing, I would call it minor.\n\nOverall, the central predictive claim holds up as an out-of-sample result, the limitations are acknowledged candidly, and the dataset is reusable. This paper is for people working on diachronic lexical dynamics, cultural evolution, or GBNC methodology. It deserves serious refereeing, not a desk reject. My recommendation is to engage with it, and to ask the authors to run the birth-year check on their own data.","headline":"Predicting which synonym will win a WordNet synset fifty years out works modestly but robustly; the released dataset is the most durable contribution.","tokens_in":17630,"tokens_out":1809,"would_cite":true,"duration_ms":21873,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A synonym's chance of dominating its meaning can be predicted fifty years ahead at roughly double the accuracy of random guessing.","keywords":["synset","word frequency","language change","natural selection","naive Bayes","Google Books Ngram Corpus","WordNet","lexical replacement"],"falsifier":"Check a sample of the 4,000 synsets against historical dictionaries or sense-annotated texts to see whether synset membership actually stayed fixed from 1800 to 2000. If a material fraction of synsets gained or lost words, rerun the classifier on only the historically stable synsets; if the F-score advantage over random guessing vanishes, the predictability result depends on the stability assumption.","tokens_in":16710,"feed_emoji":"📈","tokens_out":8669,"duration_ms":86782,"temperature":0.7,"pith_summary":"The paper's central claim is that competition among synonyms is partly predictable: the word that will dominate a meaning fifty years later can be forecast from cheap surface features and past frequency trends. The authors assemble 4,000 WordNet synsets—sets of interchangeable words—covering 9,000 single-sense English words, with frequencies from the Google Books Ngram Corpus sampled at fifty-year intervals from 1800 to 2000. A supervised classifier predicts the future leader of each synset with F-scores of 38.5–43.3%, versus 17.3–24.8% for random guessing, and it also reproduces known linguistic pressures such as -ize overtaking -ise and -ity overtaking -ness. The payoff is that language change is not just noise: it has a measurable selective direction, and that direction can be learned from data.","feed_headline":"Which synonym wins is predictable 50 years out","feed_subtitle":"A model trained on 200 years of book frequencies beats random guessing by roughly two to one.","key_machinery":"The machinery is the synset treated as an evolutionary unit: each word's token count is its population, and the highest-frequency word is the leader. To make the problem a classification, the paper aligns three time points (past, present, future) separated by fifty years, with future data hidden at test time. Each word is converted to a feature vector with eight components: normalized length, syllable count, unique letter trigrams, the fraction of trigrams shared with synset rivals, number of categorial variations, relative growth from past to present, linear extrapolation of relative frequency, and present age. The classifier is naive Bayes with Gaussian class-conditional densities; synset-level predictions are made by picking the word with the highest estimated win probability. The mapping between the two corpora is made by restricting to single-sense WordNet words, so that a part-of-speech-tagged frequency can be assigned unambiguously to one sense.","core_discovery":"The central claim is that a synset—a set of synonymous words in WordNet—has a leader (its highest-frequency member), and that leadership is governed by a predictable selective process. Using frequencies from the Google Books Ngram Corpus sampled every fifty years from 1800 to 2000, a naive Bayes classifier built on word length, syllable count, distinctive trigrams, relative growth, linear extrapolation, and word age predicts the leader fifty years later with F-scores of 38.5–43.3%, compared with 17.3–24.8% for random guessing. The paper further claims that the learned models are interpretable: winners tend to be shorter but still distinctive, older, and growing faster than their rivals, and the trigram analysis reproduces documented trends such as -ize replacing -ise and -ity gaining on -ness. The conclusion is that lexical replacement in this period is not random drift but is shaped by persistent selective pressures.","pith_inferences":["An untested extension is to apply the same feature set to non-book corpora, such as newspapers or transcribed speech; if the predictive advantage shrinks there, the signal may be specific to the written, bookish register of the Google Books sample.","The paper's evolutionary framing suggests a transferable prediction: within any competitive set with measurable growth rates and distinctiveness, such as brand shares in a product category, the same class of features should forecast future leaders if cultural selection is the underlying mechanism.","A further implication of the coupling between meaning-change and word-change, which the paper leaves untested, is that a word's loss of leadership could be anticipated by tracking its rising ambiguity in historical text; the authors note this is enabled only by full sense-tagging of the corpus."],"forward_implications":["A fifty-year-ahead prediction of a synset's leader is feasible at F-scores of 38.5–43.3%, roughly double the 17.3–24.8% of random guessing, on test periods ending in 1950 and 2000.","Linear extrapolation of relative frequency is the strongest single feature, implying that the direction of selection is steady enough over decades to be projected.","Leadership change is frequent but declining: 42% of synsets changed leader at least once between 1800 and 2000, and the percentage changing in a fifty-year window falls over time.","The model's trigram analysis supports known morphological pressures: -ize is replacing -ise, -ity is gaining on -ness, and short words win only when they keep enough distinctive characters."],"supporting_citations":[{"why":"Supplies the Google Books Ngram Corpus, the source of historical word frequency counts that define leaders.","marker":"[1]"},{"why":"Supplies the specific ngram dataset (English Version 20120701) from which frequencies are extracted.","marker":"[2]"},{"why":"Defines WordNet and the synset structure that serves as the unit of competition.","marker":"[4]"},{"why":"Provides the naive Bayes classifier with Gaussian density estimation used to generate win probabilities.","marker":"[34]"},{"why":"Defines precision, recall, and F-score, the evaluation measures behind the central performance claim.","marker":"[37]"},{"why":"Documents the historical rivalry between -ity and -ness that the trigram analysis reproduces.","marker":"[40]"},{"why":"Supplies the 'languages cool as they expand' hypothesis that the paper's declining change rate confirms.","marker":"[23]"}],"fun_headline_variants":["Synonym survival: predict the winner 50 years early","Book ngrams forecast which word will lead a synonym set","Machine learning predicts synonym dominance from book history","Word fitness: algorithm picks winning synonym five decades ahead","Fifty-year word evolution predicted from book frequency data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that WordNet's synonym sets describe the same meanings in 1800 as they do today; if words entered or left a synset over those two centuries, the frequency counts, leader labels, and the whole prediction task would be unreliable.","fun_headline_variants_meta":{"raw":{"variants":["Synonym survival: predict the winner 50 years early","Book ngrams forecast which word will lead a synonym set","Machine learning predicts synonym dominance from book history","Word fitness: algorithm picks winning synonym five decades ahead","Fifty-year word evolution predicted from book frequency data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000558,"raw_usage":{"total_tokens":2661,"prompt_tokens":961,"completion_tokens":1700,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":577,"completion_tokens_details":{"reasoning_tokens":1624}},"tokens_in":577,"tokens_out":1700,"duration_ms":11895,"temperature":1.0,"reasoning_tokens":1624,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:28:14.699149+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Check a sample of the 4,000 synsets against historical dictionaries or sense-annotated texts to see whether synset membership actually stayed fixed from 1800 to 2000. If a material fraction of synsets gained or lost words, rerun the classifier on only the historically stable synsets; if the F-score advantage over random guessing vanishes, the predictability result depends on the stability assumption.","supporting_citations":[{"cited_title":"Quantitative analysis of culture using millions of digitized books","cited_arxiv_id":null,"evidence_quote":"Supplies the Google Books Ngram Corpus, the source of historical word frequency counts that define leaders."},{"cited_title":"Google books ngram corpus; 2012","cited_arxiv_id":null,"evidence_quote":"Supplies the specific ngram dataset (English Version 20120701) from which frequencies are extracted."},{"cited_title":"Wordnet: An Electronic Lexical Database","cited_arxiv_id":null,"evidence_quote":"Defines WordNet and the synset structure that serves as the unit of competition."},{"cited_title":"Estimating continuous distributions in Bayesian classifiers","cited_arxiv_id":null,"evidence_quote":"Provides the naive Bayes classifier with Gaussian density estimation used to generate win probabilities."},{"cited_title":"Information Retrieval","cited_arxiv_id":null,"evidence_quote":"Defines precision, recall, and F-score, the evaluation measures behind the central performance claim."},{"cited_title":"Analogy in suffix rivalry: The case of English -ity and -ness","cited_arxiv_id":null,"evidence_quote":"Documents the historical rivalry between -ity and -ness that the trigram analysis reproduces."},{"cited_title":"Languages cool as they expand","cited_arxiv_id":null,"evidence_quote":"Supplies the 'languages cool as they expand' hypothesis that the paper's declining change rate confirms."}],"review_version":1}