REVIEW 4 major objections 7 minor 29 references
Word Sense Detection Leveraging Maximum Mean Discrepancy
T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper proposes MMD-Sense-Analysis, a method that uses Maximum Mean Discrepancy variable selection to identify which words change meaning across time and to explain the change through the embedding dimensions that drive it.
desk verdict A clear new application of MMD variable selection to sense change detection, but the evaluation is qualitative and the word scores are partly circular; needs benchmark validation before it can be cited as effective. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the ARD-kernel MMD estimator with sparse weights. Maximum Mean Discrepancy gives a kernel distance between two time-period embedding distributions; the ARD kernel assigns per-dimension weights whose L1 penalty drives most weights to zero, so the surviving dimensions index the sense change variables. MMD-CV-AGG avoids choosing the sparsity constant by aggregating selections over many lambda values. Words are then scored by projecting each word vector onto the selected dimensions and averaging cosine similarity across time-period pairs, so a high score means the word's embedding moves within the sense-relevant subspace.
What would settle it
Run MMD-Sense-Analysis on a synthetic corpus with a known injected shift: give one word a new sense in one period by changing only a single embedding dimension, then check whether that dimension is recovered by MMD-CV-AGG and whether the word tops the global-time score; failing that recovery would directly contradict the method's central mechanism.
Extended reading notes
Core claim
On its own terms, the paper establishes that sense change detection can be cast as a two-sample distribution shift problem in which the relevant signal is carried by a sparse subset of embedding dimensions. The algorithm first runs MMD-CV-AGG, an L1-regularized ARD-kernel variable selection procedure, on every pair of time periods to identify sense change variables. It then runs a permutation test on the selected variables to decide whether two periods differ significantly, and scores each word by the average cosine similarity between its sub-vectors restricted to the selected variables across periods. The paper reports that on two corpora, the resulting heatmaps and word rankings track documented social events, and that interpreting the output requires enough vocabulary coverage.
Load-bearing premise
The load-bearing premise is that the PMI-SVD joint word vectors from different time periods are directly comparable, so a cosine between their sense-selected sub-vectors measures semantic change rather than alignment noise; if the embedding spaces drift or are misaligned, every score and heatmap inherits that artifact.
Editorial extensions
If this is right
- If MMD-Sense-Analysis is correct, word sense change detection can be done without a learned classifier: a distribution-distance test plus sparse dimension selection is enough to rank shifted words.
- The same pipeline transfers across languages and corpus types, since it only needs aligned embedding matrices per period; the paper demonstrates it on Japanese news and historical American English.
- Period-level p-values from the permutation test give a direct, interpretable map of which years are linguistically exceptional, useful for linking corpus change to external historical events.
- Because it returns dimensions rather than just scores, the method opens the way to explaining a word's change by which co-occurrence dimensions moved, not merely by ranking the word.
- The method's reliance on abundant vocabulary is a design constraint: with too few words, the selected variables do not support social interpretation.
Reading between the lines
- A natural next test would be a benchmark with ground-truth sense changes; the author lists this as future work, and a positive result there would separate the MMD mechanism from the evocative historical coincidences in the demonstration.
- The dimension-level output suggests a testable extension: comparing the selected dimensions across different word pairs could reveal whether a single dominant sense shift drives a whole period, or whether changes are word-specific.
- The method assumes aligned embedding spaces; an informative stress test would be to train embeddings separately per period and apply the algorithm with and without post-hoc alignment, to see how much of the signal survives alignment artifacts.
- Because MMD is kernel-based, the procedure could be applied to contextualized embeddings from language models by projecting token representations of a target word into a shared space, potentially giving a fine-grained sense-change detector for individual words.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes MMD-Sense-Analysis, a method that applies Maximum Mean Discrepancy (MMD) variable selection to diachronic word embeddings in order to (i) select embedding dimensions ('sense change variables') that differ between time periods and (ii) score individual words by the average cosine distance over those dimensions across periods. The author demonstrates the method on a Japanese Mainichi Shimbun corpus (2003–2020) and on CCOHA (1810–2010), interpreting high-scoring words and period-pair heatmaps in relation to events such as the 2011 Great East Japan Earthquake, the Lehman shock, and the COVID-19 pandemic. The paper claims to be the first application of MMD to word sense change detection and states that the empirical results demonstrate the method's effectiveness.
Significance. The idea of using MMD-based variable selection for sense change detection is novel and potentially useful: it addresses the recognized problem that cosine distance in full embedding space mixes sense-relevant and sense-irrelevant dimensions. The paper is transparent about its code and about some limitations (e.g., the small-vocabulary CCOHA result). However, the central claim of effectiveness is not quantitatively supported. There is no benchmark evaluation, no comparison to existing sense change detection methods, and no word-level null model. The word score in Eq. (4) is constructed from variables that were selected to maximize the global distribution shift, so it may rank words merely for participating in that global shift rather than for word-specific sense change. As presented, the paper is an exploratory case study rather than a validated method; the stated future-work plan to evaluate on a benchmark confirms this gap.
major comments (4)
- [Section 4] The paper explicitly states 'As future work, I plan to evaluate MMD-Sense-Analysis on a benchmark dataset for the sense change detection.' This admission, combined with the absence of any comparison to existing sense change methods, gold-standard labels, or quantitative metrics (precision/recall, AP, or similar) in Sections 3.1 and 3.2, means the abstract's claim that 'Empirical assessment results demonstrate the effectiveness of the proposed approach' is not supported. The current experiments are qualitative narratives around selected words and p-value heatmaps, which do not establish effectiveness in the sense used by the field.
- [Section 2.2, Eq. (4)] The word-level score score_v^t averages cosine distances between e^{S_{t,t'}}_{v,t} and e^{S_{t,t'}}_{v,t'}, where S_{t,t'} are the variables selected to maximize the aggregate MMD between the full word distributions of periods t and t'. Because the variable sets are chosen to capture the largest global distribution shift, any word whose embedding moves along that global direction will receive a high score, regardless of whether its own sense changed independently. The score has no null baseline, is not centered or normalized over words, and is not compared to a distribution expected under a word-level null hypothesis. The saturated value 2.0 for all top words in 2011 (Table 1) is consistent with a single dominant topical shift rather than with three distinct word-specific sense changes. A word-level permutation or a comparison to scores from randomly selected variable subsets is needed to support the claim that the method identifies words undergoing sense shifts.
- [Section 3.1 (permutation test)] The description of the permutation test is incomplete. The manuscript does not specify how the permutation is performed (e.g., permuting period labels of word vectors, or permuting variable labels), which statistic is recomputed after variable selection, or whether any multiple-testing correction is applied across the 153 pairwise comparisons. Without these details, the p-value heatmaps in Figures 1 and 2 are not reproducible, and the threshold p<0.05 may yield false positives simply because of the large number of tests. This directly affects the claims that 2020 and 2009–2011 are significantly distinct from other periods.
- [Section 3.2] The second experiment (CCOHA) is presented as a demonstration, yet the authors state that the selected words for 1980s and 1990s are 'insufficient to explain the social background' and attribute this to the small vocabulary (505 words). This shows that the method's ability to 'explain the evolution over multiple historical periods', a central claim of the abstract, is not reliably achieved when the vocabulary is limited. Since the paper later generalizes (Section 4) that the method 'selects words concerning the semantic aware variables' that 'can clearly describe the social backgrounds', the CCOHA result undermines this generalization unless the scope conditions are explicitly stated and addressed.
minor comments (7)
- [Abstract / Section 4] The phrase 'qualifying similarities' in the Introduction and conclusion should be 'quantifying similarities'; the abstract uses 'quantify' but the body uses 'qualify' in several places.
- [Section 2.2, Eq. (4)] The notation in Eq. (4) is confusing: T′ is defined as 'the number of time periods without t, t′ ∈ {1, ..., T′}', but the summation index and the range over which t′ runs are not clearly specified. Please rewrite the definition to state explicitly that the sum runs over all t′ != t, and define the cosine distance (e.g., 1 − cosine similarity).
- [Section 3.2] In the paragraph before Table 3, the text says 'Table 1 represents the top three words...' but the displayed table is numbered 'Table 3'. Please correct the cross-reference.
- [Section 2.2] The description of the permutation test is vague: 'The p-value represents the degree to which the observation data pair Et, Et′ is significantly against the null hypothesis' is not a standard statistical statement. Provide a concrete permutation procedure and the exact test statistic.
- [Section 3.1] The paper reports no error bars or repeated runs over the random selection of 2,300 vocabularies and the 2,000/300 split. Given that the top-word lists (Table 1) are a main result, their stability under different random seeds should be reported or at least discussed.
- [Section 2.1] The ARD kernel in Eq. (2) includes constants γ_d said to 'unit-normalize each variable', but the paper does not state how these are computed beyond citing [16, Appendix C]. A brief summary would make the method self-contained.
- [Section 3.1] The paper uses PMI-SVD joint embeddings to ensure comparability across time periods, but does not explicitly explain why these embeddings are in a shared vector space. Please add a sentence citing the joint factorization property from [2] or [9] to justify the use of cosine distance in Eq. (4).
Circularity Check
Word-level scores are computed on the same MMD-selected global shift directions that were fitted to the full corpus, so the reported 'sense-changed' words are a projection of that global fit rather than an independent word-level prediction.
-
fitted input called prediction
[Section 2.2, Eq. (4), Table 1 (building on Eq. (3) and MMD-CV-AGG)]
"Given {St,t′|t, t′ ∈ {1, .., T}, t̸= t′}, the scoring function is defined as score_v^t = 1/T' Σ_{t'≠t} cosine(e^{S_{t,t'}}_{v,t}, e^{S_{t,t'}}_{v,t'}) ... This score expresses the average of cosine distances between the word embedding vectors using only sense change variables; therefore, a higher score value implies that a vocabulary v is significantly related to the sense change variables at the time periods t."
The variables S are not neutral or independently validated semantic dimensions. They are fitted by Eq. (3) via MMD-CV-AGG to maximize the aggregate MMD between the full embedding distributions of the two periods, i.e., to find whichever global directions best separate all words in period t from all words in period t'. Eq. (4) then scores an individual word by the cosine distance between its two period vectors restricted to those same fitted directions. A high score is therefore, by construction, a word whose vector moved along the global corpus-shift directions; it is not evidence of word-specific sense change. The paper then presents the top scorers in Table 1 as words undergoing sense shifts and explains them by earthquakes and COVID.
full rationale
The core method is internally coherent: MMD variable selection is described, Eqs. (1)-(3) are concrete, and the use of the author's earlier MMD variable-selection preprint [16] is transparently cited, so the self-citation alone is not treated as circular. However, the paper's word-level claim—that it identifies words whose senses changed—is not an independent output. The subset S in Eq. (4) is fitted to maximize the aggregate distribution shift over all words in the two periods; the word score simply measures how much each word's vector moves along those already-selected global directions. The saturated 2.0 scores for all top 2011 words in Table 1 are consistent with one dominant global direction rather than with three independently detected sense changes. The paper also flags missing benchmark validation in the conclusion and notes in Section 3.1 that it cannot inspect the raw corpus to verify words like 'golf' and 'carbon,' which further weakens the post hoc historical-event interpretation. These limitations are stated in the text and are weighed here. Still, the derivation is not fully equivalent to its input: the MMD objective and the cosine scoring are distinct functions, and the historical-event checks provide external, if weak, evidence. The partial circularity is centered on using the fitted global shift directions as the word-level predictor, so the appropriate score is 5 rather than 8 or 10.
Assumptions & free parameters
free parameters (3)
- lambda (L1 regularization constant in Eq. 3) =
not reported
- PMI-SVD joint embedding hyperparameters =
not reported
- Random vocabulary sampling size and split =
2300 words total, 2000 train / 300 test
assumptions (5)
- standard math MMD variable selection with ARD kernel and L1 regularization correctly identifies variables where two distributions differ.
- domain assumption PMI-SVD joint word embeddings from different time periods are directly comparable in a shared vector space.
- domain assumption Sense change variables selected on 2,000 training vocabularies generalize to the 300 held-out vocabularies in the permutation test.
- domain assumption Nouns are the word class most relevant for sense change detection.
- domain assumption Known historical events (Lehman shock, 2011 earthquake, COVID-19, Civil War) are a valid ground truth for evaluating detected sense changes.
Cite this review
Pith. "Pith review of Word Sense Detection Leveraging Maximum Mean Discrepancy." pith.science (2026). https://pith.science/paper/ENODBTXZ
@misc{pith2026250601602,
author = {Pith},
title = {Pith review of: Word Sense Detection Leveraging Maximum Mean Discrepancy},
year = {2026},
howpublished = {\url{https://pith.science/paper/ENODBTXZ}},
note = {Machine review of arXiv:2506.01602}
}
read the original abstract
Word sense analysis is an essential analysis work for interpreting the linguistic and social backgrounds. The word sense change detection is a task of identifying and interpreting shifts in word meanings over time. This paper proposes MMD-Sense-Analysis, a novel approach that leverages Maximum Mean Discrepancy (MMD) to select semantically meaningful variables and quantify changes across time periods. This method enables both the identification of words undergoing sense shifts and the explanation of their evolution over multiple historical periods. To my knowledge, this is the first application of MMD to word sense change detection. Empirical assessment results demonstrate the effectiveness of the proposed approach.
Figures
Reference graph
Works this paper leans on
-
[1]
A semantic distance metric learning approach for lexical semantic change detection
Taichi Aida and Danushka Bollegala. A semantic distance metric learning approach for lexical semantic change detection. In Findings of the Association for Computational Linguistics: ACL 2024, pages 7570–7584, August 2024
work page 2024
-
[2]
A comprehensive analysis of PMI-based models for measuring semantic differences
Taichi Aida, Mamoru Komachi, Toshinobu Ogiso, Hiroya Takamura, and Daichi Mochihashi. A comprehensive analysis of PMI-based models for measuring semantic differences. In Proceedings of the 35th Pacific Asia Conference on Language, Information and Computation, pages 21–31, 11 2021
work page 2021
-
[3]
CCOHA: Clean corpus of historical American English
Reem Alatrash, Dominik Schlechtweg, Jonas Kuhn, and Sabine Schulte im Walde. CCOHA: Clean corpus of historical American English. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 6958–6966, May 2020
work page 2020
-
[4]
Universals versus historical contingencies in lexical evolution
Vladimir Bochkarev, Valery Solovyev, and Søren Wichmann. Universals versus historical contingencies in lexical evolution. Journal of The Royal Society Interface, 11(101):20140841, 2014
work page 2014
-
[5]
Japan’s Lost Decade: Policies for Economic Revival
Tim Callen and Jonathan David Ostry. Japan’s Lost Decade: Policies for Economic Revival . International Monetary Fund, USA, 2003
work page 2003
-
[6]
Borgwardt, Malte J
Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(25):723–773, 2012
2012
-
[7]
T. Hofmann, B. Schölkopf, and A. J. Smola. Kernel methods in machine learning.Annals of Statistics, 36(3):1171– 1220, 2008
work page 2008
-
[8]
Yoshio Kajitani and Hirokazu Tatano. Estimation of production capacity loss rate after the great east japan earthquake and tsunami in 2011. Economic Systems Research, 26(1):13–38, 2014
work page 2011
Show all 29 references
-
[9]
Analyzing continuous semantic shifts with diachronic word similarity matrices
Hajime Kiyama, Taichi Aida, Mamoru Komachi, Toshinobu Ogiso, Hiroya Takamura, and Daichi Mochihashi. Analyzing continuous semantic shifts with diachronic word similarity matrices. In Proceedings of the 31st International Conference on Computational Linguistics, pages 1613–1631...
2025
-
[10]
Diachronic word embeddings and semantic shifts: a survey
Andrey Kutuzov, Lilja Øvrelid, Terrence Szymanski, and Erik Velldal. Diachronic word embeddings and semantic shifts: a survey. In Proceedings of the 27th International Conference on Computational Linguistics , pages 1384–1397, August 2018
2018
-
[11]
Mind the gap: Assessing temporal generalization in neural language models
Angeliki Lazaridou, Adhi Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d'Autume, Tomas Kocisky, Sebastian Ruder, Dani Yogatama, Kris Cao, Susannah Young, and Phil Blunsom. Mind the gap: Assessing temporal generalization in...
2021
-
[12]
Neural word embedding as implicit matrix factorization
Omer Levy and Yoav Goldberg. Neural word embedding as implicit matrix factorization. In Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014
2014
-
[13]
TimeLMs: Diachronic language models from Twitter
Daniel Loureiro, Francesco Barbieri, Leonardo Neves, Luis Espinosa Anke, and Jose Camacho-collados. TimeLMs: Diachronic language models from Twitter. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 251–260, 2022
2022
-
[14]
Distributed representations of words and phrases and their compositionality
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, volume 26, 2013
2013
-
[15]
Damage from the great east japan earthquake and tsunami-a quick report
Nobuo Mimura, Kazuya Yasuhara, Seiki Kawagoe, Hiromune Yokoki, and So Kazama. Damage from the great east japan earthquake and tsunami-a quick report. Mitigation and adaptation strategies for global change, 16:803–818, 2011
2011
-
[16]
Variable selec- tion in maximum mean discrepancy for interpretable distribution comparison
Kensuke Mitsuzawa, Motonobu Kanagawa, Stefano Bortoli, Margherita Grossi, and Paolo Papotti. Variable selec- tion in maximum mean discrepancy for interpretable distribution comparison. arXiv preprint arXiv:2311.01537, 2023
2023
-
[17]
Muandet, K
K. Muandet, K. Fukumizu, B. K. Sriperumbudur, and B. Schölkopf. Kernel mean embedding of distributions : A review and beyond. Foundations and Trends in Machine Learning, 10(1–2):1–141, 2017
2017
-
[18]
Influence of natural disasters on social engagement and post-disaster well-being: The case of the great east japan earthquake
Naoko Okuyama and Yoji Inaba. Influence of natural disasters on social engagement and post-disaster well-being: The case of the great east japan earthquake. Japan and the World Economy, 44:1–13, 2017
2017
-
[19]
Lexical semantic change through large language models: a survey
Francesco Periti and Stefano Montanelli. Lexical semantic change through large language models: a survey. ACM Comput. Surv., 56(11), June 2024
2024
-
[20]
Lexical change and stability in 100 years of english in us newspapers
Robert Poole and Qudus Ayinde Adebayo. Lexical change and stability in 100 years of english in us newspapers. Applied Corpus Linguistics, 3(3):100073, 2023
2023
-
[21]
Performance analysis of japanese manufacturing industry before and after lehman shock
Yukari Shirota and Michiya Morita. Performance analysis of japanese manufacturing industry before and after lehman shock. International Journal of Trade, Economics and Finance, 11(5):87–91, 2020
2020
-
[22]
Relative novelty detection
Alex Smola, Le Song, and Choon Hui Teo. Relative novelty detection. In Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics , volume 5 of Proceedings of Machine Learning Research, pages 536–543, Hilton Clearwater Beach Resort, Clearwate...
2009
-
[23]
Sutherland, Hsiao-Yu Tung, Heiko Strathmann, Soumyajit De, Aaditya Ramdas, Alex Smola, and Arthur Gretton
Danica J. Sutherland, Hsiao-Yu Tung, Heiko Strathmann, Soumyajit De, Aaditya Ramdas, Alex Smola, and Arthur Gretton. Generative models and model criticism via optimized maximum mean discrepancy. In International Conference on Learning Representations, 2017
2017
-
[24]
Regression shrinkage and selection via the lasso
Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), 58(1):267–288, 1996
1996
-
[25]
vaderland”, volk” and natie”: Semantic change related to nationalism in Dutch literature between 1700 and 1880 captured with dynamic Bernoulli word embeddings
Marije Timmermans, Eva Vanmassenhove, and Dimitar Shterionov. vaderland”, volk” and natie”: Semantic change related to nationalism in Dutch literature between 1700 and 1880 captured with dynamic Bernoulli word embeddings. In Proceedings of the 3rd Workshop on Computational App...
2022
-
[26]
Variable selection for kernel two-sample tests
Jie Wang, Santanu S Dey, and Yao Xie. Variable selection for kernel two-sample tests. arXiv preprint arXiv:2302.07415, 2023
2023 arXiv
-
[27]
Rethinking maximum mean discrepancy for visual domain adaptation
Wei Wang, Haojie Li, Zhengming Ding, Feiping Nie, Junyang Chen, Xiao Dong, and Zhihui Wang. Rethinking maximum mean discrepancy for visual domain adaptation. IEEE Transactions on Neural Networks and Learning Systems, 34(1):264–277, 2021
2021
-
[28]
The lehman brothers bankruptcy a: overview.Journal of financial crises, 1(1):39–62, 2019
Rosalind Z Wiggins, Thomas Piontek, and Andrew Metrick. The lehman brothers bankruptcy a: overview.Journal of financial crises, 1(1):39–62, 2019
2019
-
[29]
Mind the class weight bias: Weighted maximum mean discrepancy for unsupervised domain adaptation
Hongliang Yan, Yukang Ding, Peihua Li, Qilong Wang, Yong Xu, and Wangmeng Zuo. Mind the class weight bias: Weighted maximum mean discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2272–2281, 2017
2017
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.