REVIEW 4 major objections 5 minor 28 references
Neighbors and relatives: How do speech embeddings reflect linguistic connections across the world?
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Speech embeddings of 106 languages track both genealogy and geography.
desk verdict Global-scale speech-embedding distances that are plausible and useful, but the headline correlation is chosen in-sample and overstated; needs out-of-sample validation before the strong claims. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the LDA-compressed language embedding. Utterance-level 512-dimensional vectors from the final pooling layer of the LID model are projected by linear discriminant analysis using language labels as classes; the per-language centroid in the resulting 105-dimensional discriminant space is a language embedding, and cosine distances between normalized centroids are the pairwise language distances used in all analyses. LDA suppresses within-language variation such as the gender-correlated structure visible in PCA, keeping the between-language signal, and the paper deliberately uses all LDA dimensions because correlation with lexical distance keeps increasing with dimensionality. A secondary mechanism is the benchmark setup: ASJP lexical distances computed by the Levenshtein Distance Normalized Divided method stand in for genealogy, and Glottolog coordinates supply geography, with transforms applied before regression.
What would settle it
The decisive check is a held-out split: fit the projection and choose the dimension count using only half of the languages, then measure the embedding-distance correlations on the untouched half. If the combined correlation falls from the reported 0.56 toward the 0.36 correlation between geography and lexicon alone, the headline alignment is mostly an artifact of in-sample dimension selection. A second check is to re-run the analysis on a family-balanced subset of the languages; if Europe's dense sampling drives the result, non-European families would show little or no alignment.
Extended reading notes
Core claim
The central claim is that the sound-based latent space of a fine-tuned language-identification model is a valid proxy for both genealogy and geography. For each of 106 languages, the 512-dimensional utterance embeddings are projected with LDA onto language-discriminant axes, and the language's centroid in that space represents the language; cosine distance between centroids is the language distance. Against external benchmarks, these distances track Glottolog coordinates and ASJP lexical distances, and the two influences combine in a regression that reaches adjusted $R^2 = 0.31$ ($r = 0.56$). The correlation with lexical distance grows as more LDA dimensions are added, while the first few dimensions already capture geography, so the geometry encodes contact as well as descent. Languages absent from the LID model's fine-tuning data are placed sensibly in the resulting trees and networks, supporting the idea that the embedding geometry generalizes beyond the training set.
Load-bearing premise
All the reported correlations assume that fitting the data-reduction projection and choosing to keep all of its 105 dimensions on the same 106-language dataset, after watching agreement with the lexical and geographic benchmarks rise with dimension count, does not inflate the numbers; if it does, the alignment would be weaker on new recordings and on languages the identification model was never trained on.
Editorial extensions
If this is right
- Around 50 recordings per language are enough for the distance structure to stabilize, so the approach scales to small corpora and low-resource languages.
- Languages absent from the LID model's fine-tuning data still land in sensible places in the trees and network, so the method transfers to unseen languages without retraining.
- Embedding distances respond to both descent and contact: the first few LDA dimensions track geography, higher dimensions steadily add lexical and genealogical signal, and the combined model reaches a correlation of 0.56 with geography and lexicon together.
- The negative interaction between geography and lexical distance means geographic proximity matters most for closely related languages; for distantly related pairs, geography's influence on sound similarity weakens.
- Outlier pairs are interpretable as contact and multilingualism effects, such as Uralic languages spoken in Russia clustering with Slavic and Turkic neighbors, so the method offers a synchronic window on language change.
Reading between the lines
- A practical extension the paper leaves implicit: with roughly 50 recordings per language being enough for stable distances, the method becomes a fieldwork triage tool, collect a small speech sample from an undocumented language, embed it, and rank candidate relatives by cosine distance before doing traditional comparative work.
- Because the raw 512-dimensional embeddings show gender-correlated structure, any sociolinguistic extension that reuses this pipeline should control speaker gender and age; LDA damps this signal but does not remove it, so distances between speaker groups could partly reflect demographics.
- The negative geography-by-lexicon interaction is a specific, testable prediction about language contact: geography's influence on how languages sound should be strongest among close relatives, and running the same regression inside individual world regions would test whether that pattern is universal or an artifact of the dataset's European majority.
- The paper's own removal experiments with Breton, Albanian, and Portuguese show that changing the language roster rearranges distant branches of the global tree; a natural next study would map how much each added language shifts macro-level clusters, treating tree instability as a measurable quantity.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper extracts 512-dimensional embeddings from a fine-tuned XLS-R language identification model (voxlingua107-xls-r-300m-wav2vec) on Common Voice test recordings for 106 languages, applies linear discriminant analysis (LDA) using language labels, and defines each language by the centroid of its recordings in LDA space. Cosine distances between these language embeddings are then compared with ASJP lexical distances and Glottolog geographical distances, using Pearson correlations and linear models, and the resulting geometry is visualized with dendrograms, NeighborNet networks, and maps. The paper also tests robustness by resampling recordings for languages with at least 1000 samples. The central claim is that embedding-based distances align closely with lexical and geographical distances and capture both global and local typological patterns, including for languages not seen during LID fine-tuning.
Significance. If the central claim holds, the paper offers a scalable, speech-based complement to text-based and expert-driven approaches to studying language relationships, with potential applicability to low-resource and unwritten languages. The study has notable strengths: it compares against external benchmarks (ASJP lexical distances, Glottolog geography) rather than relying only on internal clustering consistency; it explicitly discusses the overlap between the LID model's training languages and the analyzed languages; and it includes a resampling-based robustness analysis. However, the main quantitative evidence is weakened by in-sample model selection and by the absence of held-out validation, so the paper's central claim is currently stronger than what the reported numbers establish.
major comments (4)
- [Results, Fig. 3] The headline correlation is an in-sample quantity. The text states that correlations with lexical distance 'steadily increase with added LDA dimensions' and that 'therefore in the remainder of this paper we use the embedding-based distances calculated using all available dimensions.' Because the same Common Voice embeddings are used to fit the LDA, to select its dimensionality, and to compute the r=0.56 (adjusted R²=0.31) reported in the full model, the reported fit is not an estimate of how well the method would perform on new recordings or on languages not used in model selection. Please add an out-of-sample validation, e.g., cross-validation that refits the LDA on a training subset and evaluates cosine distances on a held-out subset, and report correlations for held-out data. Without this, the sentence in the abstract that embedding distances 'align closely' with traditional measures is not supported.
- [Discussion, 'LID model' paragraph] The claim that languages absent from the LID model's training data are clustered appropriately is supported only by visual inspection of Figs 8 and 9. Because 32 of the 106 languages were not used in LID fine-tuning, this claim is testable. Please report correlations between embedding distances and lexical/geographical distances separately for the 74 seen and 32 unseen languages, or perform a leave-one-out analysis over languages. Either would allow the reader to judge whether the method generalizes to unseen languages, which is a central part of the paper's stated contribution.
- [Materials and methods, Dataset; Results, Robustness] The data-processing choices are post hoc with respect to the same embeddings used in the evaluation. The 10% Silhouette-based outlier removal and the 20-recording inclusion threshold were selected after examining the embedding distributions, and the robustness analysis in Fig 10 only covers the 70 languages with at least 1000 recordings. The main correlations in Table 1 therefore include languages with as few as 21 recordings whose stability is not assessed. Please add a sensitivity analysis varying the outlier removal threshold and the inclusion threshold, and clarify whether the small-sample languages are stable under resampling.
- [Results, Table 1] The family-level correlations in Table 1 are computed over language pairs that are not independent, and some families have very few languages (e.g., Afro-Asiatic n=8, Turkic n=11, Uralic n=7). The absence of confidence intervals or significance tests makes it difficult to interpret the family-specific claims such as the exceptionally high correlations for Uralic. Please provide confidence intervals (e.g., bootstrap) or a mixed-effects model that accounts for non-independence, and state the number of pairs underlying each correlation.
minor comments (5)
- [Throughout] Please proofread the author names and model names; for example, 'T¨or¨o' and 'voxlingua107-xls-r-300m-wav2vec' contain spacing or encoding artifacts.
- [Results, Fig. 3] The text refers to 'the actual dimensionality of the LDA space for 101 languages'; the dataset contains 106 languages, so clarify whether the LDA in Fig 3 was fitted on the 101 languages with ASJP data or on all 106.
- [Results, Table 1] Define 'Overall-related' and 'Overall-nonrelated' and report the number of language pairs in each subset, since these subsets differ in size and composition.
- [Results, Robustness and data size] State whether the average correlation across the 10 sample sets is computed on Fisher z-transformed correlations before averaging, and whether the LDA is re-fit for each resampled set.
- [Figs. 8 and 9 captions] The captions should state how many languages are marked with an asterisk and how these languages were identified, to aid reproducibility.
Circularity Check
Headline correlation is selected in-sample: LDA dimensionality is chosen by maximizing the same benchmark correlations later reported, so R²=0.31 is partly a fitted quantity.
-
fitted input called prediction
[Results — Correlations with geography and lexicon based distances (Fig. 3 paragraph and full-model paragraph)]
"Also, the correlation (quality of fit) increases with added dimensions; therefore in the remainder of this paper we use the embedding-based distances calculated using all available dimensions of the LDA space. ... The adjustedR-square value of the “full” linear model with embedding distances calculated using all LDA dimensions is 0.31, corresponding to a correlation equal to 0.56."
The number of LDA dimensions is not fixed a priori. The authors scan n from 2 to 100, observe in Fig. 3 that correlations with lexical and geographical distances increase with n, and then select 'all available dimensions' because it maximizes the fit. The headline R²=0.31 and r=0.56 are then reported on the same sample and at the selected n, making the reported alignment partly a maximum over the scanned configurations rather than an unbiased out-of-sample estimate. The robustness analyses (Figs 10–11) resample recordings but do not re-select dimensionality or evaluate held-out languages, so they do not correct this in-sample selection. The central comparison to external ASJP and Glottolog benchmarks is not itself circular, but the specific headline number is a fitted quantity.
full rationale
The central comparison is to external benchmarks (ASJP lexical distances and Glottolog geography), so the study is not circular in the strict sense. The LDA projection is supervised only by language identity, not by genealogy or geography, and the clustering/visualization results are independently interpretable against known language relationships. However, the headline quantitative claim rests on a dimension-selection step that is performed on the same data used to compute the reported correlation: the authors scan LDA dimensionalities, observe that correlation with the external benchmarks rises with dimension count, then adopt all available dimensions and report R²=0.31 from that same sample. This makes the headline number partly a fitted maximum rather than a clean prediction. The robustness analysis resamples recordings but does not re-select dimensionality or test held-out languages, so the concern is not addressed. Self-citations (refs 15–18) are contextual and not load-bearing. Overall, one partial fitted-input step is present, but the central claim retains substantial independent content, so the circularity score is 4 rather than higher.
Assumptions & free parameters
free parameters (6)
- LDA dimensionality used for distances =
105 (all available dimensions)
- Outlier removal threshold =
10% lowest Silhouette scores per language
- Robustness sample size target =
50 recordings per language
- Maximum subsample per language =
1000 recordings
- Transformations in linear model =
exponential for embedding and lexical distances, square root for geographic distance
- 5th percentile threshold for map visualization =
5%
assumptions (6)
- domain assumption The topological relationships among the numeric latent representations of languages systematically reflect underlying typological relations.
- domain assumption Lexical distance (ASJP Levenshtein distances) approximates genealogical distance.
- domain assumption The fine-tuned LID model extracts relevant discriminative information from speech for languages included and not included in its training set.
- domain assumption Glottolog coordinates and language family assignments are accurate enough for distance computations.
- ad hoc to paper Linear discriminant analysis reduces within-language variance and highlights between-language differences without distorting inter-language geometry in a way that would invalidate distance comparisons.
- standard math Statistical assumptions of linear regression and Pearson correlations apply to the transformed distance measures.
Cite this review
Pith. "Pith review of Neighbors and relatives: How do speech embeddings reflect linguistic connections across the world?." pith.science (2026). https://pith.science/paper/QEYOYWPI
@misc{pith2026250608564,
author = {Pith},
title = {Pith review of: Neighbors and relatives: How do speech embeddings reflect linguistic connections across the world?},
year = {2026},
howpublished = {\url{https://pith.science/paper/QEYOYWPI}},
note = {Machine review of arXiv:2506.08564}
}
read the original abstract
Investigating linguistic relationships on a global scale requires analyzing diverse features such as syntax, phonology and prosody, which evolve at varying rates influenced by internal diversification, language contact, and sociolinguistic factors. Recent advances in machine learning (ML) offer complementary alternatives to traditional historical and typological approaches. Instead of relying on expert labor in analyzing specific linguistic features, these new methods enable the exploration of linguistic variation through embeddings derived directly from speech, opening new avenues for large-scale, data-driven analyses. This study employs embeddings from the fine-tuned XLS-R self-supervised language identification model voxlingua107-xls-r-300m-wav2vec, to analyze relationships between 106 world languages based on speech recordings. Using linear discriminant analysis (LDA), language embeddings are clustered and compared with genealogical, lexical, and geographical distances. The results demonstrate that embedding-based distances align closely with traditional measures, effectively capturing both global and local typological patterns. Challenges in visualizing relationships, particularly with hierarchical clustering and network-based methods, highlight the dynamic nature of language change. The findings show potential for scalable analyses of language variation based on speech embeddings, providing new perspectives on relationships among languages. By addressing methodological considerations such as corpus size and latent space dimensionality, this approach opens avenues for studying low-resource languages and bridging macro- and micro-level linguistic variation. Future work aims to extend these methods to underrepresented languages and integrate sociolinguistic variation for a more comprehensive understanding of linguistic diversity.
Figures
Reference graph
Works this paper leans on
-
[1]
The myth of language universals: Language diversity and its importance for cognitive science
Evans N, Levinson SC. The myth of language universals: Language diversity and its importance for cognitive science. Behavioral and brain sciences. 2009;32(5):429–448
work page 2009
-
[2]
Campbell L. Historical Linguistics. Edinburgh University Press; 2013
work page 2013
-
[3]
Language Universals and Linguistic Typology: Syntax and Morphology
Comrie B. Language Universals and Linguistic Typology: Syntax and Morphology. University of Chicago press; 1989
work page 1989
-
[4]
Croft W. Typology and Universals. Cambridge university press; 2002
work page 2002
-
[5]
Comparative concepts and descriptive categories in crosslinguistic studies
Haspelmath M. Comparative concepts and descriptive categories in crosslinguistic studies. Language. 2010;86(3):663–687. September 16, 2025 20/27
work page 2010
-
[6]
Trees, waves and linkages: Models of language diversification
Fran¸ cois A. Trees, waves and linkages: Models of language diversification. In: The Routledge handbook of historical linguistics. Routledge; 2015. p. 161–189
work page 2015
-
[7]
Neighbor-net: an agglomerative method for the construction of phylogenetic networks
Bryant D, Moulton V. Neighbor-net: an agglomerative method for the construction of phylogenetic networks. Molecular biology and evolution. 2004;21(2):255–265
work page 2004
-
[8]
What do language representations really represent? Computational Linguistics
Bjerva J, ¨Ostling R, Veiga MH, Tiedemann J, Augenstein I. What do language representations really represent? Computational Linguistics. 2019;45(2):381–389
work page 2019
Show all 28 references
-
[9]
Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations
Oncevay A, Haddow B, Birch A. Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP); 2020. p. 2391–2406
2020
-
[10]
Continuous multilinguality with language vectors
¨Ostling R, Tiedemann J. Continuous multilinguality with language vectors. In: 15th Conference of the European Chapter of the Association for Computational Linguistics, Valencia, Spain, April 3-7, 2017. vol. 2. Association for Computational Linguistics; 2017. p. 644–649
2017
-
[11]
XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale
Babu A, Wang C, Tjandra A, Lakhotia K, Xu Q, Goyal N, et al. XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale. arXiv preprint arXiv:211109296. 2022
2022
-
[12]
Scaling speech technology to 1,000+ languages
Pratap V, Tjandra A, Shi B, Tomasello P, Babu A, Kundu S, et al. Scaling speech technology to 1,000+ languages. Journal of Machine Learning Research. 2024;25(97):1–52
2024
-
[13]
VoxLingua107: A Dataset for Spoken Language Recognition
Valk J, Alum¨ ae T. VoxLingua107: A Dataset for Spoken Language Recognition. In: Proc. IEEE SLT Workshop; 2021
2021
-
[14]
Why are some languages confused for others? Investigating data from the Great Language Game
Skirg ˚ ard H, Roberts SG, Yencken L. Why are some languages confused for others? Investigating data from the Great Language Game. PloS One. 2017;12(4):e0165934
2017
-
[15]
Comparative analysis of prosodic characteristics using wavenet embeddings
Suni A, Wlodarczak M, Vainio M, ˇSimko J. Comparative analysis of prosodic characteristics using wavenet embeddings. In: Annual Conference of the International Speech Communication Association: Interspeech 2019. ISCA; 2019. p. 2538–2542
2019
-
[16]
Comparative analysis of majority language influence on North S´ ami prosody using WaveNet-based modeling
Hiovain K, Suni A, Kakouros S, ˇSimko J. Comparative analysis of majority language influence on North S´ ami prosody using WaveNet-based modeling. Language and Speech. 2022;65(4):859–888
2022
-
[17]
Quantifying sociolinguistic change: Effects of age and gender on dialectal variation in a large corpus of spontaneous Finnish speech
T¨ or¨ o T, Suni A,ˇSimko J. Quantifying sociolinguistic change: Effects of age and gender on dialectal variation in a large corpus of spontaneous Finnish speech. In: Proceedings of the 5th International Symposium on Applied Phonetics (ISAPh 2024); 2024. p. 105–109
2024
-
[18]
Emergent Dialectal Patterns: Analysis of regional variants in a vast corpus of Finnish spontaneous speech using a large-scale self-supervised model
T¨ or¨ o T, Suni A,ˇSimko J. Emergent Dialectal Patterns: Analysis of regional variants in a vast corpus of Finnish spontaneous speech using a large-scale self-supervised model. In: Proc. Speech Prosody 2024; 2024. p. 37–41
2024
-
[19]
Self-supervised learning for Formosan speech representation and linguistic phylogeny
Hsieh SK, Tseng YH, Lian DC, Wang CW. Self-supervised learning for Formosan speech representation and linguistic phylogeny. Frontiers in Language Sciences. 2024;3:1338684. September 16, 2025 21/27
2024
-
[20]
Pretraining approaches for spoken language recognition: TalTech submission to the OLR 2021 challenge
Alum¨ ae T, Kukk K. Pretraining approaches for spoken language recognition: TalTech submission to the OLR 2021 challenge. arXiv preprint arXiv:220507083. 2022
2021
-
[21]
Common Voice: A Massively-Multilingual Speech Corpus
Ardila R, Branson M, Davis K, Kohler M, Meyer J, Henretty M, et al. Common Voice: A Massively-Multilingual Speech Corpus. In: Proceedings of the Twelfth Language Resources and Evaluation Conference; 2020. p. 4218–4222
2020
-
[22]
Glottolog 5.1; 2024
Hammarstr¨ om H, Forkel R, Haspelmath M, Bank S. Glottolog 5.1; 2024. Available from:http://glottolog.org
2024
-
[23]
SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python
Virtanen P, Gommers R, Oliphant TE, Haberland M, Reddy T, Cournapeau D, et al. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods. 2020;17:261–272. doi:10.1038/s41592-019-0686-2
2020 doi
-
[24]
The ASJP Database (version 20); 2022
Wichmann S, Holman EW, Brown CH. The ASJP Database (version 20); 2022. Available from:https://asjp.clld.org/
2022
-
[25]
Adding typology to lexicostatistics: A combined approach to language classification
Bakker D, M¨ uller A, Velupillai V, Wichmann S, Brown CH, Brown P, et al. Adding typology to lexicostatistics: A combined approach to language classification. Linguistic Typology. 2009;13(1):169–181. doi:doi:10.1515/LITY.2009.009
2009 doi
-
[26]
On the accuracy of language trees
Pompei S, Loreto V, Tria F. On the accuracy of language trees. PloS one. 2011;6(6):e20109
2011
-
[27]
Blue Marble Collection; 2004
NASA Earth Observatory. Blue Marble Collection; 2004. Available from: https://visibleearth.nasa.gov/collection/1484/blue-marble
2004
-
[28]
Microbial phylogenetic context using phylogenetic outlines
Bagci C, Bryant D, Cetinkaya B, Huson DH. Microbial phylogenetic context using phylogenetic outlines. Genome biology and evolution. 2021;13(9). Appendix 1: List of languages The following table lists all the languages in the analyzed dataset, with ISO codes, language family (s...
2021
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.