{"id":"d905288b-b545-4e52-8a1f-64e0344087e5","arxiv_id":"2509.01419","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Dharawal speech embeddings from a 107-language model rank Latin, Maori, Korean, Thai, and Welsh as the most similar high-resource languages, though confusion-based and geometry-based rankings differ.","lead":"Researchers collected 475 public recordings of Dharawal, an endangered Australian Aboriginal language, and used a pre-trained 107-language speech model to see which better-studied languages it sounds like. They report Dharawal is most often confused with Latin, Maori, Korean, Thai, and Welsh, a result they propose could guide future speech AI transfer learning.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Dharawal similarity rankings are confounded by a single-speaker corpus; no control separates language from voice.","rationale":"The paper's central claim is a statement about Dharawal as a language. For that to hold, the embeddings of the collected clips must encode language identity, not the single speaker's voice or the concatenation process. Section III shows the corpus consists of 475 short, mostly word-level recordings from the Dharawal Words website, apparently from one native speaker, and these were manually concatenated into 10-15 s samples. The VoxLingua107 ECAPA-TDNN model is trained for language ID, but its backbone is a speaker-recognition architecture; whether it is speaker-invariant is untested. Without a single-speaker control from a high-resource language, the reported cosine/FID distances to multi-speaker language centroids and the softmax misclassification rates cannot be attributed to language. The reader's conditional verdict already calls for baselines and uncertainty quantification; my concern specifies the most damaging confound. I therefore leave the verdict unchanged (conditional) but require the control experiment. Note also Eq. (1) defines MR=1 identically because Dharawal is not among the 107 classes, so the per-language rates used in Figure 3 are undefined in the text; this is a secondary correctness issue that should be fixed.","tokens_in":8095,"tokens_out":6663,"duration_ms":74899,"concrete_test":"Run the exact pipeline (VoxLingua107 ECAPA-TDNN embeddings, cosine/FID, softmax confusion) on a matched single-speaker control: 475 word-level English clips from one Common Voice speaker, concatenated to 10-15 s. If this control ranks Latin/Thai/Welsh among its top-10 similar languages, or if its top-10 set overlaps with Dharawal's as much as Dharawal's metrics overlap each other, the reported similarity is a speaker/recording artifact, not a language property.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that Dharawal is acoustically similar to Latin, Maori, Korean, Thai, and Welsh is not supported because the Dharawal corpus is a single speaker's 475 short word-level clips (Section III), concatenated into 10-15 s sequences. The VoxLingua107 ECAPA-TDNN embeddings are learned for language identification but are not guaranteed to be speaker-invariant; ECAPA-TDNN is a speaker-recognition backbone. Thus the computed cosine/FID distances between one Dharawal voice and multi-speaker language centroids, and the softmax misclassifications, may reflect vocal-tract, recording, or concatenation artifacts rather than language properties. No control uses a single-speaker subset of a known language, no per-speaker statistics are reported, and no multi-speaker Dharawal data exists. Eq. (1) additionally defines MR as 1 for every utterance because Dharawal is not in the 107-label set, so the reported per-language 'misclassification rates' are undefined as written; the ranking in Fig. 3 must be based on an unreported per-language definition. Both issues undermine the central conclusion.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper collects and cleans a private Dharawal speech dataset (475 utterances, mostly word-level clips concatenated into 10–15 s sequences) and analyzes its similarity to 107 high-resource languages using the pretrained VoxLingua107 ECAPA-TDNN encoder. Similarity is measured in three ways: softmax misclassification rates from the language identification head, cosine similarity between Dharawal and stored language centroids, and FID between embedding distributions of Dharawal and the top-10 languages selected by misclassification rate. The central claim is that Dharawal is most similar to Latin, Māori, Korean, Thai, and Welsh, with top-10 overlap across metrics, and that this ranking can guide transfer learning.","tokens_in":8259,"tokens_out":2234,"duration_ms":28894,"significance":"If the central claim is supported, the paper would make two contributions: the first collected Dharawal speech dataset (though kept private), and a model-grounded methodology for identifying acoustically similar high-resource languages to bootstrap ASR or TTS for an untranscribed endangered language. The dataset collection and cleaning effort is a genuine contribution. The paper also explicitly frames its results as falsifiable rankings and reports exact numbers (e.g., 14.52% misclassification for Latin; cosine similarities 0.8238–0.9078), which is commendable. However, the significance is contingent on the validity of using VoxLingua107 embeddings as an unbiased measure of linguistic speech similarity and on a correct, clearly defined misclassification statistic.","major_comments":[{"comment":"Equation (1) defines MR as (1/N) Σ I(ŷ_i ≠ Dharawal). Since the VoxLingua107 model has no Dharawal class, every utterance satisfies ŷ_i ≠ Dharawal, so MR is identically 1. The per-language rates plotted in Fig. 3 must therefore come from a different, unstated definition, presumably the fraction of utterances classified as each particular language. This is load-bearing: the entire confusion-based ranking depends on the unstated statistic. Please state the per-language confusion rate explicitly, report the sum of the top-10 rates and the number of utterances classified outside the top 10, and correct Eq. (1).","section":"§IV.B, Eq. (1), Fig. 3"},{"comment":"The central conclusion—that Dharawal is acoustically similar to Latin, Māori, Korean, Thai, and Welsh—is read directly from a model trained exclusively on the 107 VoxLingua107 languages. Dharawal is out-of-distribution for this encoder, so the measured distances and classifier outputs reflect the training distribution's geometry and the model's decision boundaries, not necessarily Dharawal's sound system. In addition, the Dharawal data come from a single native speaker, whereas the language centroids are multi-speaker. ECAPA-TDNN is a speaker-recognition backbone, and the model is not demonstrated to be speaker-invariant. A concrete control is needed: repeat the cosine/FID analysis using a single-speaker subset of one or more high-resource languages, and compare the observed Dharawal-to-language distances against the distribution of distances between held-out language centroids. Without","section":"§III, §IV, §VI"},{"comment":"The cosine similarities in Table I are all between 0.82 and 0.91 for the ten selected languages. Because no baseline is reported—e.g., the cosine similarity between a Dharawal centroid and all 107 centroids, or the typical centroid-to-centroid cosine similarity between unrelated VoxLingua107 languages—it is unclear whether these values indicate strong language-specific similarity or simply that all language centroids are mutually close in this 256-dimensional embedding space. The same concern applies to FID scores. Please report the full distribution of scores across all 107 languages and a null distribution from random or unrelated language pairs.","section":"§VI.B, Table I, Fig. 4"},{"comment":"The FID analysis uses the top-10 languages selected by the same model whose space is being measured, and the covariance matrices are estimated from an unstated number of utterances ('kept consistent across all languages'). The t-SNE interpretation in §VI.C (e.g., 'sparse Māori data points and large clusters... pull the centroid away') is post hoc and not quantitatively supported. The paper should state the number of utterances per language used for covariance estimation, report FID confidence intervals or sensitivity analysis, and avoid reading fine-grained phonetic causes (e.g., 'small phoneme inventories' for Latin and Māori) from embedding geometry without independent phonological evidence.","section":"§III, §VI.C"}],"minor_comments":[{"comment":"There are several typos: 'Dharahal' in §VI.A, 'EPACA-TDNN' in the Fig. 2 caption and §V.A, 'Multilinguial' in §V.A, 'lingusitic' in §II.B, and 'M ¯aori' rendering issues. Please run a spelling pass.","section":"Throughout"},{"comment":"The text attributes the ECAPA-TDNN architecture to [25], but the more standard citation for this architecture is Desplanques et al., Interspeech 2020. The current reference [25] is about speaker extraction and is not the original source.","section":"§V.A"},{"comment":"The notation in Eq. (1) is ambiguous: it mixes a per-utterance indicator and a per-language rate. Please define a separate quantity, e.g., MR_L = (1/N) Σ I(ŷ_i = L), and reserve MR for the overall recall of Dharawal (which is 0 here).","section":"§IV.B"},{"comment":"The concatenation procedure is described only as 'preserved contextual coherence.' Since most clips are individual words and are being concatenated for embedding extraction, please specify how many clips were concatenated per final utterance and whether any overlap or cross-fade was used.","section":"§III"},{"comment":"The interpretive statements in §VI.A ('Classical Latin has a relatively simple and consistent vowel system... making them acoustically similar') are unsupported by any phonological analysis of Dharawal or by a phonetic feature comparison. These causal explanations should be softened or removed unless backed by external evidence.","section":"§VI.A"}],"recommendation":"major_revision","confidential_remarks":"The paper's central claim is interesting but currently rests on an undefined misclassification statistic (Eq. (1)) and on the assumption that VoxLingua107 embeddings encode language identity rather than speaker/recording artifacts for an out-of-distribution single-speaker corpus. Both issues are fixable in principle with a corrected definition, baseline distributions, and a single-speaker control, but they require additional experiments rather than just presentation changes. For an APSIPA conference paper, the corrected version could be a useful contribution; as it stands, the evidence does not yet support the strong linguistic-similarity conclusions in the abstract."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper gives the first speech embedding analysis of Dharawal and contributes a carefully sourced, though private, 475-utterance dataset. The three-metric pipeline is transparent, and the top-10 language lists are internally consistent across metrics. But the central claim—that Dharawal is acoustically similar to Latin, Maori, Korean, Thai, and Welsh—is a statement about how VoxLingua107’s embedding space treats those inputs, not a demonstrated fact about Dharawal’s sound system. The paper overstates it as practical guidance for transfer learning.\n\nWhat is genuinely new: no prior work has done embedding-based similarity analysis for Dharawal, and the dataset collection is ethically framed. The method itself is the authors’ Interspeech 2025 approach applied to a new language, but that is not a strike against it; the new data and case study justify the paper. The writing is clear and the limitations are partially acknowledged.\n\nThe soft spots are real. First, Eq. (1) defines misclassification rate as the fraction of utterances not predicted as Dharawal, but Dharawal is not one of the model’s 107 classes, so that quantity is identically 1. The per-language rates in Fig. 3 (e.g., 14.52% for Latin) must come from a different, unstated definition of confusion rate. This is load-bearing because the ranking depends on those numbers. Second, the Dharawal corpus is a single speaker’s 475 short word-level clips, concatenated into 10–15 s sequences. ECAPA-TDNN is not speaker-invariant, so the measured distances to multi-speaker language centroids could reflect voice, recording, or concatenation artifacts rather than language properties. No control (such as a single-speaker subset of a known language) is provided. Third, cosine similarities between 0.82 and 0.91 are reported without a between-language baseline, so we do not know whether these values are unusually high. No error bars or significance tests appear. Fourth, the promised transfer-learning guidance is untested: no adaptation or ASR experiment is run. The FID sample count is also unclear.\n\nThe paper is honest and readable, and the dataset is a contribution even if private. As a model-relative similarity result, the ranking is a useful hypothesis for transfer learning; as a linguistic finding, it is under-supported. A serious referee could fix the metric definitions, add a single-speaker control, and force the claims to be softened accordingly. I would take it for peer review rather than desk-reject.","headline":"First Dharawal speech dataset and an honest three-metric pipeline, but the similarity ranking is model-relative, Eq. (1) is misspecified, and the single-speaker corpus confounds voice with language; worth refereeing after fixes.","tokens_in":8877,"tokens_out":3264,"would_cite":true,"duration_ms":34359,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A new Dharawal speech dataset lets a 107-language encoder rank which languages Dharawal sounds most like.","keywords":["Dharawal","Australian Aboriginal languages","low-resource speech","speech embeddings","language similarity","VoxLingua107","ECAPA-TDNN","transfer learning"],"falsifier":"Re-run the identical analysis with a different pretrained multilingual encoder or with Dharawal clips matched for duration, signal-to-noise ratio, and speaker count to the VoxLingua107 corpora; if the top-10 language list changes substantially, the ranking is an artifact of the embedding space rather than a stable property of Dharawal speech.","tokens_in":7882,"feed_emoji":"🗣️","tokens_out":4795,"duration_ms":50552,"temperature":0.7,"pith_summary":"This paper collects the first Dharawal speech dataset—475 short, mostly word-level recordings from a public language-preservation site, cleaned and concatenated into 10–15 second clips—and asks where Dharawal sits in the embedding space of a pretrained 107-language speech encoder. By counting how often Dharawal is misclassified as another language and by measuring centroid cosine similarity and FID between Dharawal and each high-resource language, the authors claim Dharawal is acoustically closest to Latin, Maori, Korean, Thai, and Welsh. The claim matters because endangered languages usually lack transcriptions for ASR training; if the similarity ranking is credible, models trained on those five languages are the natural starting points for transfer learning. The paper also argues that the coarse confusion-based ranking and the fine-grained geometric ranking largely overlap in the top 10, giving two complementary views of the same proximity.","feed_headline":"Dharawal speech maps closest to Latin, Thai, Maori","feed_subtitle":"A new 475-clip dataset lets a 107-language encoder rank acoustic neighbours for an endangered language.","key_machinery":"The load-bearing object is the VoxLingua107 ECAPA-TDNN encoder: a pretrained multilingual model that maps each utterance to a fixed 256-dimensional embedding and, through a softmax head, to probabilities over 107 languages. The paper uses three measurements in that space: (1) the misclassification rate, the fraction of Dharawal clips whose top predicted language is a given high-resource language; (2) cosine similarity between the mean Dharawal embedding and each language's pretrained class centroid; and (3) FID between the Dharawal embedding distribution and each candidate language's embedding distribution, using mean and covariance. The supporting data object is the cleaned Dharawal corpus:","core_discovery":"On the authors' own terms, the central discovery is that a language never seen in training—Dharawal—has a stable, measurable location in the VoxLingua107 ECAPA-TDNN embedding space, and that location places it near a small set of high-resource languages. Dharawal is misclassified as Latin 14.52% of the time, followed by Maori, Korean, Thai, and Welsh; cosine similarities range from 0.8238 (Latin) to 0.9078 (Thai), and FID scores order the same top-10 set with minor rank shifts. The authors present these as complementary evidence: misclassification captures practical confusability, while cosine and FID capture geometric closeness of centroid and distribution. They interpret the Latin and Maor","pith_inferences":["Editorial inference: the ranking is probably shaped by recording conditions and speaker demographics in the public Dharawal clips and in the VoxLingua107 corpora, so the true linguistic similarity could differ from the acoustic ranking reported here.","Editorial inference: a testable extension is to recompute the ranking with a second multilingual encoder or with matched recording conditions; high agreement would make the result a stable property of Dharawal speech rather than of one model.","Editorial inference: phonetic validation—comparing Dharawal consonant-vowel structure against the claimed Latin and Maori patterns—would tell whether the geometric proximity is linguistically meaningful or an artifact of limited word-level data."],"forward_implications":["A transfer-learning pipeline for Dharawal ASR or TTS can prioritize Latin, Maori, Korean, Thai, and Welsh as source languages instead of guessing.","Untranscribed low-resource languages can be characterized without labels, since the method relies only on audio and a pretrained encoder.","The overlap between confusion-based and geometry-based top-10 lists gives a practical way to select a shortlist of candidate source languages, then refine with FID.","The same pipeline can be pointed at other Australian Aboriginal languages as recordings become available."],"supporting_citations":[{"why":"Supplies the VoxLingua107 dataset and the pretrained multilingual encoder whose embedding space carries the whole similarity analysis.","marker":"[23]"},{"why":"Provides the ECAPA-TDNN architecture that the encoder is built on, explaining the 256-dimensional embedding mechanism.","marker":"[25]"},{"why":"Defines the FID score used as the distribution-level similarity metric between Dharawal and target languages.","marker":"[17]"},{"why":"Prior study of speech embedding similarities between Australian Aboriginal and high-resource languages that this work extends to Dharawal.","marker":"[15]"},{"why":"Cited for Classical Latin phonology to explain why Dharawal is frequently confused with Latin.","marker":"[26]"},{"why":"Cited for Maori phonology to explain the Maori–Dharawal acoustic overlap.","marker":"[27]"}],"fun_headline_variants":["Dharawal speech clusters with Latin, Thai, Maori","Dharawal's closest sound neighbors: Latin, Thai, Maori","Endangered Dharawal maps to Latin, Thai, Maori in sound","AI ranks Dharawal's acoustic kin: Latin, Thai, Maori","Dharawal's acoustic relatives: Latin, Thai, Maori"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The pretrained 107-language embedding space is a valid, unbiased ruler for linguistic speech similarity, so that Dharawal's nearest neighbours in that space tell us about the language itself rather than about recording conditions, speaker demographics, or the model's training distribution.","fun_headline_variants_meta":{"raw":{"variants":["Dharawal speech clusters with Latin, Thai, Maori","Dharawal's closest sound neighbors: Latin, Thai, Maori","Endangered Dharawal maps to Latin, Thai, Maori in sound","AI ranks Dharawal's acoustic kin: Latin, Thai, Maori","Dharawal's acoustic relatives: Latin, Thai, Maori"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000501,"raw_usage":{"total_tokens":2295,"prompt_tokens":763,"completion_tokens":1532,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":507,"completion_tokens_details":{"reasoning_tokens":1451}},"tokens_in":507,"tokens_out":1532,"duration_ms":14948,"temperature":1.0,"reasoning_tokens":1451,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T12:32:51.554086+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the identical analysis with a different pretrained multilingual encoder or with Dharawal clips matched for duration, signal-to-noise ratio, and speaker count to the VoxLingua107 corpora; if the top-10 language list changes substantially, the ranking is an artifact of the embedding space rather than a stable property of Dharawal speech.","supporting_citations":[{"cited_title":"V oxLingua107: a dataset for spoken language recognition,","cited_arxiv_id":null,"evidence_quote":"Supplies the VoxLingua107 dataset and the pretrained multilingual encoder whose embedding space carries the whole similarity analysis."},{"cited_title":"Hierarchical speaker representation for target speaker extraction,","cited_arxiv_id":null,"evidence_quote":"Provides the ECAPA-TDNN architecture that the encoder is built on, explaining the 256-dimensional embedding mechanism."},{"cited_title":"A study of speech embedding similarities between australian aboriginal and high-resource languages,","cited_arxiv_id":null,"evidence_quote":"Prior study of speech embedding similarities between Australian Aboriginal and high-resource languages that this work extends to Dharawal."},{"cited_title":"Aspects of the phonology and morphology of classical latin,","cited_arxiv_id":null,"evidence_quote":"Cited for Classical Latin phonology to explain why Dharawal is frequently confused with Latin."},{"cited_title":"Harlow, Maori: A linguistic introduction","cited_arxiv_id":null,"evidence_quote":"Cited for Maori phonology to explain the Maori–Dharawal acoustic overlap."}],"review_version":1}