Pith. sign in

REVIEW 4 major objections 5 minor 57 references

A Theoretical Framework for Acoustic Neighbor Embeddings

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that L2 distances in acoustic neighbor embeddings have a precise probabilistic meaning: text-text distance is a Gaussian approximation of phonetic similarity (the two-class Bayes error), and audio-text distance is a…

desk verdict Genuine framework with one striking dialect result; the equal-variance assumption is only half-supported and sigma is never measured, but the paper is honest and worth refereeing. read the letter →

arxiv 2412.02164 v1 pith:GPYI5STM submitted 2024-12-03 eess.AS cs.CLcs.SD

Woojay Jeon This is my paper · ORCID
classification eess.AScs.CLcs.SD
keywords acousticwordembeddingsphoneticsimilarityBayeserrorratestochasticneighborembeddingnearestsearchspeechrecognitiondialectclusteringwake-upconfusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that distances in a jointly trained audio-and-text word-embedding space carry a precise probabilistic meaning rather than being ad hoc similarity scores. It defines phonetic similarity as the overlap of the acoustic distributions of two words, equivalently the two-class Bayes error, and shows that under a Gaussian approximation with shared isotropic covariance this overlap reduces to a monotone function of Euclidean distance. The consequence is that ordinary nearest-neighbor search over text embeddings is an approximate Bayes decision rule for isolated-word classification, and audio-to-text distance is a Gaussian likelihood. Four experiments then use these identities for word classification, out-of-vocabulary recovery, dialect clustering, and wake-word confusion prediction.

What carries the argument

The load-bearing construction is the definition of phonetic similarity as the area of overlap between two class-conditional acoustic densities, Eq. (2). Acoustic neighbor embeddings repurpose stochastic neighbor embedding [16], a dimensionality-reduction method that matches neighbor-selection probabilities, replacing input distances with binary same-label distances; the resulting audio embedder pulls same-word utterances together, and a second MSE-trained text embedder converges to each cluster's mean. Combining those centroids with the Gaussian Bhattacharyya bound, and assuming constant cluster-wise isotropy, meaning every cluster has covariance $\sigma^2 I$ with the same $\sigma^2$, turns text-text distance into phonetic similarity and audio-text distance into a likelihood. Appendix B supports the isotropy approximation with a gradient argument and with cluster-wise IsoScore [55] measurements, while reporting that the equal-variance part is only partially confirmed.

What would settle it

A decisive check is to estimate per-word covariance matrices from the released audio embeddings and compare the measured overlap integral in Eq. (2), via kernel density estimates, with the prediction of Eq. (18); if word clusters with equal centroid distance but visibly different widths have different overlap, the shared-$\sigma^2$ formula will misrank them.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is Eq. (18): the L2 distance between two text embeddings $g_i$ and $g_j$ computes the phonetic similarity $s(w_i, w_j; \lambda) = \frac{1}{2}\exp(-\|g_i - g_j\|^2 / (8\sigma^2))$, while Eq. (17) makes the audio-text distance a Gaussian likelihood $p(f|w;\lambda) = (\sqrt{2\pi}\sigma)^{-d}\exp(-\|f-g\|^2 / (2\sigma^2))$. The route starts from a definition of phonetic similarity as the overlap of two class-conditional acoustic densities, identifies that overlap with the Bayes error rate, approximates it with the Bhattacharyya bound, and assumes each word's audio embeddings form a Gaussian cluster with one shared isotropic covariance $\sigma^2 I$, which collapses the bound to a Euclidean form. Text embeddings are then cluster centroids, so ordinary L2 distance between text vectors is a phonetic-similarity score, and L2 distance from an audio vector to a text vector is a likelihood. The four experiments are offered as validation of this reading.

Load-bearing premise

The argument depends on every word's audio embeddings forming a cluster with the same size and the same spread in every direction; the paper's own measurements show the spread becomes even, but not that cluster sizes become equal.

Editorial extensions

If this is right

  • Nearest-neighbor audio-to-text search is a Bayes decision rule; the paper reports 48-dimensional phone embeddings matching FST accuracy at a 500k vocabulary and coming within 1.1 points at 900k.
  • Text-text distance can be used in place of phone edit distance for out-of-vocabulary recovery, reaching 55.5 percent versus 56.0 percent for minimum edit distance with 64 dimensions.
  • Pairwise dialect dissimilarities from audio embeddings reproduce the human-listening clustering hierarchy on the TIMIT sa1 sentence with 64-dimensional embeddings, and near-identically at other dimensions.
  • Expected confusion of a candidate wake-up word can be computed from Eq. (24), letting a designer rank wake words before deployment.
  • Because the distances reduce to Euclidean form, large-scale approximate nearest-neighbor indexing applies without changing the probabilistic interpretation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If cluster widths differ, Eq. (18) is only approximate; a natural correction is a per-pair denominator such as $8\sigma_i\sigma_j$, and the released model makes this directly checkable by estimating each word's covariance.
  • The same Bayesian reading should be testable on any encoder trained from binary same/different word pairs; failure there would point at the training objective rather than at the Gaussian assumption.
  • The wake-word confusion score could be inverted into a mining tool: search a vocabulary or transcribed audio for items whose embeddings sit close to the wake-word embedding under the LM-weighted score, and use them as adversarial trigger examples; the paper gestures at this use.
  • The dialect result invites a stronger perceptual test: if text-text distance is human-like phonetic similarity, embedding distances should also predict human phone-confusion patterns, not only dialect hierarchies.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes a theoretical framework for interpreting acoustic neighbor embeddings. It defines phonetic similarity between two words as the Bayes error rate of the two-class classification problem in an acoustic feature space (Eq. (2)), approximates this via the Bhattacharyya bound, and assumes that each word's audio embeddings form a Gaussian cluster with a common isotropic covariance. Under these assumptions, the L2 distance between a text embedding and an audio embedding is claimed to evaluate a Gaussian acoustic likelihood (Eq. (17)), and the L2 distance between two text embeddings is claimed to compute phonetic similarity (Eq. (18)). The framework is applied to four experiments: isolated-word classification, OOV word recovery, dialect clustering, and wake-up word confusion prediction. The authors provide source code and pretrained models.

Significance. If the framework's assumptions were fully validated, the paper would offer a principled interpretation of acoustic neighbor embedding distances and a practical justification for using Euclidean nearest-neighbor search as a substitute for FST decoding at large vocabularies, with strong experimental results (e.g., matching FST accuracy at a 500k vocabulary, and recovering OOV words within 0.5% of phone edit distance). The dialect-clustering result, which reproduces the hierarchy from human listening experiments, is an impressive empirical validation of the embedding space's perceptual relevance. A notable strength is the release of source code and pretrained models, which supports reproducibility. However, the central theoretical result depends on an equal-variance isotropic-Gaussian assumption that the paper's own Appendix B finds only partially supported, and the Bhattacharyya bound is used as an approximation without a tightness check. These gaps currently leave the quantitative claims in Eqs. (17) and (18) not fully established.

major comments (4)
  1. [Section IV-A, Eq. (18)] The derivation of Eq. (18) requires that every word-specific audio-embedding cluster be Gaussian with the same isotropic covariance σ²I. Appendix B explicitly states that the ratio measuring uniformity of intra-cluster variance "does not improve" during training (Fig. 11), so the equal-variance half of the assumption is not empirically supported. If the clusters have unequal isotropic variances, the Bhattacharyya bound in Eq. (4) becomes (1/2) exp(-||gi - gj||²/[4(σ_i² + σ_j²)] - (d/2) ln((σ_i² + σ_j²)/(2σ_i σ_j))), which does not reduce to Eq. (18). The paper should either provide an estimate and validation of a common σ or substantially weaken the claim that L2 distances compute the phonetic similarity in Eq. (18).
  2. [Section IV-A, Experiments 1 and 4] The paper notes in Section IV-A that the isotropy approximation "only holds for the phone embedder, not the grapheme embedder," because the audio embedder is trained with phone labels in Eq. (12). Nevertheless, Experiments 1 and 4 use the grapheme embedder and are presented as validations of the framework. No argument is given for why Eqs. (17)–(18) should apply to grapheme embeddings. This is load-bearing because two of the four experiments rely on the grapheme embedder; the authors should either extend the theoretical justification or clearly separate the empirical observations from the framework's domain of validity.
  3. [Section II-B, Eq. (3)] The Bhattacharyya bound is used as a proxy for the Bayes error in Eq. (2) without any justification of its tightness. The bound is an upper bound, and for overlapping non-Gaussian distributions it can be substantially loose. The experiments in Section V provide only indirect evidence; there is no direct comparison between the bound and a Monte Carlo estimate of the Bayes error on a subset of word pairs. Without such a check, the quantitative interpretations in Eqs. (17) and (18) are not fully supported.
  4. [Section IV-B / Eq. (24)] The common variance σ is the only free parameter in the quantitative forms of Eqs. (17), (18), and (24), yet no estimate of σ is reported anywhere in the paper. While ranking applications can ignore σ when it is a constant, the log-expected-confusion values in Eq. (24) and Figure 8 depend quantitatively on σ, and changing σ alters the balance between the phonetic-similarity and LM terms. The paper should report how σ is determined or state clearly that all quantitative predictions are up to an unknown scale.
minor comments (5)
  1. [Section II-A, Eq. (1)] The definition of phonetic similarity as p/(n+m) mixes a count of pairs (p) with a count of recordings (n+m); the text "ratio between the number of identical pairs and the total number of recordings" is mathematically inconsistent. The probabilistic abstraction in Eq. (2) rescues the intended meaning, but the initial definition would benefit from rewording.
  2. [Throughout] The name "Bhatacharyya" is misspelled; the standard spelling is "Bhattacharyya."
  3. [Section IV-B, Eq. (17)] The normalization constant is written as (√(2π)σ)^(-d); using (2πσ²)^(-d/2) would match standard notation and avoid ambiguity.
  4. [Section V-E, Eq. (24)] The interpolation between log-similarity and log-LM probability through α is asserted without derivation. It would be helpful to state explicitly that this is a heuristic combination, not a consequence of the probabilistic framework.
  5. [Section V-D, Eq. (22)] The dissimilarity in Eq. (22) is defined as 1 - exp(-||...||²/(8σ²)), omitting the factor 1/2 present in the similarity Eq. (18). The factor is inconsequential for ordering and clustering, but the text should note this simplification for consistency.

Circularity Check

2 steps flagged · score 4.0 of 10

The central distance-to-similarity relation follows from the paper's own definition of phonetic similarity plus the assumed Gaussian equal-isotropic cluster model, and the Experiment 2 edit-distance baseline is constructed from the same embedding similarities; the framework is openly stated and has independent external checks, so the circularity is partial.

  1. self definitional [Section IV-B, Eqs. (16)-(18); Section II-B Eq. (5); Section II-A Eq. (2)]
    "If we can make a further approximation that all the clusters are equally isotropic, we can use the much simpler form in (5) ... In the ideal case, the mean square error criterion will cause the embedding for a given text to converge to the mean of all the audio embeddings with that text label: gphone(B) = E[f(X)|B] ... When computing the L2 distance between two text embeddings gi and gj for two words wi and wj, respectively, we are in effect computing the phonetic similarity between wi and wj: s(wi, wj; lambda) = 1/2 exp(-||gi - gj||^2/(8 sigma^2))."

    Equation (18) is obtained by substituting the text embeddings g_i, g_j into the Gaussian equal-isotropic Bhattacharyya form (5), where Eq. (16) defines those text embeddings as the audio-embedding cluster means E[f|B]. The quantity called 'phonetic similarity' was itself defined in Eq. (2) as the Bayes-error overlap in the embedding feature space X. Thus the claim that L2 distances between text embeddings compute phonetic similarity is a definitional consequence of the chosen similarity definition plus the assumed Gaussian isotropic cluster model, rather than an independently anchored empirical prediction.

  2. other [Section V-C, Eq. (21) and surrounding text]
    "To obtain the substitution costs for the minimum edit distance, we constructed a phone confusion matrix by running single phones through the phone encoder and calculating their pairwise distances. The cost of substituting phone wi with phone wj (and vice versa) is defined as: 1 - 2s(wi, wj; lambda), (21) where s(wi, wj; lambda) is defined in (18)."

    The 'traditional edit-distance' comparator in Experiment 2 is not an independent baseline: its substitution costs are computed from the very embedding similarity s(.,.; lambda) that the experiment is meant to validate (Table I lists Experiment 2's purpose as 'Validate (18) and compare with traditional edit-distance'). Both the proposed full-word embedding distance and the edit-distance comparator are therefore functions of the same fitted model, so the reported 0.5% point difference is a comparison between two aggregations of the same embedding similarities rather than a validation of Eq. (18) against an independent phone edit-distance measure.

full rationale

The paper does not fit Eq. (17) or Eq. (18) to the experimental targets, and it does not rely on a load-bearing self-citation or an imported uniqueness theorem; the core derivation is a stated chain from a definition (Eq. 2) through the Bhattacharyya bound (Eq. 3) to the Gaussian isotropic form (Eq. 5), with text embeddings identified as cluster means via the training objective (Eq. 16). That derivation is internally valid conditional on the assumptions, so it is not a hidden circularity of the 'fitted parameter renamed as prediction' kind. However, the central identity (Eq. 18) is self-definitional in the sense that what is called 'phonetic similarity' was defined as Bayes-error overlap in the embedding space, and the reduction to Euclidean distance follows by assuming the unvalidated equal-isotropic Gaussian cluster model. Appendix B explicitly acknowledges that the equal-variance component of the assumption is not supported, and no estimate of the common scale sigma is reported, so the quantitative form of Eqs. (17), (18), (22), and (24) is not independently established. The Experiment 2 edit-distance baseline is also partly self-referential because its substitution costs come from Eq. (18). These are genuine but partial circularity concerns: the paper contains external checks (word classification against FST, dialect clustering against human listening data) that give the framework some independent content, so a score of 6 or higher would be disproportionate. The honest verdict is partial circularity, primarily definitional and benchmark-contamination in nature, with an explicitly admitted unsupported equal-variance assumption that is a correctness risk more than a circularity.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The framework rests on a series of modeling assumptions: the Bayes-error definition of similarity, equal priors, Gaussian isotropic clusters with a common variance, and the use of the Bhattacharyya bound as a proxy. The equal-isotropy assumption is explicitly only partially validated. The free parameters are the cluster standard deviation sigma (used in the similarity and dissimilarity formulas but not reported) and the LM scale alpha in the wake-up word experiment, which is tuned by inspection.

free parameters (2)
  • sigma = not reported
    The assumed equal isotropic cluster standard deviation appears in Eqs. (17), (18), (21), and (22). It must be estimated from embedding data to evaluate the similarity and dissimilarity values, but the paper never reports its value or estimation procedure, and the dialect dissimilarity matrix depends on it through the exponential.
  • alpha = 0.85-0.95 (empirically chosen)
    In the wake-up word confusion experiment, the LM scale alpha in Eq. (24) is tuned by inspecting nearest neighbors, as stated in Section V-E.
assumptions (6)
  • domain assumption The phonetic similarity between two words equals the Bayes error rate of a two-class classifier on acoustic features (Eq. (2)).
    This is the foundational redefinition of similarity, taken from the hypothetical experiment in Section II-A; it is not derived from prior theory.
  • domain assumption Equal priors P(w1)=P(w2) are assumed for all pairwise similarity computations.
    Stated in Section II-A; affects the constant factor and the Bhattacharyya form.
  • domain assumption Each word's audio embedding cluster is Gaussian with mean equal to the text embedding g(B) = E[f|B].
    Assumed in Section IV-A; the MSE training in Eq. (15) is claimed to converge to the conditional mean in the ideal case, but the Gaussian shape is an approximation.
  • ad hoc to paper All clusters share the same isotropic covariance sigma^2 I (uniform cluster-wise isotropy).
    This is the key simplifying approximation in Eq. (5); Appendix B provides simulation and partial empirical support, but explicitly notes the equal-variance part is not confirmed.
  • domain assumption The Bhattacharyya bound (3) is used as a proxy for the Bayes error similarity in (2).
    The bound is an upper bound, not equality; the paper calls it an approximation without a tightness analysis.
  • ad hoc to paper Gradient dynamics in Appendix B drive embeddings toward isotropic clusters.
    The argument treats negative-sample q terms as zero and assumes the neural network can realize arbitrary embedding updates; this is a heuristic, not a proof.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Theoretical Framework for Acoustic Neighbor Embeddings." pith.science (2026). https://pith.science/paper/GPYI5STM

@misc{pith2026241202164,
  author       = {Pith},
  title        = {Pith review of: A Theoretical Framework for Acoustic Neighbor Embeddings},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GPYI5STM}},
  note         = {Machine review of arXiv:2412.02164}
}
read the original abstract

This paper provides a theoretical framework for interpreting acoustic neighbor embeddings, which are representations of the phonetic content of variable-width audio or text in a fixed-dimensional embedding space. A probabilistic interpretation of the distances between embeddings is proposed, based on a general quantitative definition of phonetic similarity between words. This provides us a framework for understanding and applying the embeddings in a principled manner. Theoretical and empirical evidence to support an approximation of uniform cluster-wise isotropy are shown, which allows us to reduce the distances to simple Euclidean distances. Four experiments that validate the framework and demonstrate how it can be applied to diverse problems are described. Nearest-neighbor search between audio and text embeddings can give isolated word classification accuracy that is identical to that of finite state transducers (FSTs) for vocabularies as large as 500k. Embedding distances give accuracy with 0.5% point difference compared to phone edit distances in out-of-vocabulary word recovery, as well as producing clustering hierarchies identical to those derived from human listening experiments in English dialect clustering. The theoretical framework also allows us to use the embeddings to predict the expected confusion of device wake-up words. All source code and pretrained models are provided.

Figures

Figures reproduced from arXiv: 2412.02164 by the authors.

Figure 1
Figure 1. A definition of phonetic similarity between two words [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Example 2-dimensional embedding space where an [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Fig. 2b where we now approximate each cluster to [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Normalized distribution of word pronunciation lengths [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Hypothesized distribution shift for two different dialects [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: ADDTREE/P [47] clustering result (not accurately [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Interpretation of phonetic similarity in a detection [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Log expected confusion per Eq. (24) for different [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Simulation of embeddings for two clusters being trained [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 11
Figure 11. Figure 11: Two ratios for measuring the degree of uniformity [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 48 canonical work pages

  1. [1]

    Word embeddings for speech recognition,

    S. Bengio and G. Heigold, “Word embeddings for speech recognition,” in Proc. Interspeech 2014 , 2014, pp. 1053–1057

    work page 2014

  2. [2]

    Fixed-dimensional acoustic embeddings of variable-length segments in low-resource set- tings,

    K. Levin, K. Henry, A. Jansen, and K. Livescu, “Fixed-dimensional acoustic embeddings of variable-length segments in low-resource set- tings,” in 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, 2013, pp. 410–415

    work page 2013

  3. [3]

    Query-by-Example Search with Discriminative Neural Acoustic Word Embeddings,

    S. Settle, K. Levin, H. Kamper, and K. Livescu, “Query-by-Example Search with Discriminative Neural Acoustic Word Embeddings,” in Proc. Interspeech 2017 , 2017, pp. 2874–2878

    work page 2017

  4. [4]

    Query-by-example keyword spotting using long short-term memory networks,

    G. Chen, C. Parada, and T. N. Sainath, “Query-by-example keyword spotting using long short-term memory networks,” in 2015 IEEE In- ternational Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp. 5236–5240

    work page 2015

  5. [5]

    Learned in speech recognition: Contextual acoustic word embeddings,

    S. Palaskar, V . Raunak, and F. Metze, “Learned in speech recognition: Contextual acoustic word embeddings,” in ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp. 6530–6534

    work page 2019

  6. [6]

    Deep convolutional acoustic word embeddings using word-pair side information,

    H. Kamper, W. Wang, and K. Livescu, “Deep convolutional acoustic word embeddings using word-pair side information,” in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2016, pp. 4950–4954

    work page 2016

  7. [7]

    Discriminative acoustic word embeddings: Recurrent neural network-based approaches,

    S. Settle and K. Livescu, “Discriminative acoustic word embeddings: Recurrent neural network-based approaches,” in 2016 IEEE Spoken Language Technology Workshop (SLT), 2016, pp. 503–510

    work page 2016

  8. [8]

    Audio word2vec: Sequence-to-sequence autoencoding for unsupervised learn- ing of audio segmentation and representation,

    Y .-C. Chen, S.-F. Huang, H.-y. Lee, Y .-H. Wang, and C.-H. Shen, “Audio word2vec: Sequence-to-sequence autoencoding for unsupervised learn- ing of audio segmentation and representation,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 27, no. 9, pp. 1481– 1493, 2019

    work page 2019

Show all 57 references
  1. [9]

    Improved acoustic word embeddings for zero-resource languages using multilingual transfer,

    H. Kamper, Y . Matusevych, and S. Goldwater, “Improved acoustic word embeddings for zero-resource languages using multilingual transfer,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 29, pp. 1107–1118, 2021

    2021

  2. [10]

    Multi-view recurrent neural acoustic word embeddings,

    W. He, W. Wang, and K. Livescu, “Multi-view recurrent neural acoustic word embeddings,” in International Conference on Learning Represen- tations (ICLR), 2017

    2017

  3. [11]

    The measurement of phonetic similarity,

    P. Ladefoged, “The measurement of phonetic similarity,” in International Conference on Computational Linguistics COLING 1969: Preprint No. 57, Sep. 1969. [Online]. Available: https://aclanthology.org/C69-5701

    1969

  4. [12]

    Predicting the judged “similarity of sound

    P. C. Vitz and B. S. Winkler, “Predicting the judged “similarity of sound” of english words,” Journal of Verbal Learning and Verbal Behavior, vol. 12, no. 4, pp. 373–388, 1973. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0022537173800167

    1973

  5. [13]

    Phonetic comparison algorithms,

    B. Kessler, “Phonetic comparison algorithms,” Transactions of the Philological Society , vol. 103, no. 2, pp. 243–260, 2005. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1467-968X. 2005.00153.x

    2005 arXiv

  6. [14]

    What makes words sound similar?

    U. Hahn and T. M. Bailey, “What makes words sound similar?” Cognition, vol. 97, no. 3, pp. 227–267, 2005. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0010027704001842

    2005

  7. [15]

    Acoustic neighbor embeddings,

    W. Jeon, “Acoustic neighbor embeddings,” 2020 (preprint). [Online]. Available: https://arxiv.org/abs/2007.10329

    2020 arXiv

  8. [16]

    Stochastic neighbor embedding,

    G. E. Hinton and S. T. Roweis, “Stochastic neighbor embedding,” in Advances in Neural Information Processing Systems 15 , S. Becker, S. Thrun, and K. Obermayer, Eds. MIT Press, 2003, pp. 857–864

    2003

  9. [17]

    User-Initiated Repetition-Based Recovery in Multi-Utterance Dialogue Systems,

    H. L. Nguyen, V . Renkens, J. Pelemans, S. P. Potharaju, A. K. Nala- malapu, and M. Akbacak, “User-Initiated Repetition-Based Recovery in Multi-Utterance Dialogue Systems,” in Proc. Interspeech 2021 , 2021, pp. 226–230

    2021

  10. [18]

    Improvements to embedding-matching acoustic-to-word ASR using multiple-hypothesis pronunciation-based embeddings,

    H. Yen and W. Jeon, “Improvements to embedding-matching acoustic-to-word ASR using multiple-hypothesis pronunciation-based embeddings,” in 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2023. [Online]. Available: https://arxiv.org/abs/...

    2023 arXiv

  11. [19]

    Retrieval augmented correction of named entity speech recognition errors,

    E. Pusateri, A. Walia, A. Kashi, B. Bandyopadhyay, N. Hyder, S. Mahinder, R. Anantha, D. Liu, and S. Gondala, “Retrieval augmented correction of named entity speech recognition errors,” 2024. [Online]. Available: https://arxiv.org/abs/2409.06062

    2024 arXiv

  12. [20]

    Libriheavy: a 50,000 hours ASR corpus with punctuation casing and context,

    W. Kang, X. Yang, Z. Yao, F. Kuang, Y . Yang, L. Guo, L. Lin, and D. Povey, “Libriheavy: a 50,000 hours ASR corpus with punctuation casing and context,” 2023

    2023

  13. [21]

    R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification (2nd Edition), 2nd ed. Wiley-Interscience, November 2000

    2000

  14. [22]

    A derivation of minimum classification error from the theoretical classification risk using parzen estimation,

    E. McDermott and S. Katagiri, “A derivation of minimum classification error from the theoretical classification risk using parzen estimation,” Computer Speech & Language , vol. 18, no. 2, pp. 107–122, 2004. [Online]. Available: https://www.sciencedirect.com/science/article/pii...

    2004

  15. [23]

    An analysis of perceptual confusions among some english consonants,

    G. A. Miller and P. A. Nicely, “An analysis of perceptual confusions among some english consonants,” Journal of the Acoustical Society of America, vol. 27, pp. 338–352, 1955

    1955

  16. [24]

    Low dimensional measurement of vowels using machine perception,

    J. Burridge and B. Vaux, “Low dimensional measurement of vowels using machine perception,” The Journal of the Acoustical Society of America, vol. 153, no. 1, pp. 304–315, 01 2023. [Online]. Available: https://doi.org/10.1121/10.0016845

    2023 doi

  17. [25]

    Phonetic confusion matrix based spoken document retrieval,

    S. Srinivasan and D. Petkovic, “Phonetic confusion matrix based spoken document retrieval,” in Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval , ser. SIGIR ’00. New York, NY , USA: Association for Computing...

  18. [26]

    Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences,

    S. Davis and P. Mermelstein, “Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences,” MANUSCRIPT PREPRINT 13 IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 28, no. 4, pp. 357–366, 1980

    1980

  19. [27]

    Query-by-example spoken term detection using phonetic posteriorgram templates,

    T. J. Hazen, W. Shen, and C. White, “Query-by-example spoken term detection using phonetic posteriorgram templates,” in 2009 IEEE Workshop on Automatic Speech Recognition Understanding , 2009, pp. 421–426

    2009

  20. [28]

    Approximate nearest neighbors: towards removing the curse of dimensionality,

    P. Indyk and R. Motwani, “Approximate nearest neighbors: towards removing the curse of dimensionality,” in Proceedings of the Thirtieth Annual ACM Symposium on Theory of Computing , ser. STOC ’98. New York, NY , USA: Association for Computing Machinery, 1998, p. 604–613. [Onli...

    1998

  21. [29]

    Fast k nearest neighbor search using gpu,

    V . Garcia, E. Debreuve, and M. Barlaud, “Fast k nearest neighbor search using gpu,” in 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops , 2008, pp. 1–6

    2008

  22. [30]

    Soar: improved indexing for approximate nearest neighbor search,

    P. Sun, D. Simcha, D. Dopson, R. Guo, and S. Kumar, “Soar: improved indexing for approximate nearest neighbor search,” in Proceedings of the 37th International Conference on Neural Information Processing Systems, ser. NIPS ’23. Red Hook, NY , USA: Curran Associates Inc., 2024

    2024

  23. [31]

    Isotropy, clusters, and classifiers,

    T. Mickus, S.-A. Gr ¨onroos, and J. Attieh, “Isotropy, clusters, and classifiers,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , L.-W. Ku, A. Martins, and V . Srikumar, Eds. Bangkok, Thailand: Association ...

    2024

  24. [32]

    The kaldi speech recognition toolkit,

    D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y . Qian, P. Schwarz, J. Silovsky, G. Stem- mer, and K. Vesely, “The kaldi speech recognition toolkit,” inIEEE 2011 Workshop on Automatic Speech Recognition and Understanding . IEEE ...

    2011

  25. [33]

    Paszke, S

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. K ¨opf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, PyTorch: an imperative style, high- ...

    2019

  26. [34]

    The CMU Pronouncing Dictionary,

    “The CMU Pronouncing Dictionary,” www.speech.cs.cmu.edu/cgi- bin/cmudict

  27. [35]

    Librispeech: An ASR corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: An ASR corpus based on public domain audio books,” in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp. 5206–5210

    2015

  28. [36]

    Librispeech language models, vocabulary and G2P models,

    “Librispeech language models, vocabulary and G2P models,” https:// www.openslr.org/11/, accessed: 2024-07-11

    2024

  29. [37]

    Conformer: Convolution- augmented Transformer for Speech Recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu, and R. Pang, “Conformer: Convolution- augmented Transformer for Speech Recognition,” in Proc. Interspeech 2020, 2020, pp. 5036–5040

    2020

  30. [38]

    Bourlard and N

    H. Bourlard and N. Morgan, Connectionist Speech Recognition: A Hybrid Approach. Kluwer Academic Publishers, 01 1994

    1994

  31. [39]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Conference on Learning Representations , 2015

    2015

  32. [40]

    Contextual Recovery of Out-of-Lattice Named Entities in Automatic Speech Recog- nition,

    J. Serrino, L. Velikovich, P. Aleksic, and C. Allauzen, “Contextual Recovery of Out-of-Lattice Named Entities in Automatic Speech Recog- nition,” in Proc. Interspeech 2019 , 2019, pp. 3830–3834

    2019

  33. [41]

    Phonetic Embedding for ASR Robustness in Entity Resolution,

    X. Zhou, R. Bao, and W. M. Campbell, “Phonetic Embedding for ASR Robustness in Entity Resolution,” in Proc. Interspeech 2022 , 2022, pp. 3268–3272

    2022

  34. [42]

    Computational dialectology in Irish Gaelic,

    B. Kessler, “Computational dialectology in Irish Gaelic,” in Seventh Conference of the European Chapter of the Association for Computational Linguistics , S. P. Abney and E. W. Hinrichs, Eds. Dublin, Ireland: Association for Computational Linguistics, Mar

  35. [43]

    Evaluation of acoustic word embeddings,

    S. Ghannay, Y . Est `eve, N. Camelin, and P. Deleglise, “Evaluation of acoustic word embeddings,” in Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP . Berlin, Germany: Association for Computational Linguistics, Aug. 2016, pp. 62–66. [Online]....

    2016

  36. [44]

    Do Acoustic Word Embeddings Capture Phonological Similarity? An Empirical Study,

    B. M. Abdullah, M. Mosbach, I. Zaitova, B. M ¨obius, and D. Klakow, “Do Acoustic Word Embeddings Capture Phonological Similarity? An Empirical Study,” in Proc. Interspeech 2021 , 2021, pp. 4194–4198

    2021

  37. [45]

    Keyword search using modified minimum edit distance measure,

    K. Audhkhasi and A. Verma, “Keyword search using modified minimum edit distance measure,” in 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP ’07 , vol. 4, 2007

    2007

  38. [46]

    Coarticulation in recent speech production models,

    R. Kent and F. Minifie, “Coarticulation in recent speech production models,” Journal of Phonetics , vol. 5, no. 2, pp. 115–133, 1977. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S0095447019311234

    1977

  39. [47]

    ADDTREE/P: A pascal program for fitting additive trees based on Sattath and Tversky’s ADDTREE algorithm

    J. G. Orter, “ADDTREE/P: A pascal program for fitting additive trees based on Sattath and Tversky’s ADDTREE algorithm.” Behavior Re- search Methods and Instrumentation , vol. 14, pp. 353–354, 1982

    1982

  40. [48]

    Some acoustic cues for the perceptual categorization of american english regional dialects,

    C. G. Clopper and D. B. Pisoni, “Some acoustic cues for the perceptual categorization of american english regional dialects,” Journal of Phonetics, vol. 32, no. 1, pp. 111–140, 2004. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0095447003000093

    2004

  41. [49]

    TIMIT acoustic-phonetic continuous speech corpus,

    J. Garofolo, L. Lamel, W. Fisher, J. Fiscus, D. Pallett, N. Dahlgren, and V . Zue, “TIMIT acoustic-phonetic continuous speech corpus,”Linguistic Data Consortium, 11 1992

    1992

  42. [50]

    ADDTREE/P version 1.5,

    “ADDTREE/P version 1.5,” https://www.netlib.org/mds/addtree.pas, ac- cessed: 2024-07-23

    2024

  43. [51]

    Hey siri: An on-device DNN-powered voice trigger for apple’s per- sonal assistant,

    “Hey siri: An on-device DNN-powered voice trigger for apple’s per- sonal assistant,” https://machinelearning.apple.com/research/hey-siri, ac- cessed: 2024-07-11

    2024

  44. [52]

    Exploring accidental triggers of smart speakers,

    L. Sch ¨onherr, M. Golla, T. Eisenhofer, J. Wiele, D. Kolossa, and T. Holz, “Exploring accidental triggers of smart speakers,” Computer Speech and Language , vol. 73, p. 101328, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0885230821001212

    2022

  45. [53]

    Generating TTS Based Adversarial Samples for Training Wake-Up Word Detection Systems Against Confusing Words,

    H. Wang, Y . Jia, Z. Zhao, X. Wang, J. Wang, and M. Li, “Generating TTS Based Adversarial Samples for Training Wake-Up Word Detection Systems Against Confusing Words,” inProc. The Speaker and Language Recognition Workshop (Odyssey 2022) , 2022, pp. 402–406

    2022

  46. [54]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” 2023. [Online]. Available: https://arxiv.org/abs/1706.03762

    2023 arXiv

  47. [55]

    IsoScore: Measuring the uniformity of embedding space utilization,

    W. Rudman, N. Gillman, T. Rayne, and C. Eickhoff, “IsoScore: Measuring the uniformity of embedding space utilization,” in Findings of the Association for Computational Linguistics: ACL 2022 , S. Muresan, P. Nakov, and A. Villavicencio, Eds. Dublin, Ireland: Association for Com...

    2022

  48. [56]

    Analyzing the representational geometry of acoustic word embeddings,

    B. Abdullah and D. Klakow, “Analyzing the representational geometry of acoustic word embeddings,” in Proceedings of the Fifth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP , J. Bastings, Y . Belinkov, Y . Elazar, D. Hupkes, N. Saphra, and S. Wiegre...

    2022

  49. [1995]

    Available: https://aclanthology.org/E95-1009

    [Online]. Available: https://aclanthology.org/E95-1009

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.