Pith. sign in

REVIEW 2 major objections 6 minor 65 references

Phone Segmentation and Recognition through Phonological Activation Mapping

T0 review · 2 major / 6 minor · reviewed 2026-07-13 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Self-supervised speech models already encode phonetic structure; two simple heads recover phone labels and boundaries from under a minute of transcriptions.

desk verdict Clean engineering result: SPAM plus two gradient-free heads turns latent S3M phonetics into joint segmentation+recognition from under a minute of labels, with solid OOD evidence. read the letter →

arxiv 2607.09020 v1 pith:Q6WBZERT submitted 2026-07-10 eess.AS cs.AIcs.CLcs.LGcs.SD

Shikhar Bharadwaj , Kwanghee Choi , Stephen McIntosh , Chin-Jou Li , Eunjung Yeo , Daisuke Saito , Nobuaki Minematsu , Shinji Watanabe
show 3 more authors
Jian Zhu David Harwath David R. Mortensen
This is my paper · ORCID
classification eess.AScs.AIcs.CLcs.LGcs.SD
keywords phonesegmentationrecognitionself-supervisedlearningphonologicalfeaturesSPAMsampleefficiencyunseenphones
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Phone segmentation and recognition are usually trained as separate, data-hungry models. This paper argues that the representations of modern self-supervised speech models already contain the needed phonetic structure, so the tasks can be solved by reading that structure out rather than by heavy retraining. The authors define S3M-based Phonological Activation Mapping (SPAM): each frame is projected onto linear directions that correspond to phonological features such as voicing or nasality, producing a time-aligned map of feature activations. On top of that map they place two lightweight, gradient-descent-free heads—one that matches activations to known feature vectors for phone identity, and one that finds boundaries by detecting peaks of change across several signals. Because the heads need only enough labeled data to estimate the feature directions, the method works from less than a minute of phonetic transcriptions, can name phones never seen in training, and remains competitive on accented, atypical, and multilingual speech where fully supervised baselines overfit.

What carries the argument

S3M-based Phonological Activation Mapping (SPAM): each frame representation is projected onto difference-of-means phonological vectors (one per binary feature channel) and affinely normalized, yielding a time-aligned matrix of feature activations that both the recognition and segmentation heads read directly.

What would settle it

Estimate the phonological vectors only on TIMIT, then measure recognition error and boundary R-value on a held-out language or atypical-speech set that contains many phones and feature combinations never seen in TIMIT; if performance collapses relative to a model that sees even a few minutes of that language’s data, the claim that the directions transfer fails.

Watch

Extended reading notes

Core claim

Phonetic structure is already latent in the representation geometry of self-supervised speech models; once that structure is isolated as linear phonological directions and turned into a frame-wise activation map (SPAM), two simple non-gradient heads recover both phone boundaries and phone labels with under a minute of labeled speech and generalize to unseen phones and out-of-domain data.

Load-bearing premise

The linear phonological directions estimated from center-pooled frames of a single English corpus stay aligned enough with true phonetic features across languages, accents, and atypical speech that nearest-neighbor matching and peak detection on the resulting activations still work.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper argues that phonetic structure is already latent in self-supervised speech model (S3M) representations and can be steered for joint phone segmentation and recognition without gradient-based training of the heads. It constructs S3M-based Phonological Activation Mapping (SPAM) by estimating linear phonological vectors as differences of means over center-pooled frames (Eq. 1), projecting each frame onto those vectors with affine normalization (Eqs. 2–3), and stacking the resulting activations. Two gradient-descent-free heads then operate on SPAM: a recognition head that nearest-neighbor matches the center-frame activations to PanPhon canonical feature vectors (Eq. 4), and a segmentation head that ensembles multi-scale cosine differences, backward-contrast signals, and a mel-spectrogram difference via a product of non-negative signals followed by peak detection (Eqs. 5–11). With phonological vectors estimated from TIMIT (or fractions thereof), the method is evaluated for R-value segmentation and PFER recognition across English, accented, atypical, and multilingual corpora, showing competitive or superior out-of-domain performance relative to TIMIT-trained CTC/FCE/BCE baselines and usable results with under a minute of labeled data.

Significance. If the empirical claims hold, the work offers a practical, sample-efficient route to time-aligned phonetic transcription that is especially relevant for low-resource languages, atypical speech, and fieldwork settings where large transcribed corpora and heavy fine-tuning are unavailable. Strengths include the explicit design of parameter-free (or closed-form) heads, the sample-efficiency ablation down to ~18 utterances (Fig. 3), the unseen-phone analysis under oracle segmentation, the broad multi-domain evaluation (Tables I–II), and the public release of modeling and evaluation code. The approach also supplies an interpretable intermediate representation (SPAM) that unifies segmentation and recognition under a single phonological geometry, which is a useful conceptual contribution even if absolute topline numbers remain higher for large supervised systems.

major comments (2)
  1. [Table II / §IV-B] Table II: SPAM’s in-domain PFER on PR-tmt (22.9) is substantially worse than the CTC baseline (7.2) and the SotA toplines, while remaining competitive only on the multilingual average. The abstract and §IV-B claim “strong” recognition performance; the manuscript should either qualify this claim more carefully (e.g., “strong OOD generalization relative to TIMIT-trained baselines”) or provide additional analysis of when the nearest-neighbor head fails (phonotactics, inventory size, silence/closure handling).
  2. [§V-B] §V-B (oracle segmentation) correctly identifies the segmenter as the primary bottleneck (PFER drops to 11.1 on TIMIT and 8.4 on VoxAngeles with GT boundaries). Given that the central claim is joint segmentation-and-recognition, the paper should either strengthen the segmentation head (e.g., by reporting precision/recall or boundary-error distributions) or more explicitly frame recognition results under predicted vs. oracle boundaries so readers can assess the joint system’s practical utility.
minor comments (6)
  1. [§III-A] §III-A: Center pooling is asserted to be preferable for phonological arithmetic, citing prior work; a one-sentence quantitative comparison (center vs. average pooling) on the same TIMIT vectors would make the design choice self-contained.
  2. [Eq. (2)] Eq. (2) and footnote 1: γ is fixed to 4 for all experiments. A brief sensitivity check (or statement that results are insensitive within a range) would reassure readers that the constant is not a hidden free parameter.
  3. [Table I / Fig. 3] Table I / §IV-A: R-value is reported without error bars or multiple random seeds for the data-subsampling experiments in Fig. 3. Even a single standard deviation over a few seeds would strengthen the sample-efficiency claim.
  4. [§III-D] §III-D: The product ensemble (Eq. 11) and the seven-signal list are clear, but the theoretical minima φ_k are not tabulated; listing them (or stating they are the analytic lower bounds of each cosine-based term) would aid reproducibility.
  5. [§III-B] Fig. 1 caption and §III-B: The silence, closure, and release channels are important engineering details; a short note on how they affect the PanPhon nearest-neighbor match (Eq. 4) would clarify the recognition pipeline.
  6. [Throughout] Minor typography: “V oxangeles” / “V oxAngeles” spacing is inconsistent; “SotA” is used without expansion on first occurrence in §IV.

Circularity Check

2 steps flagged · score 2.0 of 10

Mild self-citation of linear phonological directions plus definitional reuse of PanPhon for both vector estimation and recognition readout; empirical OOD metrics remain independent of construction.

  1. self citation load bearing [Section I; Section III-A]
    "Recent work [27], [28] shows that these phonological features can be accurately modeled as linear directions, i.e., phonological vectors, in the representation space of some S3Ms. For instance, adding the voicing vector to a representation of [s] moves it toward [z] (Figure 1, left). Estimating these vectors is known to be highly sample-efficient [27]"

    The claim that S3M representations contain accurate, sample-efficient linear phonological directions recoverable by difference-of-means is the foundation of SPAM and is supported only by citations whose author lists substantially overlap with the present paper. Downstream task numbers still stand independently, but the geometric premise itself is not re-derived here.

  2. self definitional [Section III-A Eq. (1); Section III-C Eq. (4)]
    "We use PanPhon [34] to assign each phone its phonological features. ... The phonological vector for channel i is a difference of means: vi = µi − µ∁i ... Because the SPAM channels are PanPhon features, recognition requires no trained classifier: a segment is labeled with the phone whose phonological feature vector best matches its SPAM activations. ... ˆv = arg max_v σ(mc(s))⊤ pv. This amounts to a nearest-neighbor lookup in PanPhon"

    Phonological channels and their vectors are defined from PanPhon feature assignments on the training phones; the recognition head then classifies by matching the resulting activations against the identical PanPhon canonical vectors. For phones whose features appear in training, successful recovery is a direct readout of the same feature system used to construct the directions (quality still depends on S3M geometry).

full rationale

The paper's load-bearing empirical results (R-value on OOD/atypical/multilingual sets in Table I, PFER on PRiSM in Table II, sample-efficiency curves in Fig. 3, and seen/unseen phone PFER under oracle segmentation) are obtained by applying closed-form heads to held-out ground-truth annotations that are independent of the TIMIT-fitted means. The recognition head is a parameter-free nearest-neighbor match (Eq. 4) and the segmenter is an ensemble of cosine differences plus a closed-form least-squares backward contrast (Eqs. 5–11); neither quantity is forced by the fit. Circularity is limited to (a) the premise that difference-of-means recovers accurate linear phonological directions, which rests on self-citations [27],[28], and (b) the fact that both vector construction and phone readout use the same PanPhon feature inventory. These do not make the reported numbers tautological. Score 2 is therefore appropriate; no fitted-input-as-prediction, uniqueness-import, or ansatz-smuggling reductions exist.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central claim rests on the existence of recoverable linear phonological directions in S3M space (imported from prior work), the adequacy of PanPhon’s ternary feature inventory, center-pooling, a handful of hand-chosen constants and signal choices, and the assumption that TIMIT-derived vectors transfer. No new physical entities are postulated; free parameters are few and mostly scaling/ensemble choices.

free parameters (4)
  • gamma (SPAM scaling constant)
    Fixed to 4 for all experiments (§III-B); controls the dynamic range of activation values and is not derived from first principles.
  • ensemble of seven segmentation signals and their theoretical minima phi_k
    Choice of which multi-scale differences, backward contrasts, and mel signal to multiply (§III-D, Table III) is empirical; product form and offsets are hand-selected.
  • S3M layer and model choice (final layer of WavLM-large)
    Selected after ablation (Fig. 4) because it yields the best R-value; other layers/models degrade performance, so the claim depends on this selection.
  • 20 ms boundary hit threshold and strict R-value mode
    Evaluation hyper-parameter taken from prior work but still a free choice that affects reported scores (§IV-A).
assumptions (4)
  • domain assumption Phonological features appear as approximately linear directions recoverable by difference-of-means in S3M representation space.
    Imported from [27],[28] and used as the foundation of Eq. 1 and SPAM construction (§III-A/B).
  • domain assumption PanPhon’s 21 ternary articulatory features (plus silence/closure/release channels) are a sufficient and language-universal basis for phone identity.
    Recognition head is nearest-neighbor lookup in this inventory (Eq. 4); tones are explicitly excluded.
  • domain assumption Center-pooling of phone spans yields more reliable phonological vectors than average-pooling.
    Stated in §III-A with citations to [33],[43],[28]; used for all vector estimation.
  • ad hoc to paper Cosine distance peaks (and product ensemble thereof) on SPAM activations correspond to phone boundaries.
    Core of the segmentation head (§III-D); classical acoustic precedent exists but the multi-scale + backward-contrast ensemble is paper-specific.
invented entities (2)
  • S3M-based Phonological Activation Mapping (SPAM) independent evidence
    purpose: Time-aligned matrix of phonological feature activations obtained by projecting S3M frames onto estimated phonological vectors; serves as the sole input to both heads.
    Formalized and applied to joint segmentation+recognition here; earlier papers used related projections mainly for visualization. Independent evidence is partial (prior vector arithmetic results) but the joint-task utility is new to this paper.
  • Backward-contrast segmentation signals beta_ell
    purpose: Exploit position-dependent phonological subspaces to detect boundaries via predicted previous-phone activations.
    Derived from a least-squares regressor on adjacent pairs (Eq. 8–9); new signal family introduced for the ensemble.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Phone Segmentation and Recognition through Phonological Activation Mapping." pith.science (2026). https://pith.science/paper/Q6WBZERT

@misc{pith2026260709020,
  author       = {Pith},
  title        = {Pith review of: Phone Segmentation and Recognition through Phonological Activation Mapping},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q6WBZERT}},
  note         = {Machine review of arXiv:2607.09020}
}
read the original abstract

Phone segmentation and recognition are inherently related tasks, yet modern approaches typically model them separately. We argue that phonetic structure is already latent in the representations of self-supervised speech models (S3Ms), and one only needs to steer them to solve both tasks. We leverage S3M-based Phonological Activation Mapping (SPAM), which maps each S3M representation frame to a vector of phonological feature activations, such as voicing and nasality. On top of SPAM, we introduce two simple but effective lightweight, gradient-descent-free prediction heads: a recognition head and a segmentation head. Our method requires less than a minute of phonetic transcriptions, and generalizes to unseen phones during training. Across a diverse range of datasets, our approach attains strong segmentation and recognition performance.

Figures

Figures reproduced from arXiv: 2607.09020 by the authors.

Figure 1
Figure 1. Overview of S3M-based Phonological Activation Mapping (SPAM). Left: Phonological vectors [27], [28] can be found in S3M representation space by taking the difference of means. For example, vvoi+ is the difference between the mean representations of voiced phones and other phones. Right: S3M frame representations rt and rt ′ from different timesteps t and t ′ are projected onto phonological vectors [8], [28] (e.g., v… view at source ↗
Figure 2
Figure 2. Overview of our method (Section III). (FCE), Wav2Vec2-FS uses forward-sum alignment without frame-level labels (similar to CTC), and TIPAA-SSL combines CTC with FCE. In contrast, we use no training loss, relying on the phonetic structure SPAM already exposes. III. METHOD A. S3M-based Phonological Vectors S3M representation. A self-supervised speech model (S3M) encodes a waveform into a sequence D dimensional frame-l… view at source ↗
Figure 3
Figure 3. Ablation on the training dataset size. We randomly subsample TIMIT [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Segmentation performance on different S3Ms and layers for SPAM. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 3 linked inside Pith

  1. [1]

    TIMIT acoustic-phonetic continuous speech corpus,

    J. S. Garofolo, L. F. Lamel, W. M. Fisher, D. S. Pallett, N. L. Dahlgren, V . Zue, and J. G. Fiscus, “TIMIT acoustic-phonetic continuous speech corpus,” 1993

    1993

  2. [2]

    Phonetic segmentation of the UCLA phonetics lab archive,

    E. Chodroff, B. Pa ˇzon, A. Baker, and S. Moran, “Phonetic segmentation of the UCLA phonetics lab archive,” inLREC-COLING, 2024

    2024

  3. [3]

    L. D. Shriberg, R. D. Kent, T. McAllister, J. L. Preston, and M. L. Speights,Clinical phonetics. Plural Publishing, 2025

    2025

  4. [4]

    EduSpeak®: A speech recognition and pronunciation scoring toolkit for computer-aided language learning applications,

    H. Franco, H. Bratt, R. Rossier, V . R. Gadde, E. Shriberg, V . Abrash, and K. Precoda, “EduSpeak®: A speech recognition and pronunciation scoring toolkit for computer-aided language learning applications,” Language Testing, vol. 27, pp. 401 – 418, 2010. [Online]. Available: https://api.semanticscholar.org/CorpusID:143273296

    2010

  5. [5]

    PRiSM: Benchmarking phone realization in speech models,

    S. Bharadwaj, C.-J. Li, Y . Kim, K. Choi, E. Yeo, R. S.-E. Shim, H. Zhou, B. Boldt, K. R. Jacome, K. Chang, D. Agrawal, K. Xu, C.-H. H. Yang, J. Zhu, S. Watanabe, and D. R. Mortensen1, “PRiSM: Benchmarking phone realization in speech models,” inACL, 2026

    2026

  6. [6]

    Tusom2021: A Phonetically Transcribed Speech Dataset from an Endangered Language for Universal Phone Recognition Experiments,

    D. R. Mortensen, J. Picone, X. Li, and K. Siminyu, “Tusom2021: A Phonetically Transcribed Speech Dataset from an Endangered Language for Universal Phone Recognition Experiments,” inProc. Interspeech, 2021, pp. 3660–3664

    2021

  7. [7]

    Prosodic abx: A language-agnostic method for measuring prosodic contrast in speech representations,

    H. Sun, S. McIntosh, K. Choi, E. Yeo, D. Saito, and N. Minematsu, “Prosodic abx: A language-agnostic method for measuring prosodic contrast in speech representations,”Interspeech, 2026

    2026

  8. [8]

    Speech playground: An interactive tool for speech analysis and comparison,

    S. McIntosh, D. Saito, and N. Minematsu, “Speech playground: An interactive tool for speech analysis and comparison,”arXiv preprint arXiv:2607.00418, 2026

    arXiv 2026

Show all 65 references
  1. [9]

    Towards language-agnostic stipa: Universal phonetic transcription to support language documentation at scale,

    J. L. Suchardt, H. El-Shazli, and P. Cassotti, “Towards language-agnostic stipa: Universal phonetic transcription to support language documentation at scale,” inEMNLP, 2025

    2025

  2. [10]

    Language documentation twenty-five years on,

    F. Seifart, N. Evans, H. Hammarstr ¨om, and S. C. Levinson, “Language documentation twenty-five years on,”Language, vol. 94, no. 4, pp. e324– e345, 2018

    2018

  3. [11]

    The buckeye corpus of conversational speech: labeling conventions and a test of transcriber reliability,

    M. A. Pitt, K. Johnson, E. Hume, S. Kiesling, and W. Raymond, “The buckeye corpus of conversational speech: labeling conventions and a test of transcriber reliability,”Speech Communication, vol. 45, pp. 89–95, 2005

    2005

  4. [12]

    Reliability studies in broad and narrow phonetic transcription,

    L. D. Shriberg and G. L. Lof, “Reliability studies in broad and narrow phonetic transcription,”Clinical Linguistics & Phonetics, vol. 5, no. 3, pp. 225–279, 1991

    1991

  5. [13]

    Simple and Effective Zero-shot Cross- lingual Phoneme Recognition,

    Q. Xu, A. Baevski, and M. Auli, “Simple and Effective Zero-shot Cross- lingual Phoneme Recognition,” inInterspeech, 2022

    2022

  6. [14]

    ZIPA: A family of efficient models for multilingual phone recognition,

    J. Zhu, F. Samir, E. Chodroff, and D. R. Mortensen, “ZIPA: A family of efficient models for multilingual phone recognition,” inACL, 2025

    2025

  7. [15]

    POWSM: A phonetic open whisper-style speech foundation model,

    C.-J. Li, K. Chang, S. Bharadwaj, E. Yeo, K. Choi, J. Zhu, D. Mortensen, and S. Watanabe, “POWSM: A phonetic open whisper-style speech foundation model,” inACL, 2026

    2026

  8. [16]

    An Empirical Recipe for Universal Phone Recognition,

    S. Bharadwaj, C.-J. Li, K. Choi, E. Yeo, W. Chen, S. Watanabe, and D. R. Mortensen, “An Empirical Recipe for Universal Phone Recognition,” in Interspeech, 2026

    2026

  9. [17]

    Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,

    A. Graves, “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” inICML, 2006

    2006

  10. [18]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” inNeurIPS, 2017

    2017

  11. [19]

    Attention-based models for speech recognition,

    J. K. Chorowski, D. Bahdanau, D. Serdyuk, K. Cho, and Y . Bengio, “Attention-based models for speech recognition,”Advances in neural information processing systems, vol. 28, 2015

    2015

  12. [20]

    Phoneme Segmentation Using Self-Supervised Speech Models,

    L. Strgar and D. Harwath, “Phoneme Segmentation Using Self-Supervised Speech Models,” inSLT, 2023

    2023

  13. [21]

    Phone-to-audio alignment without text: A semi-supervised approach,

    J. Zhu, C. Zhang, and D. Jurgens, “Phone-to-audio alignment without text: A semi-supervised approach,” inICASSP, 2022

    2022

  14. [22]

    Explore wav2vec 2.0 for mispronunciation detection

    X. Xu, Y . Kang, S. Cao, B. Lin, and L. Ma, “Explore wav2vec 2.0 for mispronunciation detection.” inInterspeech, 2021

    2021

  15. [23]

    Speech intelligibility assessment of dysarthric speech by using goodness of pronunciation with uncertainty quantification,

    E. J. Yeo, K. Choi, S. Kim, and M. Chung, “Speech intelligibility assessment of dysarthric speech by using goodness of pronunciation with uncertainty quantification,” inInterspeech, 2023

    2023

  16. [24]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” in NeurIPS, 2020

    2020

  17. [25]

    HuBERT: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “HuBERT: Self-supervised speech representation learning by masked prediction of hidden units,”IEEE/ACM TASLP, 2021

    2021

  18. [26]

    WavLM: Large-scale self-supervised pre- training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiaoet al., “WavLM: Large-scale self-supervised pre- training for full stack speech processing,”J-STSP, 2022

    2022

  19. [27]

    [b]=[d]- [t]+[p]: Self-supervised speech models discover phonological vector arithmetic,

    K. Choi, E. Yeo, C. J. Cho, D. Harwath, and D. R. Mortensen, “[b]=[d]- [t]+[p]: Self-supervised speech models discover phonological vector arithmetic,” inACL Findings, 2026

    2026

  20. [28]

    Self- supervised speech models encode phonetic context via position-dependent orthogonal subspaces,

    K. Choi, E. Yeo, C. J. Cho, D. R. Mortensen, and D. Harwath, “Self- supervised speech models encode phonetic context via position-dependent orthogonal subspaces,”arXiv preprint arXiv:2603.12642, 2026

    2026

  21. [29]

    Opening the black box of wav2vec feature encoder,

    K. Choi and E. J. Yeo, “Opening the black box of wav2vec feature encoder,”arXiv preprint arXiv:2210.15386, 2022

    2022 arXiv

  22. [30]

    Analysing discrete self supervised speech representation for spoken language modeling,

    A. Sicherman and Y . Adi, “Analysing discrete self supervised speech representation for spoken language modeling,” inICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023, pp. 1–5

    2023

  23. [31]

    Leveraging allophony in self-supervised speech models for atypical pronunciation assessment,

    K. Choi, E. Yeo, K. Chang, S. Watanabe, and D. R. Mortensen, “Leveraging allophony in self-supervised speech models for atypical pronunciation assessment,” inNAACL, 2025

    2025

  24. [32]

    Layer-wise analysis of a self- supervised speech representation model,

    A. Pasad, J.-C. Chou, and K. Livescu, “Layer-wise analysis of a self- supervised speech representation model,” inASRU, 2021

    2021

  25. [33]

    What do Speech Foundation Models Learn? Analysis and Applications,

    A. Pasad, “What do Speech Foundation Models Learn? Analysis and Applications,” Ph.D. dissertation, Toyota Technical Institute at Chicago, 2025, accessed on https://arxiv.org/abs/2508.12255

    2025 arXiv

  26. [34]

    Panphon: A resource for mapping ipa segments to articulatory feature vectors,

    D. R. Mortensen, P. Littell, A. Bharadwaj, K. Goyal, C. Dyer, and L. Levin, “Panphon: A resource for mapping ipa segments to articulatory feature vectors,” inCOLING, 2016

    2016

  27. [35]

    Multi-level acoustic segmentation of continuous speech,

    J. Glass and V . Zue, “Multi-level acoustic segmentation of continuous speech,” inICASSP, 1988

    1988

  28. [36]

    Segmentation and modeling in segment- based recognition,

    J. W. Chang and J. R. Glass, “Segmentation and modeling in segment- based recognition,” inEurospeech, 1997

    1997

  29. [37]

    Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentation,

    F. Kreuk, J. Keshet, and Y . Adi, “Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentation,” inInterspeech, 2020

    2020

  30. [38]

    A simple hmm with self-supervised represen- tations for phone segmentation,

    G.-P. Yang and H. Tang, “A simple hmm with self-supervised represen- tations for phone segmentation,” inSLT, 2024

    2024

  31. [39]

    Unsupervised Speech Segmentation and Variable Rate Representation Learning Using Segmental Contrastive Predictive Coding,

    S. Bhati, J. Villalba, P. ˙Zelasko, L. Moro-Velazquez, and N. Dehak, “Unsupervised Speech Segmentation and Variable Rate Representation Learning Using Segmental Contrastive Predictive Coding,”IEEE/ACM TASLP, 2022

    2022

  32. [40]

    Universal phone recognition with a multilingual allophone system,

    X. Li, S. Dalmia, J. Li, M. Lee, P. Littell, J. Yao, A. Anastasopoulos, D. R. Mortensen, G. Neubig, A. W. Blacket al., “Universal phone recognition with a multilingual allophone system,” inICASSP, 2020

    2020

  33. [41]

    Allophant: Cross-lingual Phoneme Recognition with Articulatory Attributes,

    K. Glocker, A. Herygers, and M. Georges, “Allophant: Cross-lingual Phoneme Recognition with Articulatory Attributes,” inInterspeech, 2023

    2023

  34. [42]

    Text-Independent Phone-to-Audio Alignment Leveraging SSL (TIPAA-SSL) Pre-Trained Model Latent Representation and Knowledge Transfer,

    N. Tits, P. Bhatnagar, and T. Dutoit, “Text-Independent Phone-to-Audio Alignment Leveraging SSL (TIPAA-SSL) Pre-Trained Model Latent Representation and Knowledge Transfer,”Acoustics, vol. 6, no. 3, pp. 772–781, Sep. 2024

    2024

  35. [43]

    Self-Supervised Speech Representations are More Phonetic than Semantic,

    K. Choi, A. Pasad, T. Nakamura, S. Fukayama, K. Livescu, and S. Watanabe, “Self-Supervised Speech Representations are More Phonetic than Semantic,” inInterspeech, 2024

    2024

  36. [44]

    SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python,

    P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, ˙I. Polat, Y . Fen...

    2020

  37. [45]

    A new text-independent method for phoneme segmentation,

    G. Aversano, A. Esposito, and M. Marinaro, “A new text-independent method for phoneme segmentation,” inMidwest Symposium on Circuits and Systems (MWSCAS). IEEE, 2001

    2001

  38. [46]

    Montreal forced aligner: Trainable text-speech alignment using kaldi,

    M. McAuliffe, M. Socolof, S. Mihuc, M. Wagner, and M. Sonderegger, “Montreal forced aligner: Trainable text-speech alignment using kaldi,” inInterspeech, vol. 2017, 2017, pp. 498–502

    2017

  39. [47]

    XLSR Inclusive English Speech-to-IPA,

    K. Labs, “XLSR Inclusive English Speech-to-IPA,” 2025. [Online]. Available: https://huggingface.co/collections/KoelLabs/ xlsr-inclusive-english-speech-to-ipa

    2025

  40. [48]

    L2-ARCTIC: A Non-native English Speech Corpus,

    G. Zhao, S. Sonsaat, A. Silpachai, I. Lucic, E. Chukharev-Hudilainen, J. M. Levis, and R. Gutierrez-Osuna, “L2-ARCTIC: A Non-native English Speech Corpus,” inInterspeech, 2018

    2018

  41. [49]

    Speech Accent Archive,

    S. Weinberger, “Speech Accent Archive,” 2015, retrieved from https: //accent.gmu.edu

    2015

  42. [50]

    Building a time-aligned cross-linguistic reference corpus from language documentation data (DoReCo),

    L. Paschen, F. Delafontaine, C. Draxler, S. Fuchs, M. Stave, and F. Seifart, “Building a time-aligned cross-linguistic reference corpus from language documentation data (DoReCo),” inProc. LREC. European Language Resources Association, 2020

    2020

  43. [51]

    Scaling human and g2p supervision for robust phonetic transcription,

    A. Metzger, A. Srivastava, and R. Mukhamedvaleev, “Scaling human and g2p supervision for robust phonetic transcription,” inInterspeech, 2026

    2026

  44. [52]

    Global TIMIT learner simple english,

    H. Ding, S. Liao, Y . Zhan, H. Feng, W. He, X. Hu, Y . Wu, J. Yuan, and M. Liberman, “Global TIMIT learner simple english,” Web Download. LDC2020S11, Philadelphia, 2020. [Online]. Available: https://catalog.ldc.upenn.edu/LDC2020S11

    2020

  45. [53]

    Global TIMIT learner treebank english,

    H. Luan, Y . Wang, H. Feng, W. He, X. Hu, Y . Wu, J. Yuan, and M. Liberman, “Global TIMIT learner treebank english,” Web Download. LDC2020S09, Philadelphia, 2020. [Online]. Available: https://catalog.ldc.upenn.edu/LDC2020S09

    2020

  46. [54]

    Global TIMIT Thai,

    M. Liberman, J. Yuan, C. Cieri, J. Wright, and N. Chanchaochai, “Global TIMIT Thai,” Web Download. LDC2022S13, Philadelphia, 2022. [Online]. Available: https://catalog.ldc.upenn.edu/LDC2022S13

    2022

  47. [55]

    Dysarthric speech corpus in Tamil for rehabilitation research,

    T. A. Mariya Celin, T. Nagarajan, and P. Vijayalakshmi, “Dysarthric speech corpus in Tamil for rehabilitation research,” in2016 IEEE Region 10 Conference (TENCON), 2016, pp. 2610–2613

    2016

  48. [56]

    A weighted speaker-specific confusion transducer-based augmentative and alternative speech communication aid for dysarthric speakers,

    T. A. Mariya Celin, G. Anushiya Rachel, T. Nagarajan, and P. Vi- jayalakshmi, “A weighted speaker-specific confusion transducer-based augmentative and alternative speech communication aid for dysarthric speakers,”IEEE Transactions on Neural Systems and Rehabilitation Engineeri...

    2019

  49. [57]

    The TORGO database of acoustic and articulatory speech from speakers with dysarthria,

    F. Rudzicz, A. K. Namasivayam, and T. Wolff, “The TORGO database of acoustic and articulatory speech from speakers with dysarthria,”Language Resources and Evaluation, vol. 46, no. 4, pp. 523–541, 2012

    2012

  50. [58]

    Dysarthria detection and severity assessment using rhythm-based metrics

    A. Hernandez, E. J. Yeo, S. Kim, and M. Chung, “Dysarthria detection and severity assessment using rhythm-based metrics.” inInterspeech, 2020

    2020

  51. [59]

    An improved speech segmentation quality measure: the r-value,

    O. J. R ¨as¨anen, U. K. Laine, and T. Altosaar, “An improved speech segmentation quality measure: the r-value,” inInterspeech, 2009

    2009

  52. [60]

    English mfa acoustic model v3.1.0,

    M. McAuliffe and M. Sonderegger, “English mfa acoustic model v3.1.0,” https://mfa-models.readthedocs.io/acoustic/English/ EnglishMFAacousticmodelv3 1 0.html, Tech. Rep., Jun 2024

    2024

  53. [61]

    V oxcommunis corpus,

    E. Ahn and E. Chodroff, “V oxcommunis corpus,” https://osf.io/t957v, Jan 2022

    2022

  54. [62]

    Thai mfa acoustic model v3.0.0,

    M. McAuliffe and M. Sonderegger, “Thai mfa acoustic model v3.0.0,” https://mfa-models.readthedocs.io/acoustic/Thai/ ThaiMFAacousticmodelv3 0 0.html, Tech. Rep., Feb 2024

    2024

  55. [63]

    Wav2Gloss: Generating Interlinear Glossed Text from Speech,

    T. He, K. Choi, L. Tjuatja, N. Robinson, J. Shi, S. Watanabe, G. Neubig, D. Mortensen, and L. Levin, “Wav2Gloss: Generating Interlinear Glossed Text from Speech,” inACL, 2024

    2024

  56. [64]

    XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale,

    A. Babu, C. Wang, A. Tjandra, K. Lakhotia, Q. Xu, N. Goyal, K. Singh, P. von Platen, Y . Saraf, J. Pino, A. Baevski, A. Conneau, and M. Auli, “XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale,” inInterspeech, 2022

    2022

  57. [65]

    Towards unsupervised phone and word segmentation using self-supervised vector-quantized neural networks,

    H. Kamper and B. v. Niekerk, “Towards unsupervised phone and word segmentation using self-supervised vector-quantized neural networks,” in Interspeech, 2021

    2021

Pith tools

Reviewed July 13, 2026 · model on record in the stance chip above.