Pith. sign in

REVIEW 5 major objections 6 minor 46 references

Seamless Dysfluent Speech Text Alignment for Disordered Speech Analysis

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A new neural aligner, Neural LCS, learns phoneme similarity to map dysfluent speech onto intended text, outperforming exact-match baselines in alignment accuracy and boundary loss.

desk verdict Sensible supervised dysfluent aligner with useful new simulated corpora, but evaluation is too circular and the PPA ground truth is unverified; worth reviewing, not accepting as-is. read the letter →

arxiv 2506.12073 v1 pith:DUQURN2D submitted 2025-06-05 eess.AS cs.AIcs.CLcs.SD

Zongli Ye , Jiachen Lian , Xuanru Zhou , Jinming Zhang , Haodong Li , Shuhe Li , Chenxu Guo , Anaisha Das
show 9 more authors
Peter Park Zoe Ezzes Jet Vonk Brittany Morin Rian Bogley Lisa Wauters Zachary Miller Maria Gorno-Tempini Gopala Anumanchipalli
This is my paper
classification eess.AScs.AIcs.CLcs.SD
keywords dysfluentspeechalignmentforcedlongestcommonsubsequencephonemesimilaritysegmentationprimaryprogressiveaphasiadysfluencydetectionneuralsequence
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

To diagnose neurodegenerative speech disorders, software must map what a patient actually said onto the text they meant to say; the mismatches are the dysfluencies. This paper argues that existing aligners fail because they require exact token matches, so they cannot pair acoustically similar phonemes such as P and B. Neural LCS replaces exact matching with a learned soft alignment that marks each unit as aligned (1), dysfluent within an aligned region (0), or missing (2). On simulated text-text and LLM-generated dysfluent speech, the method lifts phoneme-level alignment accuracy to 72.55% (text-text) and 90.96% (LLM text). It also segments audio of real primary progressive aphasia patients with a lower boundary loss (17 ms) than a strong stutter-detection baseline (21 ms), which matters for automated diagnosis.

What carries the argument

The central object is Neural LCS, a neural generalization of the longest common subsequence aligner. Instead of requiring exact token equality, it uses a siamese (shared-weights) network: a transformer text encoder processes the reference and the dysfluent sequences, the resulting representations are concatenated, and a 1D convolutional network plus multilayer perceptron with softmax emits per-position alignment labels (0, 1, or 2). Contextual attention lets a K align with a G or a vowel with a similar vowel, which is what partial and substitution-heavy dysfluencies require. For speech input, a self-supervised speech feature extractor with connectionist temporal classification decoding first produces the dysfluent phoneme sequence, and Neural LCS then aligns that sequence to the reference; the simulation pipeline supplies training labels by injecting phoneme-category-based dysfluencies into English text and by generating natural dysfluent text with an LLM and audio with an end-to-end TTS model.

What would settle it

Run the speech-text alignment model against the exact-match baseline on a corpus of patient audio with clinician-verified dysfluent boundaries and phoneme-level transcriptions; if the neural aligner does not maintain substantially lower boundary loss and higher alignment accuracy, the central claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that dysfluent speech alignment should be treated as a learned, context-sensitive version of the longest common subsequence problem rather than as exact string matching. The Neural LCS model encodes reference and dysfluent sequences with a shared transformer encoder, concatenates the two representations, and applies a convolutional classifier to predict a label sequence in which 1 means aligned, 0 marks a dysfluent unit inside an aligned segment, and 2 marks a missing phoneme or word. It is trained with a class-imbalance-weighted loss to handle the fact that most units are aligned. The authors report that this soft aligner outperforms dynamic time warping and hard LCS on both phoneme and word levels, and that the speech-to-text variant built on connectionist temporal classification plus Neural LCS lowers the boundary loss of dysfluent segments on both simulated disordered speech and real primary progressive aphasia speech.

Load-bearing premise

The load-bearing premise is that the simulated labels used for training and evaluation behave like real dysfluent speech; if the simulation rules do not match how people with primary progressive aphasia actually speak, the reported gains may not appear on clinical audio.

Editorial extensions

If this is right

  • Phoneme-level alignment of dysfluent text rises from 24.78% to 72.55% on simulated text-text data and from 43.53% to 90.96% on LLM-generated text, so downstream dysfluency counts should become far more reliable.
  • On real primary progressive aphasia audio, the speech-text model segments dysfluent regions with 17 ms boundary loss versus 21 ms for the stutter-detection baseline, indicating clinically usable segmentation.
  • The 0/1/2 labeling makes each alignment output a dysfluency annotation, so insertion and deletion detection do not require a separate post-processing stage.
  • Repetition accuracy stays near 96% across different training mixes of dysfluency types, while substitution accuracy responds to the proportion of substitutions in the training data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the real-patient evaluation reports only an aggregate boundary loss with no manual annotation or per-type breakdown, a clinician-annotated comparison is the open test of whether the simulated-data gains transfer; the numbers are suggestive but not conclusive.
  • If learned phoneme similarity is the driver of the gains, replacing the hand-built phoneme categories with embeddings fit on real disordered speech should further improve substitution alignment, an experiment the paper leaves to future work.
  • The alignment objective is not tied to English: with a suitable tokenizer and speech encoder it could be applied to non-English dysfluent speech, but the current simulated data and phoneme categories are English-only.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes Neural LCS, a siamese-network-based soft alignment method for dysfluent text-text and speech-text alignment, and an associated STA model that combines CTC-based phoneme decoding with Neural LCS to segment dysfluent speech. The authors construct two simulated resources: a text-text corpus built by injecting dysfluencies into VCTK text according to hand-defined CMU phoneme similarity categories, and an LLM disorder corpus generated by Claude-generated dysfluent text synthesized with VITS TTS, where VITS internal phoneme frame boundaries serve as ground truth. They evaluate alignment accuracy against DTW and Hard LCS on simulated text, and boundary loss against YOLO-Stutter on LLM disorder audio and on PPA speech from 38 participants. The central claim is that Neural LCS significantly outperforms state-of-the-art models in both alignment accuracy and dysfluent speech segmentation.

Significance. If validated, the proposed soft, context-aware LCS formulation would be a genuinely useful contribution to disordered speech analysis: it addresses partial alignment and phonetic similarity in a linguistically grounded way, and the released code and simulated corpora could benefit the community. The paper also makes a laudable effort to generate large-scale dysfluent training data. However, the current evidence is weakened by circular evaluation on the text-text benchmark, an underspecified PPA gold standard, an overbroad abstract claim contradicted by the deletion result, and the absence of comparisons with the closest prior dysfluent aligners. The core idea is plausible, but the experimental support is not yet commensurate with the claims.

major comments (5)
  1. [§2.2.1, Table 1, §3.3.1, Table 3] The text-text evaluation is circular: the training labels are generated by injecting dysfluencies using the hand-defined CMU phoneme categories in Table 1, and the test set is generated by the same rule. The model is therefore being evaluated on its ability to reproduce the very simulation rule it was trained on, so the 72.55% phoneme-level accuracy on text-text data does not by itself demonstrate generalization to real dysfluent speech. The authors should validate on an independently annotated corpus of dysfluent transcripts, or at minimum report a human-annotated subset, and clearly separate 'simulation-rule reconstruction' from 'alignment accuracy on real dysfluencies'.
  2. [§3.3.2, Table 4] The PPA Speech evaluation does not define the ground-truth boundaries used to compute Boundary Loss. No manual annotation protocol, number of dysfluent regions, or inter-annotator agreement is reported, so the 17 ms versus 21 ms comparison is uninterpretable. If the 'actual boundaries' were obtained from the same CTC/Neural LCS pipeline used to train the STA model, the comparison with YOLO-Stutter would be circular. The authors must describe how PPA boundaries were obtained and ideally report per-dysfluency-type results on PPA as well.
  3. [§3.3.2, Table 4 (LLM disorder rows)] The abstract's blanket claim that Neural LCS 'significantly outperforms state-of-the-art models in ... dysfluent speech segmentation' is contradicted by Table 4 on the LLM disorder set: the STA model has a Boundary Loss of 27 ms for deletions versus 10 ms for YOLO-Stutter, i.e., it is substantially worse on one of the four dysfluency types. The authors should qualify the claim, explain why deletion detection is worse, and report aggregate metrics that include deletion or per-type significance tests.
  4. [§2.1, §3.3.1] The paper asserts state-of-the-art performance but never compares against the closest prior dysfluent aligners: SSDM/CSA [5, 6] and the weakly-supervised forced aligner of [4]. The Introduction explicitly discusses SSDM/CSA and motivates Neural LCS as an improvement over CSA, yet no experimental comparison is provided. Given that SSDM/CSA is the direct predecessor and one of the authors is a co-author of that work, this omission is particularly consequential for the 'state-of-the-art' claim. The authors should add these baselines to Tables 3 and 4 or justify their exclusion with quantitative evidence.
  5. [§3.3, Tables 3 and 4] No variance or significance testing is reported. The abstract uses 'significantly outperforms,' but the tables show single point estimates without error bars, confidence intervals, or statistical tests (e.g., paired bootstrap or multiple-seed runs). The authors should report standard deviations across seeds or bootstrap intervals, and perform a significance test for the headline comparisons in Tables 3 and 4.
minor comments (6)
  1. [§2.2.1, Table 1] In the Vowel row, 'IH IY' should read 'IH, IY' to be consistent with the other comma-separated lists.
  2. [§2.4] The text reads 'wax2vec2.0 feature extractor'; this should be 'wav2vec2.0'.
  3. [§3.3.1] The section title 'Speech-text alignment' appears twice (in §3.3.1 and §3.3.2), but §3.3.1 actually reports text-text alignment results; the first title should be renamed, e.g., 'Text-text alignment'.
  4. [§3.1] The PPA Speech description states 'approximately one hour of speech in total' for 38 participants reading the grandfather passage; please clarify the duration per participant and whether all recordings were analyzable.
  5. [Table 2] The notation '110.7×10^4 sentences' is unconventional; use '1.107M' or '1,107,000' for readability.
  6. [References] Reference [7] cites a Wikipedia page for the longest common subsequence; it would be more appropriate to cite a peer-reviewed source such as Hirschberg (1977), which is already [26].

Circularity Check

2 steps flagged · score 6.0 of 10

Central results are evaluated against labels produced by the paper's own simulation pipeline; PPA ground truth is unspecified.

  1. fitted input called prediction [Sec. 2.2.1 + Sec. 3.3.1 (Table 3)]
    "We heuristically categorize CMU phonemes [28] based on airflow patterns and articulatory mechanisms to capture phoneme similarities, as shown in Table 1. Phonemes in the same category tend to be more similar and often appear in phonetic errors. ... We generate alignment labels between the original reference and dysfluent sequences. We use 1 to mark aligned boundaries in the reference, 0 for dysfluent units within the aligned part, and 2 for missing phonemes or words. ..."

    The text-text training and test labels are both produced by the same 'probabilistic random dysfluency injection and alignment label annotation' (Fig. 1a), whose substitution, repetition, insertion, and deletion types are driven by the hand-defined CMU categories in Table 1. Neural LCS is trained with Focal Loss to predict exactly these 0/1/2 labels, so Table 3's 'alignment accuracy' is agreement with the same heuristic rule that generated the training data. The large gap over Hard LCS (72.55% vs. 24.78% at phoneme level) shows the model learned the authors' category table, not that the table is an independent ground truth for disordered speech.

  2. fitted input called prediction [Sec. 2.2.2 + Sec. 2.4 + Sec. 3.3.2 (Table 4)]
    "In the TTS process, we obtain the vector representing the number of time frames occupied by each phoneme in the audio. By mapping the IPA phonemes to their corresponding CMU phonemes, we derive the true alignment between the audio time and its reference phonemes. ... and is trained with the source dysfluent sequence of the generated speech through CTC-Loss. ... We evaluate the models using Boundary Loss (BL): the mean squared error between the predicted and actual boundaries of the dysfluent regions."

    For the LLM-disorder corpus, the 'true alignment' is defined by VITS's internal phoneme-frame counts. The STA model is trained with CTC on the same VITS-generated audio to align to the known source dysfluent phoneme sequence, so the boundary-loss numbers reported for LLM disorder measure how well the model reconstructs the paper's own TTS alignment rather than an independent dysfluency segmentation benchmark. The PPA rows in Table 4 would supply external grounding, but no annotation protocol, manual labels, or inter-annotator agreement is described, so the 'actual boundaries' are unspecified. The real-data claim in the abstract is therefore unverifiable from the text as written.

full rationale

The paper's central claims are evaluated primarily on simulated data whose labels are generated by the same hand-defined phoneme categories and the same TTS alignment pipeline used to train the model. The text-text evaluation in Table 3 compares the model against labels produced by the authors' injection rule (Sec. 2.2.1, Fig. 1a), so the reported accuracy reflects how well the model reproduces that rule, not an external standard for disordered speech. The LLM-disorder evaluation partly breaks this by using Claude-generated text and VITS audio, but the audio ground truth is still VITS's internal per-phoneme frame counts, which is also the target of the CTC training in Sec. 2.4. The PPA evaluation in Table 4 provides potential external grounding, but the paper never states what the 'actual boundaries' are, describes no manual annotation protocol, and reports no per-type metrics for PPA. Consequently, the abstract's claim of 'significantly outperforms state-of-the-art models in both alignment accuracy and dysfluent speech segmentation' is forced by construction on the simulated portions and unverified on the real-data portion. I therefore assign a circularity score of 6, indicating that several headline 'predictions' reduce to the training/evaluation pipeline itself.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

All quantitative results rest on labels generated by the paper's own simulation pipeline: hand-coded phoneme categories (Table 1), a closed set of four dysfluency types, Claude-generated dysfluent text, and VITS TTS alignments. The real PPA data provides an external check but is evaluated with a single coarse metric. No independent gold-standard alignment labels are used anywhere in training or in the text-text evaluation.

free parameters (3)
  • Focal loss alpha values = [0.5, 0.1, 0.8]
    Class weighting for labels 0, 1, and 2 in Eq. (1); hand-set in Sec. 2.3.2 without ablation and used for all experiments.
  • Focal loss gamma = 3
    Down-weighting exponent hand-set in Sec. 2.3.2 without ablation.
  • Dysfluency injection probabilities = not reported
    Sec. 2.2.1 uses probability-based random injection of the four dysfluency types but never states the rates; these rates define the training distribution and the reported type-specific accuracies.
assumptions (6)
  • ad hoc to paper The CMU phoneme categories in Table 1 are sufficient to model pronunciation similarity for dysfluency simulation and evaluation.
    Both label generation and the Neural LCS training target rely on this hand-coded grouping; no empirical or articulatory validation is given.
  • domain assumption The four textual dysfluency types (repetition, deletion, substitution, insertion) are exhaustive and textually representable.
    Used to construct all simulation labels; real PPA speech may include other phenomena.
  • domain assumption Claude-generated dysfluent text resembles real disordered speech.
    The LLM disorder corpus is built with prompts; no human or PPA evaluation of naturalness is reported.
  • ad hoc to paper VITS TTS internal phoneme frame boundaries are valid ground-truth alignments for dysfluent speech.
    CTC training and Boundary Loss evaluation rely on TTS-internal alignments for synthetic audio; no manual annotation is used for LLM disorder.
  • domain assumption CTC transcriptions of synthetic audio transfer to real PPA audio without adaptation.
    Section 2.4 trains on LLM disorder synthetic audio only; PPA inference uses the same model.
  • domain assumption T5 contextual embeddings encode phonetic and morphological similarity needed for soft alignment.
    Core hypothesis of Neural LCS; not separately validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Seamless Dysfluent Speech Text Alignment for Disordered Speech Analysis." pith.science (2026). https://pith.science/paper/DUQURN2D

@misc{pith2026250612073,
  author       = {Pith},
  title        = {Pith review of: Seamless Dysfluent Speech Text Alignment for Disordered Speech Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DUQURN2D}},
  note         = {Machine review of arXiv:2506.12073}
}
read the original abstract

Accurate alignment of dysfluent speech with intended text is crucial for automating the diagnosis of neurodegenerative speech disorders. Traditional methods often fail to model phoneme similarities effectively, limiting their performance. In this work, we propose Neural LCS, a novel approach for dysfluent text-text and speech-text alignment. Neural LCS addresses key challenges, including partial alignment and context-aware similarity mapping, by leveraging robust phoneme-level modeling. We evaluate our method on a large-scale simulated dataset, generated using advanced data simulation techniques, and real PPA data. Neural LCS significantly outperforms state-of-the-art models in both alignment accuracy and dysfluent speech segmentation. Our results demonstrate the potential of Neural LCS to enhance automated systems for diagnosing and analyzing speech disorders, offering a more accurate and linguistically grounded solution for dysfluent speech alignment.

Figures

Figures reproduced from arXiv: 2506.12073 by the authors.

Figure 1
Figure 1. (a)shows our probabilistic random dysfluency injection and alignment label annotation. (b)shows how we combine LLMs and TTS to generate disfluent text and corresponding audio. Statistics of these two simulated data is shown in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Structure of Speech-text alignment model 2.3.1. Feature Encoder We use the default T5 [33] feature encoder with a fully-visible mask, enabling all tokens to attend to each other. For phoneme￾level tokenization, we implemented a custom tokenizer based on the CMU phoneme dictionary, while for word-level tok￾enization, we used the default T5-small tokenizer. 2.3.2. Training Objective Our alignment label exhibits an imb… view at source ↗
Figure 3
Figure 3. (a) shows that our word-level Neural LCS model captures acoustic similarities between words, even if they contain different letters, like swiftly-wishy. (b) demonstrates the model’s ability to capture morphological similarities, even without similar pronuncia￾tions, like plays-playing. (c) highlights the phoneme-level model’s ability to capture consonant similarities, such as K-G, S-Z. (d) shows that the model captu… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

46 extracted references · 32 canonical work pages

  1. [4]

    Conclusion and Future Work We propose Neural LCS, a novel method for aligning dysflu- ent speech that overcomes limitations of existing approaches. It operates in two modes: (1) aligning transcribed phonemes or words to reference sequences using acoustic or morpho- logical cues, and (2) directly segmenting speech based on the reference. By leveraging acou...

    work page

  2. [1]

    Introduction The diagnosis and analysis of neurodegenerative speech disor- ders, such as primary progressive aphasia (PPA) [22], tradition- ally depend on real-time MRIs (rtMRIs) and manual speech transcripts by speech-language pathologists (SLPs). Recent automated approaches for diagnosing and analyzing dysfluent speech have focused on comparing uttered ...

    work page Pith review arXiv 2025

  3. [2]

    Methods 2.1. Hard LCS Dysfluencies often have aligned targets, so applying the longest common subsequence (LCS) algorithm [26] can automatically align dysfluent parts to corresponding phonemes or words. This works due to LCS’s local alignment nature, as pointed out in [5]. The standard LCS algorithm uses dynamic programming, re- quiring exact token matche...

    work page

  4. [3]

    Dataset (1) VCTK [19]:it includes 109 native English speakers with accented speech

    Experiments 3.1. Dataset (1) VCTK [19]:it includes 109 native English speakers with accented speech. It’s text is used in our text-text data sim- ulation as mentioned in Sec.2.2.1.(2) LLM disorder:We use LLM+TTS to generate large scale more natural dysfluent text-speech data. The detail is shown in Sec.2.2.2.(3) PPA Speech [22]:it includes recordings from...

    work page

  5. [5]

    Acknowledgements Thanks for support from UC Noyce Initiative, Society of Hell- man Fellows, NIH/NIDCD, and the Schwab Innovation fund

  6. [6]

    Ssdm 2.0: Time-accurate speech rich transcription with non-fluencies,

    J. Lian, X. Zhou, Z. Ezzes, J. V onk, B. Morin, D. Baquirin, Z. Mille, M. L. G. Tempini, and G. K. Anumanchipalli, “Ssdm 2.0: Time-accurate speech rich transcription with non-fluencies,” arXiv preprint arXiv:2412.00265, 2024

    arXiv 2024

  7. [7]

    Unconstrained dysfluency modeling for dysfluent speech transcription and de- tection,

    J. Lian, C. Feng, N. Farooqi, S. Li, A. Kashyap, C. J. Cho, P. Wu, R. Netzorg, T. Li, and G. K. Anumanchipalli, “Unconstrained dysfluency modeling for dysfluent speech transcription and de- tection,” in2023 IEEE Automatic Speech Recognition and Under- standing Workshop (ASRU), 2023, pp. 1–8

    work page 2023

  8. [8]

    Towards hierarchical spoken lan- guage disfluency modeling,

    J. Lian and G. Anumanchipalli, “Towards hierarchical spoken lan- guage disfluency modeling,” inProceedings of the 18th Confer- ence of the European Chapter of the Association for Computa- tional Linguistics, 2024, pp. 539–551

    work page 2024

Show all 46 references
  1. [9]

    Pattern search in dysfluent speech,

    J. P ´alfy and J. Posp ´ıchal, “Pattern search in dysfluent speech,” in2012 IEEE International Workshop on Machine Learning for Signal Processing, 2012, pp. 1–6

    2012

  2. [10]

    Weakly-supervised forced alignment of disfluent speech using phoneme-level modeling,

    T. Kouzelis, G. Paraskevopoulos, A. Katsamanis, and V . Kat- souros, “Weakly-supervised forced alignment of disfluent speech using phoneme-level modeling,” inInterspeech, 2023

    2023

  3. [11]

    Ssdm: Scalable speech dysfluency modeling,

    J. Lian, X. Zhou, Z. Ezzes, J. V onk, B. Morin, D. P. Baquirin, Z. Miller, M. L. Gorno Tempini, and G. Anumanchipalli, “Ssdm: Scalable speech dysfluency modeling,” inAdvances in Neural In- formation Processing Systems, vol. 37, 2024

    2024

  4. [12]

    Articulatory representation learning via joint factor analysis and neural matrix factorization,

    J. Lian, A. W. Black, Y . Lu, L. Goldstein, S. Watanabe, and G. K. Anumanchipalli, “Articulatory representation learning via joint factor analysis and neural matrix factorization,” inICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICA...

    2023

  5. [13]

    Longest common subsequence

    Wikipedia, “Longest common subsequence.” [Online]. Available: https://en.wikipedia.org/wiki/Longest common subsequence

  6. [14]

    Yolo-stutter: End-to-end region-wise speech dys- fluency detection,

    X. Zhou, A. Kashyap, S. Li, A. Sharma, B. Morin, D. Baquirin, J. V onk, Z. Ezzes, Z. Miller, M. Tempini, J. Lian, and G. Anu- manchipalli, “Yolo-stutter: End-to-end region-wise speech dys- fluency detection,” inInterspeech 2024, 2024, pp. 937–941

    2024

  7. [15]

    Stutter-solver: End-to-end multi-lingual dysfluency detection,

    X. Zhou, C. J. Cho, A. Sharma, B. Morin, D. Baquirin, J. V onk, Z. Ezzes, Z. Miller, B. L. Tee, M. L. Gorno-Tempiniet al., “Stutter-solver: End-to-end multi-lingual dysfluency detection,” in2024 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2024, pp. 1039–1046

    2024

  8. [16]

    Montreal forced aligner: Trainable text-speech align- ment using kaldi

    M. McAuliffe, M. Socolof, S. Mihuc, M. Wagner, and M. Son- deregger, “Montreal forced aligner: Trainable text-speech align- ment using kaldi.” inInterspeech, vol. 2017, 2017, pp. 498–502

    2017

  9. [17]

    Time and tokens: Benchmarking end- to-end speech dysfluency detection,

    X. Zhou, J. Lian, C. J. Cho, J. Liu, Z. Ye, J. Zhang, B. Morin, D. Baquirin, J. V onk, Z. Ezzes, Z. Miller, M. L. G. Tempini, and G. Anumanchipalli, “Time and tokens: Benchmarking end- to-end speech dysfluency detection,” 2024. [Online]. Available: https://arxiv.org/abs/2409.13582

    2024 arXiv

  10. [18]

    Coding speech through vocal tract kinematics,

    C. J. Cho, P. Wu, T. S. Prabhune, D. Agarwal, and G. K. Anu- manchipalli, “Coding speech through vocal tract kinematics,” in IEEE JSTSP, 2025

    2025

  11. [19]

    Deep Neural Convolutive Matrix Factorization for Articulatory Representation Decomposition,

    J. Lian, A. W. Black, L. Goldstein, and G. K. Anumanchipalli, “Deep Neural Convolutive Matrix Factorization for Articulatory Representation Decomposition,” inProc. Interspeech 2022, 2022, pp. 4686–4690

    2022

  12. [20]

    Deep Speech Synthesis from MRI-Based Articulatory Representations,

    P. Wu, T. Li, Y . Lu, Y . Zhang, J. Lian, A. W. Black, L. Goldstein, S. Watanabe, and G. K. Anumanchipalli, “Deep Speech Synthesis from MRI-Based Articulatory Representations,” inProc. INTER- SPEECH 2023, 2023, pp. 5132–5136

    2023

  13. [21]

    Seamlessm4t-massively multilingual & multimodal ma- chine translation,

    L. Barrault, Y .-A. Chung, M. C. Meglioli, D. Dale, N. Dong, P.- A. Duquenne, H. Elsahar, H. Gong, K. Heffernan, J. Hoffman et al., “Seamlessm4t-massively multilingual & multimodal ma- chine translation,”arXiv preprint arXiv:2308.11596, 2023

    2023 arXiv

  14. [22]

    Con- nectionist temporal classification: labelling unsegmented se- quence data with recurrent neural networks,

    A. Graves, S. Fern ´andez, F. Gomez, and J. Schmidhuber, “Con- nectionist temporal classification: labelling unsegmented se- quence data with recurrent neural networks,” inProceedings of the 23rd international conference on Machine learning, 2006, pp. 369–376

    2006

  15. [23]

    Bayes risk ctc: Controllable ctc alignment in sequence-to-sequence tasks,

    J. Tian, B. Yan, J. Yu, C. Weng, D. Yu, and S. Watanabe, “Bayes risk ctc: Controllable ctc alignment in sequence-to-sequence tasks,”ICLR, 2022

    2022

  16. [24]

    Simple and effective zero-shot cross-lingual phoneme recognition,

    Q. Xu, A. Baevski, and M. Auli, “Simple and effective zero-shot cross-lingual phoneme recognition,”Interspeech, 2022

    2022

  17. [25]

    Cstr vctk corpus: English multi-speaker corpus for cstr voice cloning toolkit (ver- sion 0.92),

    J. Yamagishi, C. Veaux, and K. MacDonald, “Cstr vctk corpus: English multi-speaker corpus for cstr voice cloning toolkit (ver- sion 0.92),” 2019, [sound], University of Edinburgh, The Centre for Speech Technology Research (CSTR)

    2019

  18. [26]

    Phone-to-audio alignment without text: A semi-supervised approach,

    J. Zhu, C. Zhang, and D. Jurgens, “Phone-to-audio alignment without text: A semi-supervised approach,” inICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 8167–8171

    2022

  19. [27]

    Scaling speech technology to 1,000+ languages,

    V . Pratap, A. Tjandra, B. Shi, P. Tomasello, A. Babu, S. Kundu, A. Elkahky, Z. Ni, A. Vyas, M. Fazel-Zarandiet al., “Scaling speech technology to 1,000+ languages,”Journal of Machine Learning Research, vol. 25, no. 97, pp. 1–52, 2024

    2024

  20. [28]

    Classification of primary progressive aphasia and its variants,

    M. L. Gorno-Tempini, A. E. Hillis, S. Weintraub, A. Kertesz, M. Mendez, S. F. Cappa, J. M. Ogar, J. D. Rohrer, S. Black, B. F. Boeveet al., “Classification of primary progressive aphasia and its variants,”Neurology, vol. 76, no. 11, pp. 1006–1014, 2011

    2011

  21. [29]

    Wavlm-ctc-hugginface,

    “Wavlm-ctc-hugginface,” https://huggingface.co/microsoft/ wavlm-large

  22. [30]

    Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,

    J. Kim, J. Kong, and J. Son, “Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,”Interna- tional Conference on Machine learning, 2021

    2021

  23. [31]

    Univer- sal phone recognition with a multilingual allophone system,

    X. Li, S. Dalmia, J. Li, M. Lee, P. Littell, J. Yao, A. Anastasopou- los, D. R. Mortensen, G. Neubig, A. W. Blacket al., “Univer- sal phone recognition with a multilingual allophone system,” in ICASSP. IEEE, 2020, pp. 8249–8253

    2020

  24. [32]

    Algorithms for the longest common subse- quence problem,

    D. S. Hirschberg, “Algorithms for the longest common subse- quence problem,”Journal of the ACM (JACM), vol. 24, no. 4, pp. 664–675, 1977

    1977

  25. [33]

    Dynamic-programming approach to continuous speech recognition,

    H. Sakoe, “Dynamic-programming approach to continuous speech recognition,” in1971 Proc. the International Congress of Acoustics, Budapest, 1971

    1971

  26. [34]

    Cmu phoneme dictionary

    C. M. University, “Cmu phoneme dictionary.” [Online]. Available: http://www.speech.cs.cmu.edu/cgi-bin/cmudict

  27. [35]

    During inference, we apply greedy decoding on CTC emis- sion matrix

    accepts the speech signal generated by LLM and TTS model, it converts the feature dimension of each frame of sig- nal to the length of CMU dictionary + 1 via basic conformer and projection layer, and is trained with the source dysfluent sequence of the generated speech through...

  28. [36]

    Model card addendum: Claude 3.5 haiku and upgraded claude 3.5 sonnet,

    Anthropic, “Model card addendum: Claude 3.5 haiku and upgraded claude 3.5 sonnet,” 2024. [Online]. Available: https: //www.anthropic.com

    2024

  29. [37]

    Sig- nature verification using a Siamese time delay neural network,

    J. Bromley, I. Guyon, Y . LeCun, E. S¨ackinger, and R. Shah, “Sig- nature verification using a Siamese time delay neural network,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 6, 1994, pp. 737–744

    1994

  30. [38]

    A survey on siamese network: Methodologies, applications, and opportunities,

    Y . Li, C. L. P. Chen, and T. Zhang, “A survey on siamese network: Methodologies, applications, and opportunities,”IEEE Transac- tions on Artificial Intelligence, vol. 3, no. 6, pp. 994–1014, 2022

    2022

  31. [39]

    Exploring the limits of transfer learning with a unified text-to-text transformer,

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” 2023. [Online]. Available: https://arxiv.org/abs/1910.10683

    2023 arXiv

  32. [40]

    Focal loss for dense object detection,

    T.-Y . Lin, P. Goyal, R. Girshick, K. He, and P. Doll ´ar, “Focal loss for dense object detection,” 2018. [Online]. Available: https://arxiv.org/abs/1708.02002

    2018 arXiv

  33. [41]

    wav2vec: Unsupervised pre-training for speech recognition,

    S. Schneider, A. Baevski, R. Collobert, and M. Auli, “wav2vec: Unsupervised pre-training for speech recognition,”

  34. [43]

    The im- pact of allophony versus contrast on speech perception,

    A. Boomershine, K. C. Hall, E. Hume, and K. Johnson, “The im- pact of allophony versus contrast on speech perception,”Contrast in phonology, pp. 143–172, 2008

    2008

  35. [44]

    Leveraging allophony in self-supervised speech models for atyp- ical pronunciation assessment,

    K. Choi, E. Yeo, K. Chang, S. Watanabe, and D. Mortensen, “Leveraging allophony in self-supervised speech models for atyp- ical pronunciation assessment,” inNAACL, 2025

    2025

  36. [45]

    Analysis and evaluation of synthetic data generation in speech dysfluency detection,

    J. Zhang, X. Zhou, J. Lian, S. Li, W. Li, Z. Ezzes, R. Bogley, L. Wauters, Z. Miller, J. V onk, B. Morin, M. Gorno-Tempini, and G. Anumanchipalli, “Analysis and evaluation of synthetic data generation in speech dysfluency detection,” 2025. [Online]. Available: https://arxiv.or...

    2025 arXiv

  37. [46]

    Dysfluent wfst: A framework for zero-shot speech dysfluency transcription and detection,

    C. Guo, J. Lian, X. Zhou, J. Zhang, S. Li, Z. Ye, H. J. Park, A. Das, Z. Ezzes, J. V onk, B. Morin, R. Bogley, L. Wauters, Z. Miller, M. Gorno-Tempini, and G. Anumanchipalli, “Dysfluent wfst: A framework for zero-shot speech dysfluency transcription and detection,” 2025. [Onli...

    2025 arXiv

  38. [2019]

    Available: https://arxiv.org/abs/1904.05862

    [Online]. Available: https://arxiv.org/abs/1904.05862

    1904 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.