Pith. sign in

REVIEW 5 major objections 4 minor 1 cited by

Whisper Has an Internal Word Aligner

T0 review · 5 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper shows that Whisper's decoder contains attention heads that align words to audio, and that a training-free, character-based head filter extracts timestamps more accurately than prior training-based approaches at strict tolerances.

desk verdict A genuinely useful finding — Whisper's decoder has alignment-like attention heads and character teacher forcing sharpens them — but the headline numbers are tuned on test data, so the margin over baselines is not yet nailed down. read the letter →

arxiv 2509.09987 v1 pith:VWINZ6JD submitted 2025-09-12 eess.AS cs.CL

Sung-Lin Yeh , Yen Meng , Hao Tang This is my paper · ORCID
classification eess.AScs.CL
keywords wordalignmentattentionheadsWhisperteacherforcingcharacter-leveltokenizationunsuperviseddynamictimewarpingspeechrecognition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Whisper was trained to transcribe speech, not to timestamp it, yet the paper finds that some of its decoder attention heads produce word alignments that are almost as accurate as those of a dedicated forced aligner. The authors call these oracle heads, show they are visually and quantitatively distinct from other heads, and identify them per utterance without any fixed architectural position. They also show that teacher-forcing Whisper with characters instead of wordpieces produces finer, more monotonic attention maps. Based on those findings, they propose a fully unsupervised pipeline—score each head by the sum of its row and column ℓ2 norms, keep the top ten, average, and run dynamic time warping—that outperforms previous training-based aligners at 20–100 ms tolerances. If true, accurate word timestamps can be obtained from Whisper with no extra training and no external aligner.

What carries the argument

The method rests on cross-attention maps from Whisper's decoder, teacher forcing with character-level tokenizations, and the filtering score s(A) = sum of row-wise ℓ2 norms plus column-wise ℓ2 norms (which acts like a Rényi-2 entropy concentration measure on both axes). The selected maps are averaged, column-normalized, and fed to dynamic time warping (DTW) to derive word boundaries.

What would settle it

On a held-out corpus with manual word boundaries, compute the F1 of the top-10 norm-filtered character heads; if it falls to the level of simply averaging all heads, the filtering claim fails. More sharply, if the correlation between the norm score and per-head F1 disappears on utterances with multiple plausible alignments, the heuristic is not discovering alignment heads but exploiting a confound.

Watch

Extended reading notes

Core claim

The central claim is that, even though Whisper is not trained to produce timestamps, some of its cross-attention heads do encode word-to-audio alignment. The authors call the single best head for an utterance the oracle head and show that oracle heads are close to a standard forced aligner in F1 at 50 ms. They then show that replacing wordpieces with characters under teacher forcing makes the attention maps finer and more monotonic, and that a simple unsupervised score—the sum of row and column ℓ2 norms of the attention map—picks heads that are alignment-like often enough. Keeping the top ten such heads, averaging them, and running DTW yields word timestamps that beat prior training-based ap

Load-bearing premise

The claim rests on the norm score ranking alignment-like heads highly enough that averaging the top ten makes up for the fact that the single best-scoring head is almost never the true oracle head, and on character teacher forcing producing attention maps that remain faithful to alignment even though Whisper was not trained on characters.

Editorial extensions

If this is right

  • Whisper itself can serve as a word aligner, so an external alignment model is unnecessary for timestamped transcription.
  • Characters are a better token granularity than wordpieces for attention-based alignment, even in a model not trained on characters.
  • Per-utterance dynamic head selection outperforms fixed head sets, so alignment heads are not uniformly distributed in the architecture.
  • The method transfers to larger Whisper variants and other encoder-decoder ASR models, with encoder frame rate setting a resolution limit.
  • Strict tolerances (below 100 ms) are the regime where these differences appear; loose 200 ms tolerances hide the advantage.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A possible follow-up is a per-utterance head selector that learns to predict which heads will be oracle-quality, since the paper's own results show the top-1 norm head essentially never matches the oracle; closing that gap would approach the oracle ceiling.
  • Character-level teacher forcing may generalize beyond Whisper: any attention-based encoder-decoder with a coarse subword vocabulary could inherit finer alignment maps by decoding with characters during inference.
  • The oracle heads' distribution shifts when switching from wordpieces to characters, suggesting alignment information is not tied to a fixed architectural location but emerges from the interaction of token granularity and attention dynamics; probing with phonetic labels could test whether these heads track acoustic boundaries directly.
  • Because the filtering score is unsupervised, it could be applied without ground-truth alignments to any sequence-to-sequence ASR decoder; reimplementing it on a non-Whisper model would test whether the phenomenon is Whisper-specific.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper investigates whether Whisper's decoder cross-attention heads implicitly encode word-level alignments. It identifies 'oracle' heads using ground-truth boundaries, showing that they produce alignments close to MFA. It then proposes an unsupervised head-filtering criterion based on the sum of row and column ℓ2 norms (§II-B), and combines it with teacher forcing Whisper with characters instead of wordpieces (§II-A), followed by DTW post-processing. Experiments on TIMIT, LibriSpeech, and AMI report F1 scores at 20–100 ms tolerance, comparing against Whisper's default heads, WhisperX, CrisperWhisper, a gradient-norm baseline, and MFA. The paper claims that this training-free method outperforms prior work under stricter tolerances, and that the findings transfer to Whisper large-v2 and Canary-1B.

Significance. If validated, the central finding—that Whisper contains attention heads that serve as accurate word aligners and can be selected without training—would be practically valuable, enabling training-free word timestamps from a widely used ASR model. The oracle-head analysis and the character-teacher-forcing technique are useful contributions. However, the current evidence is weakened by test-set model selection of the number of heads H, an internal inconsistency between Tables II and III, and small or negative margins on AMI. The 'characters are better' claim also does not hold for Canary-1B oracle heads. The core idea is promising, but the evaluation needs to be made rigorous before the headline claims can be accepted.

major comments (5)
  1. [§IV-D, Table V] The number of retained heads H is selected on test data. §IV-D states: 'we sweep across {5, 10, 15, 20} heads to select and report the best result.' Table V similarly searches H∈{1,5,10,20,all} on TIMIT and AMI. Since H is the only tunable parameter and F1 varies substantially (TIMIT 80.7→75.0 from top-10 to top-20; AMI 61.9→58.6), the reported F1 values reflect test-set model selection, not a purely unsupervised pipeline. Please choose H on a validation split or provide evidence that results are stable across reasonable H.
  2. [Table II vs Table III] There is an internal inconsistency for the same configuration. Table II lists 'Whisper oracle character' on AMI as 75.2 (50 ms) and 83.1 (100 ms). Table III lists 'Whisper oracle heads character' on AMI as 75.2 (50 ms) and 79.8 (100 ms). These should be identical. The discrepancy in the 100 ms value suggests a data-processing error or a typo in at least one table. This undermines confidence in the numerical values.
  3. [§IV-C1, Table VII] The abstract and §IV-C1 claim that 'using characters produces finer and more accurate alignments than using wordpieces.' This is contradicted by the Canary-1B oracle row in Table VII: with oracle heads, wordpieces give 64.8 while characters give 63.8. The paper's statement in §IV-D that the findings 'transfer to Canary-1B' is therefore not fully correct. The claim should be qualified to Whisper models or an explanation should be provided for the reversed result.
  4. [§III-C, Tables III–IV] No confidence intervals, error bars, or significance tests are reported, and the claimed advantage over prior work is not consistent across datasets. On AMI at 50 ms, WhisperX achieves 63.5 vs. the proposed 61.9 (Table III), and the margin over CrisperWhisper is only 65.3 vs. 64.9 (Table IV). Given typical utterance-level variability, these differences may be within noise. The headline 'more accurate than prior work under a stricter tolerance' should be restricted to the datasets and tolerances where the margin is meaningful, or supported with statistical measures.
  5. [§IV-C2, Table V] The paper's own Table V shows that the oracle head is never the top-1 head (0.0%) and is contained in the top-10 heads in only 77.1% (TIMIT) and 82.5% (AMI) of utterances. Thus the method's success relies on averaging many heads, not on reliably selecting the oracle head. This makes the choice of H particularly load-bearing and amplifies the test-set selection concern in the first comment. The authors should analyze sensitivity to H more thoroughly, e.g., via validation curves or aggregated results over H ranges.
minor comments (4)
  1. [§II-A, Eq. (2)] Equation (2) begins with ', a'k = Dec(...)', which appears to be a typo; the dummy variable is not explicitly defined, and the notation y'_{0:k-1} should be clarified.
  2. [§III-C] The evaluation protocol of Strgar and Harwath [43] is referenced but not described. For reproducibility, please specify how true positives are matched, how word identities are handled, and whether multiple matches are allowed.
  3. [§IV-B] The baselines 'Whisper averaging upper half layers' and 'Whisper fixed heads (commit dd985ac)' are not fully specified. Please state which layers are averaged and which fixed heads are used, or provide the exact commit/config.
  4. [Figure 2] The legend uses 'Whisper (averaged)' and 'Whisper (fixed)', which do not match the row labels in Table II. Align the terminology for clarity.

Circularity Check

1 steps flagged · score 4.0 of 10

Core Whisper alignment findings are independent; only the Canary transfer result is selected by test-set sweep.

  1. fitted input called prediction [Section IV-D, Table VII; also Section IV-C2, Table V]
    "For characters, we sweep across{5, 10, 15, 20} heads to select and report the best result."

    The number of attention heads H is the method's only tunable parameter. In the Canary-1B experiment the paper explicitly chooses H by sweeping on the test set and reports the best F1. Since that F1 is measured on the same test set used for selection, the reported Canary result is an argmax over the evaluation labels rather than an unsupervised prediction. For the main Whisper experiments, Table V likewise compares H in {1,5,10,20,all} on TIMIT and AMI, and Tables II/III report the top-10 setting; unless a separate validation split was used, the headline numbers may inherit the same test-set selection. The text does not describe a held-out validation procedure.

full rationale

The paper's central claim—that Whisper contains alignment-like attention heads and that character teacher-forcing plus norm-based head filtering yields competitive word timestamps—is not circular. The oracle-head analysis (Table II) explicitly uses ground truth to define an upper bound; the proposed score s(A) in Eq. (3) is defined without ground truth and is validated by correlation (Figure 6) rather than constructed from the target. The DTW post-processing is standard external methodology. Self-citations [14] and [25] are contextual and not load-bearing. The one genuine circular element is the explicit test-set sweep of the head count H for Canary-1B in Section IV-D: the reported 35.4 F1 is the best over H in {5,10,15,20}, so it is selected by the evaluation labels rather than produced by a fully unsupervised procedure. A similar concern applies to the main Whisper results because Table V sweeps H on the test sets before top-10 is reported, though the text does not state this was used for selection. Since the core method and its Whisper evaluation are independent of this hyperparameter choice, the circularity is partial and localized rather than pervasive.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The method rests on two unproved premises: that Whisper's cross-attention under character teacher forcing is meaningful, and that the ℓ2-norm filtering identifies aligning heads without ground truth. The former is supported only empirically, the latter only partially (Table V). No entities are invented; k is a fitted hyperparameter.

free parameters (1)
  • Number of retained attention heads k = 10 for Whisper; {5,10,15,20} swept for Canary-1B (best reported)
    Retaining top-k heads is central to the method. §IV-D reports sweeping k on the test sets and selecting the best; Table V explores k on TIMIT/AMI. The choice is empirical, not derived.
assumptions (4)
  • domain assumption Whisper's decoder cross-attention is meaningfully aligned to audio even when teacher-forced with character tokens the model was not trained to output.
    §II-A and §IV-C1 rely on character teacher forcing (Table I) to produce finer attention maps. This is an empirical premise validated by F1 scores, not guaranteed by Whisper's training.
  • domain assumption MFA (GMM-HMM) alignments are accurate enough as ground truth proxies on LibriSpeech, AMI, and TIMIT.
    §III-A uses MFA as the reference for LS and AMI; Table III uses MFA-on-TIMIT as a comparative baseline. Any error in MFA bounds the validity of reported F1 scores.
  • domain assumption The ℓ2-norm row+column concentration score identifies alignment-like heads without access to ground truth.
    §II-B proposes Eq. (3); Table V and Fig. 6 provide empirical support, but top-1 hit rate is 0.0%, so the premise is only partially satisfied.
  • standard math Monotonic DTW on averaged attention maps yields correct word boundaries when attention is concentrated.
    §II-C; standard dynamic time warping used in prior work, [21],[22],[34].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Whisper Has an Internal Word Aligner." pith.science (2026). https://pith.science/paper/VWINZ6JD

@misc{pith2026250909987,
  author       = {Pith},
  title        = {Pith review of: Whisper Has an Internal Word Aligner},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VWINZ6JD}},
  note         = {Machine review of arXiv:2509.09987}
}
read the original abstract

There is an increasing interest in obtaining accurate word-level timestamps from strong automatic speech recognizers, in particular Whisper. Existing approaches either require additional training or are simply not competitive. The evaluation in prior work is also relatively loose, typically using a tolerance of more than 200 ms. In this work, we discover attention heads in Whisper that capture accurate word alignments and are distinctively different from those that do not. Moreover, we find that using characters produces finer and more accurate alignments than using wordpieces. Based on these findings, we propose an unsupervised approach to extracting word alignments by filtering attention heads while teacher forcing Whisper with characters. Our approach not only does not require training but also produces word alignments that are more accurate than prior work under a stricter tolerance between 20 ms and 100 ms.

Figures

Figures reproduced from arXiv: 2509.09987 by the authors.

Figure 1
Figure 1. Example attention maps from Whisper. Alignments between the input [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. F1 scores at different tolerance levels. Except for WhisperX, which uses CTC to derive word boundaries, all other approaches are based on cross￾attention maps in Whisper medium. Whisper (averaged) is the approach listed in the third row of Table II B. Comparison to other aligners We present our main results comparing other aligners with the proposed attention filtering (keeping the top 10 heads) and character-level … view at source ↗
Figure 3
Figure 3. The oracle head attention map using wordpieces (top) and characters [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: A comparison of averaging attention heads in the upper half of the [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 6
Figure 6. Figure 6: We observe that heads with higher scores in general [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. REDDIT: Correcting Model-Generated Timestamp Drift in ASR without Forgetting via Replay-Based Distribution Editing

    cs.CL 2026-07 conditional novelty 7.0 of 10

    REDDIT corrects non-speech-induced timestamp drift in autoregressive ASR by editing timestamp targets under cached replay context while anchoring non-timestamp behavior to the frozen base distribution.

Reference graph

Works this paper leans on

44 extracted references · cited by 1 Pith paper

  1. [40]

    Attention is not not explanation,

    Sarah Wiegreffe and Yuval Pinter, “Attention is not not explanation,” in EMNLP, 2019

    2019

  2. [1]

    Joint phoneme alignment and text-informed speech separation on highly corrupted speech,

    Kilian Schulze-Forster, Clement SJ Doire, Ga ¨el Richard, and Roland Badeau, “Joint phoneme alignment and text-informed speech separation on highly corrupted speech,” inICASSP, 2020

    2020

  3. [2]

    Speech-text pre-training for spoken dialog understanding with explicit cross-modal alignment,

    Tianshu Yu, Haoyu Gao, Ting-En Lin, Min Yang, Yuchuan Wu, Wentao Ma, Chao Wang, Fei Huang, and Yongbin Li, “Speech-text pre-training for spoken dialog understanding with explicit cross-modal alignment,” inACL, 2023

    2023

  4. [3]

    Improving automatic forced alignment for dysarthric speech transcription.,

    Yu Ting Yeung, Ka-Ho Wong, and Helen M Meng, “Improving automatic forced alignment for dysarthric speech transcription.,” in INTERSPEECH, 2015

    2015

  5. [4]

    Yolo-stutter: End-to-end region-wise speech dysfluency detection,

    Xuanru Zhou, Anshul Kashyap, Steve Li, Ayati Sharma, Brittany Morin, David Baquirin, Jet V onk, Zoe Ezzes, Zachary Miller, Maria Luisa Gorno Tempini, et al., “Yolo-stutter: End-to-end region-wise speech dysfluency detection,” 2024

    2024

  6. [5]

    Ssdm: Scalable speech dysfluency modeling,

    Jiachen Lian, Xuanru Zhou, Zoe Ezzes, Jet V onk, Brittany Morin, David Paul Baquirin, Zachary Miller, Maria Luisa Gorno Tempini, and Gopala Anumanchipalli, “Ssdm: Scalable speech dysfluency modeling,” NeurIPS, vol. 37, 2024

    2024

  7. [6]

    Auto- matic disfluency detection from untranscribed speech,

    Amrit Romana, Kazuhito Koishida, and Emily Mower Provost, “Auto- matic disfluency detection from untranscribed speech,”TASLP, 2024

    2024

  8. [7]

    Improvements to prosodic alignment for automatic dubbing,

    Yogesh Virkar, Marcello Federico, Robert Enyedi, and Roberto Barra- Chicote, “Improvements to prosodic alignment for automatic dubbing,” inICASSP, 2021

    2021

Show all 44 references
  1. [8]

    End-to-end continuous speech recognition using attention-based recurrent nn: First results,

    Jan Chorowski, Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Ben- gio, “End-to-end continuous speech recognition using attention-based recurrent nn: First results,” inNIPS 2014 Workshop on Deep Learning, December 2014, 2014

    2014

  2. [9]

    Listen, attend and spell: A neural network for large vocabulary conversational speech recognition,

    William Chan, Navdeep Jaitly, Quoc Le, and Oriol Vinyals, “Listen, attend and spell: A neural network for large vocabulary conversational speech recognition,” inICASSP, 2016

    2016

  3. [10]

    State-of-the-art speech recognition with sequence-to-sequence models,

    Chung-Cheng Chiu, Tara N Sainath, Yonghui Wu, Rohit Prabhavalkar, Patrick Nguyen, Zhifeng Chen, Anjuli Kannan, Ron J Weiss, Kanishka Rao, Ekaterina Gonina, et al., “State-of-the-art speech recognition with sequence-to-sequence models,” inICASSP, 2018

    2018

  4. [11]

    Speech-transformer: a no- recurrence sequence-to-sequence model for speech recognition,

    Linhao Dong, Shuang Xu, and Bo Xu, “Speech-transformer: a no- recurrence sequence-to-sequence model for speech recognition,” in ICASSP, 2018

    2018

  5. [12]

    Unsu- pervised word segmentation from speech with attention,

    Pierre Godard, Marcely Zanon-Boito, Lucas Ondel, Alexandre Berard, Franc ¸ois Yvon, Aline Villavicencio, and Laurent Besacier, “Unsu- pervised word segmentation from speech with attention,” inINTER- SPEECH, 2018

    2018

  6. [13]

    Em- pirical evaluation of sequence-to-sequence models for word discovery in low-resource settings,

    Marcely Zanon Boito, Aline Villavicencio, and Laurent Besacier, “Em- pirical evaluation of sequence-to-sequence models for word discovery in low-resource settings,” inINTERSPEECH, 2019

    2019

  7. [14]

    On the difficulty of segmenting words with attention,

    Ramon Sanabria, Hao Tang, and Sharon Goldwater, “On the difficulty of segmenting words with attention,” inProceedings of the Second Workshop on Insights from Negative Results in NLP. 2021, ACL

    2021

  8. [15]

    Ctc in the context of generalized full-sum hmm training,

    Albert Zeyer, Eugen Beck, Ralf Schl ¨uter, and Hermann Ney, “Ctc in the context of generalized full-sum hmm training,” inINTERSPEECH, 2017

    2017

  9. [16]

    Timestamp-aligning and keyword-biasing end- to-end asr front-end for a kws system,

    Gui-Xin Shi, Wei-Qiang Zhang, Guan-Bo Wang, Jing Zhao, Shu-Zhou Chai, and Ze-Yu Zhao, “Timestamp-aligning and keyword-biasing end- to-end asr front-end for a kws system,”EURASIP Journal on Audio, Speech, and Music Processing, 2021

    2021

  10. [17]

    Delay-penalized ctc imple- mented based on finite state transducer,

    Zengwei Yao, Wei Kang, Fangjun Kuang, Liyong Guo, Xiaoyu Yang, Yifan Yang, Long Lin, and Daniel Povey, “Delay-penalized ctc imple- mented based on finite state transducer,”INTERSPEECH, 2023

    2023

  11. [18]

    Less peaky and more accurate ctc forced alignment by label priors,

    Ruizhe Huang, Xiaohui Zhang, Zhaoheng Ni, Li Sun, Moto Hira, Jeff Hwang, Vimal Manohar, Vineel Pratap, Matthew Wiesner, Shinji Watanabe, et al., “Less peaky and more accurate ctc forced alignment by label priors,” inICASSP, 2024

    2024

  12. [19]

    The conformer encoder may reverse the time dimension,

    Robin Schmitt, Albert Zeyer, Mohammad Zeineldeen, Ralf Schl ˝uter, and Hermann Ney, “The conformer encoder may reverse the time dimension,” inICASSP, 2025

    2025

  13. [20]

    whisper-timestamped,

    J ´erˆome Louradour, “whisper-timestamped,” https://github.com/linto-ai/ whisper-timestamped, 2023

    2023

  14. [21]

    CrisperWhis- per: Accurate timestamps on verbatim speech transcriptions,

    Laurin Wagner, Bernhard Thallinger, and Mario Zusag, “CrisperWhis- per: Accurate timestamps on verbatim speech transcriptions,”INTER- SPEECH, 2024

    2024

  15. [22]

    Robust speech recognition via large- scale weak supervision,

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever, “Robust speech recognition via large- scale weak supervision,” inICML, 2023

    2023

  16. [23]

    Montreal forced aligner: Trainable text- speech alignment using kaldi,

    Michael McAuliffe, Michaela Socolof, Sarah Mihuc, Michael Wagner, and Morgan Sonderegger, “Montreal forced aligner: Trainable text- speech alignment using kaldi,” inINTERSPEECH, 2017

    2017

  17. [24]

    Whis- perx: Time-accurate speech transcription of long-form audio,

    Max Bain, Jaesung Huh, Tengda Han, and Andrew Zisserman, “Whis- perx: Time-accurate speech transcription of long-form audio,” in INTERSPEECH, 2023

    2023

  18. [26]

    Neural machine translation of rare words with subword units,

    Rico Sennrich, Barry Haddow, and Alexandra Birch, “Neural machine translation of rare words with subword units,”ACL, 2015

    2015

  19. [27]

    Subword regularization: Improving neural network trans- lation models with multiple subword candidates,

    Taku Kudo, “Subword regularization: Improving neural network trans- lation models with multiple subword candidates,”ACL, 2018

    2018

  20. [28]

    Scheduled sampling for sequence prediction with recurrent neural networks,

    Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer, “Scheduled sampling for sequence prediction with recurrent neural networks,”NeurIPS, 2015

    2015

  21. [29]

    Attention-constrained inference for robust decoder-only text- to-speech,

    Hankun Wang, Chenpeng Du, Yiwei Guo, Shuai Wang, Xie Chen, and Kai Yu, “Attention-constrained inference for robust decoder-only text- to-speech,” inSLT, 2024

    2024

  22. [30]

    Towards better decoding and language model integration in sequence to sequence models,

    Jan Chorowski and Navdeep Jaitly, “Towards better decoding and language model integration in sequence to sequence models,” in INTERSPEECH, 2017

    2017

  23. [31]

    Opennmt: Neural machine transla- tion toolkit,

    Guillaume Klein, Yoon Kim, Yuntian Deng, Vincent Nguyen, Jean Senellart, and Alexander M. Rush, “Opennmt: Neural machine transla- tion toolkit,” 2018

    2018

  24. [32]

    On the usefulness of self-attention for automatic speech recognition with transformers,

    Shucong Zhang, Erfan Loweimi, Peter Bell, and Steve Renals, “On the usefulness of self-attention for automatic speech recognition with transformers,” in2021 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2021, pp. 89–96

    2021

  25. [33]

    Understanding self- attention of self-supervised audio transformers,

    Shu-wen Yang, Andy T Liu, and Hung-yi Lee, “Understanding self- attention of self-supervised audio transformers,”INTERSPEECH, 2020

    2020

  26. [34]

    Glow- TTS: A generative flow for text-to-speech via monotonic alignment search,

    Jaehyeon Kim, Sungwon Kim, Jungil Kong, and Sungroh Yoon, “Glow- TTS: A generative flow for text-to-speech via monotonic alignment search,”NeurIPS, 2020

    2020

  27. [35]

    Unsupervised speech-text word-level alignment with dynamic programming,

    Tianshu Yu, Zihan Gong, Minghuan Tan, Guhong Chen, and Min Yang, “Unsupervised speech-text word-level alignment with dynamic programming,” inFindings of the Association for Computational Linguistics: NAACL 2025, 2025

    2025

  28. [36]

    DARPA TIMIT acoustic-phonetic continous speech corpus cd-rom. nist speech disc 1-1.1,

    John S Garofolo, Lori F Lamel, William M Fisher, Jonathan G Fiscus, and David S Pallett, “DARPA TIMIT acoustic-phonetic continous speech corpus cd-rom. nist speech disc 1-1.1,”NASA STI/Recon technical report n, vol. 93, pp. 27403, 1993

    1993

  29. [37]

    Librispeech: an asr corpus based on public domain audio books,

    Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur, “Librispeech: an asr corpus based on public domain audio books,” in ICASSP, 2015

    2015

  30. [38]

    Recognition and understanding of meetings the AMI and AMIDA projects,

    Steve Renals, Thomas Hain, and Herv ´e Bourlard, “Recognition and understanding of meetings the AMI and AMIDA projects,” inASRU, 2007

    2007

  31. [39]

    The AMI meeting corpus: A pre- announcement,

    Jean Carletta, Simone Ashby, Sebastien Bourban, Mike Flynn, Mael Guillemot, Thomas Hain, Jaroslav Kadlec, Vasilis Karaiskos, Wessel Kraaij, Melissa Kronenthal, et al., “The AMI meeting corpus: A pre- announcement,” inInternational workshop on machine learning for multimodal in...

    2005

  32. [41]

    Tradition or innovation: A comparison of modern asr methods for forced alignment,

    Rotem Rousso, Eyal Cohen, Joseph Keshet, and Eleanor Chodroff, “Tradition or innovation: A comparison of modern asr methods for forced alignment,” inINTERSPEECH, 2024

    2024

  33. [42]

    CTC-segmentation of large corpora for german end- to-end speech recognition,

    Ludwig K ¨urzinger, Dominik Winkelbauer, Lujun Li, Tobias Watzel, and Gerhard Rigoll, “CTC-segmentation of large corpora for german end- to-end speech recognition,” inInternational Conference on Speech and Computer. Springer, 2020

    2020

  34. [43]

    Phoneme segmentation using self- supervised speech models,

    Luke Strgar and David Harwath, “Phoneme segmentation using self- supervised speech models,” inSLT, 2022

    2022

  35. [44]

    Supervised attention in sequence-to- sequence models for speech recognition,

    Gene-Ping Yang and Hao Tang, “Supervised attention in sequence-to- sequence models for speech recognition,” inICASSP, 2022

    2022

  36. [45]

    Less is more: Accurate speech recognition & translation without web-scale data,

    Krishna C Puvvada, Piotr ˙Zelasko, He Huang, Oleksii Hrinchuk, Nithin Rao Koluguri, Kunal Dhawan, Somshubra Majumdar, Elena Rastorgueva, Zhehuai Chen, Vitaly Lavrukhin, et al., “Less is more: Accurate speech recognition & translation without web-scale data,” INTERSPEECH, 2024

    2024

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.