REVIEW 3 major objections 4 minor 35 references
Graph Connectionist Temporal Classification for Phoneme Recognition
T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read Training a phoneme recognizer with several G2P pronunciations per word, via a graph variant of CTC, cuts phoneme error rate compared with a single-pronunciation CTC baseline.
desk verdict Useful application of existing GTC idea to APR with G2P pronunciation ambiguity; the empirical pattern is consistent, but the English headline gain needs a proper significance test and the upper-bound framing needs cleaning up. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the label graph: a weighted finite-state acceptor in which the candidate pronunciations of each word are placed in parallel and the word-level graphs are joined in sequence, so any path from a start to an end node spells an acceptable phoneme string for the utterance. The GTC loss is the negative log of the total probability of all CTC alignments that collapse to some path in this graph, computed with the forward-backward algorithm. CTC is the special case where the graph contains a single path.
What would settle it
Compute, on the exact TIMIT test and CGN test word lists, the lowest label error rate achievable by cherry-picking among the top candidate pronunciations. If that coverage gap between the best single candidate and the best of the top three is near zero on those test words, the reported PER gains have no source and should not be expected to replicate.
Extended reading notes
Core claim
The paper's claim is that pronunciation ambiguity in G2P output is a usable training signal rather than noise to be averaged away. By compiling multiple pronunciations per word into a label graph and training with GTC, the model is free to weight whichever pronunciation path best explains the acoustics. The authors report lower phoneme error rates than the CTC baseline in both languages, with the best English model using up to two pronunciations per word (28.8% vs 32.9%) and the best Dutch model using up to three (23.0% vs 23.9%). The oracle label error rates in Table I support the mechanism: 1-best G2P output has 11.8% LER on English training words versus 8.5% with the best of three candida
Load-bearing premise
The candidate pronunciation lists for the words in an utterance must, for enough words, actually contain the pronunciation the speaker used; the paper demonstrates this on development data, not on the exact test words used for the headline error rates.
Editorial extensions
If this is right
- APR can be trained from utterance-text pairs with G2P uncertainty, avoiding costly manual phoneme annotations.
- Because GTC reduces to CTC when each word has one pronunciation, one training pipeline covers both unambiguous and ambiguous words.
- The benefit appears already at 2 or 3 candidate pronunciations, so full G2P enumeration is not needed.
- The gap between single-best and oracle multi-best label error rates quantifies how much headroom remains for better pronunciation modeling.
- The method does not model coarticulation across word boundaries; extending the label graph with insertion, substitution, or deletion arcs is the paper's stated next step.
Reading between the lines
- The headline numbers depend on candidate pronunciation coverage on the exact test words; a direct check of oracle coverage on TIMIT test and CGN test would confirm whether the reported gains transfer or are an artifact of dev-set selection.
- The same graph-loss idea generalizes to any sequence task with multiple valid reference transcriptions, such as accented or code-mixed speech, or tasks with inter-annotator disagreement.
- If candidate coverage is the real driver, then languages or dictionaries with richer pronunciation variation should show larger GTC gains; this is testable by comparing G2P systems with different variant counts.
- Adding decoding-side pronunciation variation could compound the training-side gain, but the paper leaves that untested.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes using Graph Temporal Classification (GTC) instead of CTC for training phoneme recognizers when G2P systems provide multiple candidate pronunciations per word. A WFSA-based label graph accepts any concatenation of candidate pronunciations, so all variants are treated as valid supervision. Experiments on US English (Common Voice training, TIMIT test) and Dutch (CGN) compare a 1-best CTC baseline with GTC trained on up to 2, 3, or all pronunciations. The reported PERs are 32.9% -> 28.8% (TIMIT) and 23.9% -> 23.0% (CGN), leading the authors to conclude that incorporating multiple pronunciations consistently improves phoneme error rates.
Significance. If the result holds, the contribution is useful and practical: it adapts an existing sequence-level training objective (GTC) from ASR lattice/hypothesis use to APR with G2P candidate sets, with a clean WFSA formulation and a two-language evaluation. The method is well motivated by oracle label-error rates showing candidate-set gains. The main caveats are statistical: the headline English improvement is not significance-tested against the CTC baseline, the reported p-value interpretation is internally inconsistent, and the oracle-LER upper bounds are computed on different splits from the test PERs. These issues affect the strength of the central empirical claim rather than the validity of the method, so the paper warrants major revision. The authors provide a detailed algorithm description and implementation choices, which is a strength for reproducibility.
major comments (3)
- [§VI-B, Table II] The central claim 'consistently improves' over CTC rests on 32.9 vs 28.8 (TIMIT) and 23.9 vs 23.0 (CGN). For English, no Wilcoxon signed-rank test is reported for the 1-best vs 2-best contrast; the only reported English p-values are between GTC variants, and the text says these differences are 'not statistically significant' while listing p=0.04, which is significant at the conventional 0.05 level. For Dutch, the best 3-best system is not directly tested against 1-best (the p=1e-5 compares 1-best vs 2-best). Please report per-speaker paired tests for each 1-best vs k-best contrast, state the alpha/multiple-comparison policy, and correct the p-value interpretation.
- [§VI-A, Table I] The oracle LER upper bounds are measured on TIMIT train and CGN-phoneme dev, whereas the PER gains in Table II are measured on TIMIT test and CGN-phoneme test. The claim that GTC's benefit is bounded by candidate-set quality is only as strong as the transferability of these LERs to the test splits, which is assumed rather than demonstrated. Compute oracle LER on the same test sets used in Table II (or provide explicit evidence that train/dev LERs are representative of test words).
- [§VI-B] The number of pronunciations used in the headline result (2 for English, 3 for Dutch) appears to be selected after inspecting test-set PERs, and the conclusion highlights the best configuration. Because multiple configurations are evaluated and only the best is featured, the reported gain is a selected maximum; no multiple-testing correction or validation-based selection is described. At minimum, the authors should state whether the choice was made on a separate validation set or treat the results as exploratory.
minor comments (4)
- [§III-B, Eq. (4)] The definition of β^{-1}(G) as 'the set of sequences π such that β(π)=ŷ_k, and ŷ_k ∈ G' should be written as a union over all label sequences ŷ in G: β^{-1}(G) = ∪_{ŷ∈G} β^{-1}(ŷ). The current wording is imprecise.
- [Figure 1] The caption contains a dangling construction: 'The arc with label −1 pointing to the final state of a design requirement for all k2 WFSAs.' This should be rewritten for clarity.
- [§V-C] The model-selection criterion is described as 'the minimum loss value on the validation set,' but it is unclear whether this is the CTC/GTC loss on the pseudo-labels or a phoneme-level error metric. Please specify.
- [§VI-B] The sentence 'the differences between the 2-best, 3-best, and all systems are not statistically significant (p=0.04 between 2-best and 3-best, p=0.11 between 2-best and all)' is internally inconsistent with the usual interpretation of p=0.04. Clarify the significance level and correction method.
Circularity Check
No significant circularity: GTC training uses external G2P candidate lists and is evaluated on held-out manual phoneme labels.
full rationale
The paper's central derivation is the GTC loss (Eq. 4), which generalizes CTC by summing over all label sequences encoded in a graph G. The graph is constructed (Algorithm 1) from G2P-derived pronunciation candidates per word, using an external English G2P trained on CMUDict and the rule-based Dutch Fonilex dictionary; no manual phoneme labels enter the training loss. The oracle LER analysis (Table I) is a diagnostic computed on TIMIT train and CGN dev, not a fitted parameter inserted into the loss, and it is not offered as the headline result. The reported PERs in Table II are evaluated on TIMIT test and CGN test against manual phoneme annotations, so the improvement claim is an external empirical comparison rather than a consequence of the loss definition. The choice of n (1-best, 2-best, 3-best, all) is an experimental condition; the lack of a direct significance test for the English 1-best vs 2-best contrast and the internally inconsistent p-value wording are statistical-reporting issues, not circularity. The paper cites prior work [9] for the GTC formulation, but this is a genuine external reference to Moritz et al., not a self-citation, and the implementation details are described in the paper. No equation is defined in terms of the result, no fitted parameter is renamed as a prediction, and no self-citation chain is load-bearing. Thus the derivation chain is self-contained and not circular.
Assumptions & free parameters
free parameters (3)
- English G2P pruning threshold =
3.0
- Number of pronunciations per word n =
2 (English), 3 (Dutch), or all
- Dutch Fonilex variant selection order =
normal, low, high; random entry if multiple
assumptions (4)
- domain assumption Conditional independence assumption of CTC/GTC: P(pi_t | pi_(1:t-1), X) = P(pi_t | X)
- domain assumption G2P candidate sets cover the true pronunciations often enough
- domain assumption No system can beat the pseudo-label LER
- standard math GTC and WFSA implementations are correct as described in [9] and [25]
Cite this review
Pith. "Pith review of Graph Connectionist Temporal Classification for Phoneme Recognition." pith.science (2026). https://pith.science/paper/CMYWXMMT
@misc{pith2026250905399,
author = {Pith},
title = {Pith review of: Graph Connectionist Temporal Classification for Phoneme Recognition},
year = {2026},
howpublished = {\url{https://pith.science/paper/CMYWXMMT}},
note = {Machine review of arXiv:2509.05399}
}
read the original abstract
Automatic Phoneme Recognition (APR) systems are often trained using pseudo phoneme-level annotations generated from text through Grapheme-to-Phoneme (G2P) systems. These G2P systems frequently output multiple possible pronunciations per word, but the standard Connectionist Temporal Classification (CTC) loss cannot account for such ambiguity during training. In this work, we adapt Graph Temporal Classification (GTC) to the APR setting. GTC enables training from a graph of alternative phoneme sequences, allowing the model to consider multiple pronunciations per word as valid supervision. Our experiments on English and Dutch data sets show that incorporating multiple pronunciations per word into the training loss consistently improves phoneme error rates compared to a baseline trained with CTC. These results suggest that integrating pronunciation variation into the loss function is a promising strategy for training APR systems from noisy G2P-based supervision.
Figures
Reference graph
Works this paper leans on
-
[1]
The nature of phonetic transcription,
C. Cucchiarini and H. Strik, “The nature of phonetic transcription,” inManual of Clinical Phonetics, M. J. Ball, Ed. London: Routledge, 2021, pp. 141–149. [Online]. Available: https: //www.taylorfrancis.com/chapters/edit/10.4324/9780429320903-11/ nature-phonetic-transcription-catia-cucchiarini-helmer-strik
-
[2]
ASR-based corrective feed- back on pronunciation: does it really work?,
A. Neri, C. Cucchiarini, and H. Strik, “ASR-based corrective feed- back on pronunciation: does it really work?,” inProc. Interspeech 2006, Pittsburgh, PA, USA, 2006, paper 1372-Wed3A3O.2. doi: 10.21437/Interspeech.2006-543
-
[3]
Discriminative pronunciation modeling for dialectal speech recognition,
M. Lehr, K. Gorman, and I. Shafran, “Discriminative pronunciation modeling for dialectal speech recognition,” inProc. Interspeech, Singa- pore, 2014, pp. 1458–1462. doi: 10.21437/Interspeech.2014-356. [On- line]. Available: https://www.isca-archive.org/interspeech 2014/lehr14 interspeech.html
-
[4]
S. White, A. Hurren, S. James, and R.-A. Knight, “I think that’s what I heard? I’m not sure: Speech and language therapists’ views of, and practices in, phonetic transcription,”Int. J. Lang. Commun. Disord., vol. 57, no. 5, pp. 1071–1084, 2022. [Online]. Available: https://doi.org/10. 1111/1460-6984.12740
-
[5]
Montreal Forced Aligner: Trainable Text-Speech Alignment Using Kaldi,
M. McAuliffe, M. Socolof, S. Mihuc, M. Wagner, and M. Son- deregger, “Montreal Forced Aligner: Trainable Text-Speech Alignment Using Kaldi,” inProc. Interspeech, Stockholm, Sweden, Aug. 2017, pp. 498–502. [Online]. Available: https://montrealcorpustools.github.io/ Montreal-Forced-Aligner/images/MFA paper Interspeech2017.pdf
work page 2017
-
[6]
The CMU Pronouncing Dictionary,
R. L. Weide, “The CMU Pronouncing Dictionary,” Carnegie Mellon University, 1993. [Online]. Available: http://www.speech.cs.cmu.edu/ cgi-bin/cmudict
work page 1993
- [7]
-
[8]
A. Graves, S. Fern ´andez, F. Gomez, and J. Schmidhuber, “Connection- ist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” inProc. 23rd Int. Conf. Mach. Learn. (ICML), Pittsburgh, PA, USA, 2006, pp. 369–376. [Online]. Available: https://www.cs.toronto.edu/∼graves/icml 2006.pdf
work page 2006
Show all 35 references
-
[9]
Semi-supervised speech recognition via graph-based temporal classification,
N. Moritz, T. Hori, and J. Le Roux, “Semi-supervised speech recognition via graph-based temporal classification,” inProc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Toronto, ON, Canada, June 2021, pp. 6548–6552
2021
-
[10]
Star Temporal Classification: Sequence Modeling with Partially Labeled Data,
V . Pratap, A. Hannun, G. Synnaeve, and R. Collobert, “Star Temporal Classification: Sequence Modeling with Partially Labeled Data,” inAdvances in Neural Information Processing Systems 35 (NeurIPS 2022), 2022, pp. 13392–13403. [Online]. Available: https://papers.neurips.cc/pap...
2022
-
[11]
W-CTC: a Connectionist Temporal Classification Loss with Wild Cards,
X. Cai, J. Yuan, Y . Bian, G. Xun, J. Huang, and K. Church, “W-CTC: a Connectionist Temporal Classification Loss with Wild Cards,” inProc. Int. Conf. Learn. Represent. (ICLR), 2022. [Online]. Available: https: //openreview.net/forum?id=0RqDp8FCW5Z
2022
-
[12]
Bypass Temporal Classification: Weakly Supervised Auto- matic Speech Recognition with Imperfect Transcripts,
D. Gao, M. Wiesner, H. Xu, L. P. Garcia, D. Povey, and S. Khu- danpur, “Bypass Temporal Classification: Weakly Supervised Auto- matic Speech Recognition with Imperfect Transcripts,” inProc. INTER- SPEECH 2023, Incheon, South Korea, Aug. 2023, pp. 924–928. doi: 10.21437/Intersp...
2023 doi
-
[13]
Alterna- tive pseudo-labeling for semi-supervised automatic speech recognition,
H. Zhu, D. Gao, G. Cheng, D. Povey, P. Zhang, and Y . Yan, “Alterna- tive pseudo-labeling for semi-supervised automatic speech recognition,” IEEE/ACM Trans. Audio Speech Lang. Process., vol. 31, pp. 3320–3330,
-
[14]
Learning from flawed data: weakly supervised automatic speech recognition,
D. Gao, H. Xu, D. Raj, L. P. G. Perera, D. Povey, and S. Khu- danpur, “Learning from flawed data: weakly supervised automatic speech recognition,” inProc. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Taipei, Taiwan, Dec. 2023. doi: 10.1109/ASRU57964.202...
2023
-
[15]
Multiple-hypothesis CTC-based semi-supervised adaptation of end-to-end speech recognition,
C.-T. Do, R. Doddipatla, and T. Hain, “Multiple-hypothesis CTC-based semi-supervised adaptation of end-to-end speech recognition,” inProc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Toronto, ON, Canada, June 2021, pp. 6978–6982. [Online]. Available: https:// ie...
2021
-
[16]
An overview of the SPHINX speech recognition system,
K.-F. Lee, H.-W. Hon, and R. Reddy, “An overview of the SPHINX speech recognition system,”IEEE Trans. Acoust., Speech, Signal Pro- cess., vol. 38, no. 1, pp. 35–45, Jan. 1990
1990
-
[17]
Purely sequence-trained neural networks for ASR based on lattice-free MMI,
D. Povey, V . Peddinti, D. Galvez, P. Ghahremani, V . Manohar, X. Na, Y . Wang, and S. Khudanpur, “Purely sequence-trained neural networks for ASR based on lattice-free MMI,” inProc. Interspeech, San Francisco, CA, USA, Sep. 2016, pp. 2751–2755. doi: 10.21437/Interspeech.2016- 595
2016 doi
-
[18]
wav2vec 2.0: A framework for self-supervised learning of speech representations,
A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 33, 2020, pp. 12449–12460
2020
-
[19]
Unsupervised speech recognition,
A. Baevski, W.-N. Hsu, A. Conneau, and M. Auli, “Unsupervised speech recognition,” inAdvances in Neural Information Processing Systems, vol. 34, 2021, pp. 27826–27839. [Online]. Available: https://openreview. net/forum?id=QmxFsofRvW9
2021
-
[20]
Universal phone recognition with a multilingual allophone system,
X. Li, S. Dalmia, J. Li, M. Lee, P. Littell, J. Yao, A. Anastasopoulos, D. R. Mortensen, G. Neubig, A. W. Black, and F. Metze, “Universal phone recognition with a multilingual allophone system,” inProc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), May 2020, pp. 8249–
2020
-
[21]
Differentiable allophone graphs for language-universal speech recog- nition,
B. Yan, S. Dalmia, D. R. Mortensen, F. Metze, and S. Watanabe, “Differentiable allophone graphs for language-universal speech recog- nition,” inProc. Interspeech, Brno, Czech Republic, Aug. 2021, pp. 2471–2475. [Online]. Available: https://www.isca-speech.org/archive/ interspe...
2021
-
[22]
Epitran: Precision G2P for Many Languages,
D. R. Mortensen, S. Dalmia, and P. Littell, “Epitran: Precision G2P for Many Languages,” inProc. Eleventh Int. Conf. Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, May 2018. [Online]. Available: https://aclanthology.org/L18-1429/
2018
-
[23]
In search of better pronunciation models for speech recognition,
N. Cremelie and J.-P. Martens, “In search of better pronunciation models for speech recognition,”Speech Communication, vol. 29, no. 2–4, pp. 115–136, Nov. 1999
1999
-
[24]
Weighted finite-state transducers in speech recognition,
M. Mohri, F. Pereira, and M. Riley, “Weighted finite-state transducers in speech recognition,”Computer Speech & Language, vol. 16, no. 1, pp. 69–88, Jan. 2002. [Online]. Available: https://www.sciencedirect.com/ science/article/pii/S0885230801901846
2002
-
[25]
Fast and parallel decoding for transducer,
W. Kang, L. Guo, F. Kuang, L. Lin, M. Luo, Z. Yao, X. Yang, P. ˙Zelasko, and D. Povey, “Fast and parallel decoding for transducer,” inProc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Rhodes Island, Greece, Jun. 2023, pp. 1–5, doi: 10.1109/ICASSP49357.2023.10094567
2023
-
[26]
Common V oice: A Massively-Multilingual Speech Corpus,
R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber, “Common V oice: A Massively-Multilingual Speech Corpus,” inProc. 12th Lan- guage Resources and Evaluation Conf. (LREC), Marseille, France, May 2020, pp. 4218–4...
2020
-
[27]
DARPA TIMIT Acoustic-Phonetic Continuous Speech Corpus CD-ROM,
J. S. Garofolo, L. F. Lamel, W. M. Fisher, J. G. Fiscus, D. S. Pallett, and N. L. Dahlgren, “DARPA TIMIT Acoustic-Phonetic Continuous Speech Corpus CD-ROM,” NIST Speech Disc 1-1.1, NIST Interagency or Internal Report (NISTIR) 4930, February 1993
1993
-
[28]
CGN, an annotated corpus of spoken Dutch,
I. Schuurman, M. Schouppe, H. Hoekstra, and T. van der Wouden, “CGN, an annotated corpus of spoken Dutch,” inProceedings of the 4th International Workshop on Linguistically Interpreted Corpora (LINC-03) at EACL 2003, Budapest, 2003. [Online]. Available: https: //aclanthology.o...
2003
-
[29]
Joint-sequence models for grapheme-to- phoneme conversion,
M. Bisani and H. Ney, “Joint-sequence models for grapheme-to- phoneme conversion,”Speech Communication, vol. 50, no. 5, pp. 434– 451, May 2008. [Online]. Available: https://doi.org/10.1016/j.specom. 2008.01.002
2008 doi
-
[30]
Conformer: Convolution- augmented Transformer for Speech Recognition,
A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu, and R. Pang, “Conformer: Convolution- augmented Transformer for Speech Recognition,” inProc. Interspeech, Shanghai, China, Oct. 2020, pp. 5036–5040. [Online]. Available: https: //www...
2020
-
[31]
ESPnet: End-to-End Speech Processing Toolkit,
S. Watanabe, T. Hori, S. Karita, T. Hayashi, J. Nishitoba, Y . Unno, N. E. Yalta Soplin, J. Heymann, M. Wiesner, N. Chen, A. Renduchintala, and T. Ochiai, “ESPnet: End-to-End Speech Processing Toolkit,” inProc. Interspeech, Hyderabad, India, Sep. 2018, pp. 2207–2211. [Online]....
2018 doi
-
[32]
Adam: A Method for Stochastic Optimization,
D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” inProc. 3rd Int. Conf. Learn. Representations (ICLR), San Diego, CA, USA, May 2015. [Online]. Available: https://arxiv.org/abs/1412.6980
2015 arXiv
-
[1998]
Available: https://lirias.kuleuven.be/bitstream/handle/ 123456789/389231/Mertens 1997 Fonilex manual.pdf?sequence=1
[Online]. Available: https://lirias.kuleuven.be/bitstream/handle/ 123456789/389231/Mertens 1997 Fonilex manual.pdf?sequence=1
1997
-
[2023]
Available: https://doi.org/10.1109/TASLP.2023.3306709
[Online]. Available: https://doi.org/10.1109/TASLP.2023.3306709
2023
-
[8253]
Available: https://par.nsf.gov/servlets/purl/10175774
[Online]. Available: https://par.nsf.gov/servlets/purl/10175774
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.