Pith. sign in

REVIEW 4 major objections 4 minor 49 references

Grapheme-Coherent Phonemic and Prosodic Annotation of Speech by Implicit and Explicit Grapheme Conditioning

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Grapheme-conditioned annotation lifts grapheme–phoneme match from about 38.5% to 81.5% on JSUT and from 71.4% to 82.2% on a larger corpus, with prosody F1 essentially unchanged.

desk verdict Useful Japanese TTS annotation method, but the phonemic-accuracy claim is undercut by the paper's own PER numbers and a confounded primary metric. read the letter →

arxiv 2506.04527 v1 pith:4D4ZWAXB submitted 2025-06-05 cs.SD cs.CLeess.AS

Hien Ohnaka , Yuma Shirahata , Byeongseon Park , Ryuichi Yamamoto This is my paper
classification cs.SDcs.CLeess.AS
keywords prosodicannotationphonemicgraphemeconditioningtext-to-speechaccentestimationdataaugmentationg2pdictionary
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes an annotation model that predicts phonemic and prosodic labels (TTS labels) for speech while keeping the labels coherent with the corresponding written graphemes. It adds grapheme conditioning through two complementary routes: implicit conditioning, which injects pre-trained BERT text features into the prompt encoder of an encoder-only speech foundation model, and explicit conditioning, which prunes decoding hypotheses that an external g2p dictionary shows to be inconsistent with the grapheme. The authors report that this lifts the grapheme-to-phoneme match rate from about 38.6% to 81.5% on the JSUT dataset and from 71.4% to 82.2% on a larger proprietary corpus, with prosody F1 staying roughly constant; phoneme error rate rises only when the conditioning graphemes come from ASR, which the paper attributes to transcription errors. As an application, the parallel data produced by the method improves textual accent estimation accuracy (AP+AN from 71.18% to 75.44% with LARGE-TTSaug pre-training), making it feasible to build large-scale speech–text–TTS-label corpora for languages with one-to-many grapheme-to-reading mappings, such as Japanese.

What carries the argument

The load-bearing mechanism is grapheme conditioning, realized in two forms. Implicit conditioning: a pre-trained BERT encoder (line-distilbert-base-japanese) is connected through a linear layer to the prompt encoder of OWSM-CTC, an encoder-only speech foundation model; the prompt encoder was originally pre-trained for prefix grapheme conditioning in long-form ASR, so the frozen text features act as a soft prior over which reading and prosody are likely. Explicit conditioning: during decoding, non-blank time steps from the CTC output are searched greedily, and at each position dynamic programming checks partial phoneme hypotheses against entries of an external g2p dictionary (built with mPAligner from 2.19M grapheme-phoneme pairs); only tokens consistent with the grapheme survive, with prosodic labels always treated as matches. Together they implement the factorization $p(y \mid X, g)\,p(g \mid X)$ that replaces the previous independent prediction of labels and graphemes.

What would settle it

Have trained annotators listen to utterances where the written grapheme has multiple valid readings (e.g., the Japanese word "化学", which can be read as "kagaku" or "bakegaku") and judge whether the conditioned model's phonemic label matches the audio rather than merely the text; if the model consistently prefers the text-consistent reading over the spoken one, the coherence gain comes at the cost of acoustic fidelity.

Watch

Extended reading notes

Core claim

The central claim is that TTS-label prediction should be modeled as $p(y \mid X, g)\,p(g \mid X)$ rather than as independent predictions $p(y \mid X)$ and $p(g \mid X)$, so that the predicted phonemic and prosodic labels are tied to the same grapheme sequence that the speech is supposed to realize. The paper implements this with two mechanisms. Implicit conditioning concatenates BERT embeddings of the grapheme with the speech encoder's prompt conditioning, allowing the model to learn to use textual information without requiring large amounts of paired data. Explicit conditioning constrains the CTC-style decoding search: at each step, partial phoneme sequences are checked against an external g2p dictionary via dynamic programming, and only tokens that can match the grapheme (prosodic tokens always count as matching) are kept. In experiments, the two mechanisms together raise the G2P match rate from 38.55% to 81.47% (JSUT-TTSaug) and from 71.38% to 82.20% (LARGE-TTSaug); the ablation shows each contributes independently. Prosody F1 stays within 0.5 points of baseline, while PER increases from 0.21% to 1.12% (JSUT) and 0.15% to 0.93% (LARGE) when the conditioning graphemes come from ASR. Using ground-truth graphemes restores PER to 0.26% and 0.09%, which the authors take as evidence that the PER increase is caused by ASR transcription errors rather than by the conditioning itself. Downstream, pre-training a textual accent estimator on the method's output improves sentence-level AP+AN accuracy from 71.18% to 75.44%.

Load-bearing premise

The paper assumes that the grapheme-to-phoneme match rate is an independent measure of annotation quality, even though the explicit conditioning step enforces that same kind of match with an external dictionary, so the reported improvement mostly reflects the constraint rather than a learned gain in phonemic accuracy.

Editorial extensions

If this is right

  • Large-scale parallel corpora of speech, graphemes, and TTS labels can be produced from unlabeled speech, because the method only needs an ASR transcript as the grapheme condition; the paper demonstrates this on ReazonSpeech's 5,000-hour corpus.
  • Textual accent estimation benefits more from this augmentation than from NLP-only augmentation: LARGE-TTSaug pre-training reaches 75.44% AP+AN, beating NLPaug (74.24%) with roughly half the number of pre-training samples.
  • The two conditioning mechanisms are complementary: implicit conditioning alone gives 73.38% G2P match (JSUT), explicit alone gives 71.80%, and both together give 81.47%, so a learned text prior and a hard dictionary constraint correct different kinds of errors.
  • When ground-truth graphemes are available, as in studio recordings, the method yields phoneme error rates comparable to or better than the unconditioned baseline (0.26% vs 0.21% on JSUT), making it directly usable for clean data annotation.
  • The approach transfers to other languages and tasks that need grapheme-coherent phonemic and prosodic annotation, provided a g2p dictionary with high recall is available.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the explicit conditioning depends on an external g2p dictionary, a natural extension is to learn the phoneme–grapheme alignment from data and drop the dictionary, which would generalize the method to languages or domains without a hand-built lexicon.
  • The paper does not synthesize speech from the conditioned labels, so a listening test comparing TTS trained on conditioned versus unconditioned labels would test whether grapheme coherence actually improves synthesis quality rather than only label consistency.
  • The accent-estimation pre-training keeps only utterances whose labels pass the g2p match filter; varying that filter threshold could map how label quality trades off against corpus size, a choice the paper does not explore.
  • The same conditioning idea could be applied to other one-to-many grapheme problems, such as polyphone disambiguation in Chinese, where the grapheme condition would similarly constrain ambiguous reading selection.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a grapheme-conditioned model for annotating unlabeled speech with phonemic and prosodic TTS labels. The model fine-tunes OWSM-CTC with an implicit conditioning path (BERT features through the prompt encoder) and an explicit decoding constraint that prunes hypotheses inconsistent with the grapheme using an external g2p dictionary. The authors report that the method improves the grapheme-to-phoneme (G2P) match rate relative to an unconditional baseline, with comparable prosody F1, and that the resulting parallel data improve a downstream textual accent estimation model.

Significance. If the central claim holds, the method is practically valuable for scaling up Japanese TTS front-end data by converting large unlabeled speech corpora into speech-grapheme-TTS-label triples. The paper has clear strengths: it builds on a strong open speech foundation model, offers a sensible two-path conditioning design, and grounds the method in a concrete downstream application with positive results. The accent estimation experiment is the most independent evidence in the paper. However, the headline claim that phonemic accuracy is maintained is not established in the main ASR-transcribed operating condition, and the primary G2P-match metric is partly tautological because the explicit conditioning mechanism enforces grapheme consistency by construction.

major comments (4)
  1. [Section 3.3 and Section 4.1.2] The primary evaluation metric, G2P match, is structurally coupled to the explicit conditioning mechanism. Section 3.3 prunes hypotheses inconsistent with the grapheme using an external g2p dictionary, and Section 4.1.2 measures exactly this kind of consistency. The improvement from 38.55% to 81.47% on JSUT-TTSaug in Table 1 therefore reflects the pruning constraint rather than learned phoneme-grapheme coherence. The paper should report the match rate before explicit pruning, compare implicit-only conditioning against the baseline on the same metric, and ideally measure hypothesis quality with a metric not defined in terms of the pruned dictionary.
  2. [Table 1 and Section 4.1.2] The introduction claims the method improves G2P match 'while maintaining the accuracy of phonemic and prosodic labels,' but Table 1 shows PER degrades substantially with grapheme conditioning in the intended ASR-transcribed setting: from 0.21% to 1.12% on JSUT-TTSaug and from 0.15% to 0.93% on LARGE-TTSaug. The authors attribute this to ASR transcription errors, but Table 2 shows that even with ground-truth graphemes, PER on JSUT-TTSaug is 0.26%, which is still worse than the 0.21% baseline without conditioning. The claim that the degradation is 'not a major concern' is an assertion, not a demonstrated result; the paper should either qualify the accuracy claim to the ground-truth-grapheme setting or provide a controlled analysis isolating ASR error effects.
  3. [Tables 1-3] No error bars, confidence intervals, or significance tests are reported for any of the core metrics. Several reported differences, especially Prosody F1 values such as 95.12% versus 94.63% on JSUT-TTSaug, are small and may be within run-to-run variability. Given that the central claims rest on small differences and on a single G2P-match number per condition, the paper needs at least multiple seeds or a paired significance test to support the word 'significantly' in the abstract and Section 4.1.2.
  4. [Section 4.2] The accent estimation experiment is the strongest independent evidence, but it does not by itself validate the phonemic accuracy claim. LARGE-TTSaug achieves the best downstream scores with fewer valid samples than NLPaug, which is encouraging. However, the comparison is confounded by the amount and source of pre-training data, and the paper does not show that the downstream gain persists when the TTS labels are obtained without explicit grapheme pruning. A control condition using the same speech-derived labels without grapheme conditioning would directly test whether the G2P match improvement is what drives the accent estimation gain.
minor comments (4)
  1. [Section 1 and Section 2] There are typos: 'challanges' in the Introduction and 'fomulation' in the Section 2 heading.
  2. [Section 4.1.2] The description of the G2P match metric is compressed: it says five phoneme candidates are generated from graphemes using Mecab, but it is not stated whether these candidates are the same source as the external g2p dictionary used in explicit conditioning, or how ties among multiple matching candidates are handled. Clarifying this would help the reader assess the metric's independence from the conditioning mechanism.
  3. [Table 1] The table's notation 'Graph. cond. (Prop.)' with a checkmark is not explained in the caption; please state explicitly that the checkmark indicates the proposed method with both implicit and explicit conditioning.
  4. [Section 4.2.2] The sentence about using a cleaner TTS-specific dataset in future work is plausible, but the current experiment does not support the claim that ReazonSpeech noise is the reason JSUT-TTSaug underperforms; this should be framed as a hypothesis rather than a conclusion.

Circularity Check

1 steps flagged · score 6.0 of 10

The headline G2P-match gain is partly enforced by the explicit pruning decoder, making the primary evaluation partially tautological; prosody F1 and accent-estimation results provide independent support.

  1. self definitional [Sec. 3.3 (Explicit conditioning) and Sec. 4.1.2 (Experimental evaluation and results), Eq. (6)]
    "To correct these minor errors post hoc and further enhance the matching between graphemes and phonemes, we introduce a decoding strategy with an external g2p dictionary... Finally, the token with the highest score among the matching tokens is retained... the match rate between graphemes and predicted TTS labels (G2P match) was calculated. Five phoneme estimation candidates were generated from graphemes using Mecab, and a match was determined if the phonemic sequence of the TTS label matched any of these candidates."

    The G2P-match metric measures exactly the property that the explicit-conditioning decoder is constrained to satisfy: Eq. (6) restricts decoding to 'the set of hypotheses whose matches are ensured by the external dictionary', and the evaluation counts a phonemic sequence as matching if it equals one of several grapheme-derived candidates. The decoder therefore cannot freely emit grapheme-inconsistent phonemes, so the reported G2P-match improvements (e.g., JSUT-TTSaug 38.55% to 81.47%) are substantially forced by the pruning constraint rather than being an emergent, independently measured prediction. The ablation makes this explicit: explicit conditioning alone, without implicit conditioning, raises G2P match from 38.55% to 71.80% on JSUT-TTSaug.

full rationale

The paper's central G2P-match claim is partially circular because the evaluation metric is grapheme-phoneme consistency and the explicit-conditioning mechanism is a decoder-side constraint that enforces grapheme-phoneme consistency using an external g2p dictionary. The ablation quantifies the enforced component. The remaining claims are not circular: the prosody F1 results are comparable, the accent-estimation application is an independent downstream test, and the self-citations to prior work by the same group are used as baselines and components, not as a uniqueness argument or as the sole support for the main result. The degradation in PER under ASR-transcribed conditions is a correctness weakness rather than a circularity, so it is noted here but not scored as a circular step. Overall, the headline improvement in G2P match is partly tautological while important independent evaluations keep the paper from being fully circular.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new scientific entities or fitted constants; it relies on pre-trained models and an external pronunciation dictionary. The central risk is not free parameters but the self-referential evaluation metric.

assumptions (4)
  • domain assumption OWSM-CTC can serve as a base model for TTS label prediction, and its prompt encoder can carry grapheme information.
    Used as the backbone in Section 3.1; if the prompt encoder does not encode graphemes effectively, implicit conditioning fails.
  • domain assumption Pre-trained Japanese BERT features are a beneficial grapheme representation for conditioning.
    Section 3.2 assumes BERT's text knowledge transfers to the annotation task.
  • domain assumption The external g2p dictionary has high recall, so pruning will remove inconsistent hypotheses without deleting valid ones.
    Section 3.3 explicitly relies on this; the paper argues low precision is acceptable but high recall is required.
  • domain assumption The grapheme-to-phoneme match rate computed from Mecab candidates is a valid proxy for correct phonemes.
    Section 4.1.2 uses this metric; if the metric is not valid, the headline improvement is meaningless.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Grapheme-Coherent Phonemic and Prosodic Annotation of Speech by Implicit and Explicit Grapheme Conditioning." pith.science (2026). https://pith.science/paper/4D4ZWAXB

@misc{pith2026250604527,
  author       = {Pith},
  title        = {Pith review of: Grapheme-Coherent Phonemic and Prosodic Annotation of Speech by Implicit and Explicit Grapheme Conditioning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4D4ZWAXB}},
  note         = {Machine review of arXiv:2506.04527}
}
read the original abstract

We propose a model to obtain phonemic and prosodic labels of speech that are coherent with graphemes. Unlike previous methods that simply fine-tune a pre-trained ASR model with the labels, the proposed model conditions the label generation on corresponding graphemes by two methods: 1) Add implicit grapheme conditioning through prompt encoder using pre-trained BERT features. 2) Explicitly prune the label hypotheses inconsistent with the grapheme during inference. These methods enable obtaining parallel data of speech, the labels, and graphemes, which is applicable to various downstream tasks such as text-to-speech and accent estimation from text. Experiments showed that the proposed method significantly improved the consistency between graphemes and the predicted labels. Further, experiments on accent estimation task confirmed that the created parallel data by the proposed method effectively improve the estimation accuracy.

Figures

Figures reproduced from arXiv: 2506.04527 by the authors.

Figure 2
Figure 2. Concept of explicit conditioning. The matching between phonemes and graphemes is checked phoneme-by￾phoneme using an external g2p dictionary, and the best-scored hypothesis among the matched ones is selected. The figure il￾lustrates an example where the graphemes correspond to “ 化 学 ” (Chemistry). 3. Proposed method 3.1. Model architecture In the proposed method, we adopted OWSM-CTC [19] as the base pre-trained mode… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

49 extracted references · 46 canonical work pages

  1. [15]

    Almost unsupervised text to speech and automatic speech recognition,

    Y . Ren, X. Tan, T. Qinet al., “Almost unsupervised text to speech and automatic speech recognition,” in Proc. of ICML, 2019, pp. 5410–5419

    work page 2019

  2. [1]

    For training high-quality and diverse-styled TTS models, a large amount of text-speech paired data is re- quired [4, 5]

    Introduction The field of text-to-speech (TTS) has advanced significantly through data-driven approaches based on deep neural networks (DNNs) [1, 2, 3]. For training high-quality and diverse-styled TTS models, a large amount of text-speech paired data is re- quired [4, 5]. However, since manually preparing text tran- scriptions for a large amount of unlab...

    work page

  3. [2]

    Grapheme-Coherent Phonemic and Prosodic Annotation of Speech by Implicit and Explicit Grapheme Conditioning

    Problem fomulation The goal of the annotation is to obtain the parallel data of a TTS label sequence y = {ym ∈ Y}M m=1 and a grapheme se- quence g = {gl ∈ G}L l=1, and a speech sample X = {xn ∈ RDin }N n=1 from X. Here, Y and G represent the vocabularies of TTS label and grapheme tokens, respectively. M,L, and N denote the lengths of each sequence. Din de...

    work page Pith review arXiv 2025

  4. [3]

    <blank>

    Proposed method 3.1. Model architecture In the proposed method, we adopted OWSM-CTC [19] as the base pre-trained model. OWSM-CTC is an encoder-only speech foundation model pre-trained on large-scale multilin- gual data, including Japanese, for ASR and speech translation tasks. As shown in Fig. 1, the model consists of a speech embedding layer, a speech en...

    work page

  5. [4]

    Evaluation of proposed annotation model 4.1.1

    Experiments 4.1. Evaluation of proposed annotation model 4.1.1. Experimental setup Datasets. In training the proposed model, we adopted the fol- lowing two dataset conditions, similar to previous work [15]:

    work page

  6. [5]

    Specifically, we utilized the basic5000 subset along with its manually annotated TTS labels1

    For model construction from a limited amount of labeled data, we used the publicly available Japanese speech corpus JSUT [32], which consists of utterances from a single female speaker. Specifically, we utilized the basic5000 subset along with its manually annotated TTS labels1. This subset comprises 5, 000 text samples and 6.78 hours of speech, which was...

    work page

  7. [6]

    ”, (2) Accent change from low to high “[

    We obtained synthetic speech ˆX from the pseudo TTS la- bels to create paired data ( ˆX, ˆy). We used Period VITS [35], configured as in previous study [15]. Data augmentation was applied to the LARGE text dataset. Additionally, to further ex- pand the grapheme domain, we performed similar data augmen- tation using transcribed text from ReazonSpeech [17] ...

    work page

  8. [7]

    Applying this method to downstream tasks be- yond textual accent estimation is a challenge for future work

    Conclusion We proposed the annotation model from speech using grapheme conditioning. Applying this method to downstream tasks be- yond textual accent estimation is a challenge for future work

    work page

Show all 49 references
  1. [8]

    Tacotron: Towards end-to-end speech synthesis,

    Y . Wang, R. J. Skerry-Ryan, D. Stantonet al., “Tacotron: Towards end-to-end speech synthesis,” in Proc. of Interspeech, 2017, pp. 4006–4010

    2017

  2. [9]

    Fastspeech 2: Fast and high-quality end-to-end text to speech,

    Y . Ren, C. Hu, X. Tan et al., “Fastspeech 2: Fast and high-quality end-to-end text to speech,” in Proc. of ICLR, 2021

    2021

  3. [10]

    Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,

    J. Kim, J. Kong, and J. Son, “Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,” in Proc. of ICML, vol. 139, 2021, pp. 5530–5540

    2021

  4. [11]

    Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesiz- ers,

    K. Shen, Z. Ju, X. Tan et al., “Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesiz- ers,” in Proc. of ICLR, 2024

    2024

  5. [12]

    Mega-tts: Zero-shot text-to- speech at scale with intrinsic inductive bias,

    Z. Jiang, Y . Ren, Z. Ye et al. , “Mega-tts: Zero-shot text-to- speech at scale with intrinsic inductive bias,” arXiv preprint arXiv:2306.03509, 2023

    2023 arXiv

  6. [13]

    V oicebox: Text-guided multilin- gual universal speech generation at scale,

    M. Le, A. Vyas, B. Shi et al., “V oicebox: Text-guided multilin- gual universal speech generation at scale,” in Proc. of NeurIPS , 2023

    2023

  7. [14]

    Listening while speaking: Speech chain by deep learning,

    A. Tjandra, S. Sakti, and S. Nakamura, “Listening while speaking: Speech chain by deep learning,” inProc. of ASRU, 2017, pp. 301– 308

    2017

  8. [16]

    Prosodic features con- trol by symbols as input of sequence-to-sequence acoustic mod- eling for neural TTS,

    K. Kurihara, N. Seiyama, and T. Kumano, “Prosodic features con- trol by symbols as input of sequence-to-sequence acoustic mod- eling for neural TTS,” IEICE Transactions on Information and Systems, vol. 104, no. 2, pp. 302–311, 2021

    2021

  9. [17]

    Investigation of enhanced tacotron text-to-speech synthesis systems with self- attention for pitch accent language,

    Y . Yasuda, X. Wang, S. Takaki, and J. Yamagishi, “Investigation of enhanced tacotron text-to-speech synthesis systems with self- attention for pitch accent language,” in Proc. of ICASSP , 2019, pp. 6905–6909

    2019

  10. [18]

    A unified sequence-to-sequence front-end model for mandarin text-to-speech synthesis,

    J. Pan, X. Yin, Z. Zhang et al., “A unified sequence-to-sequence front-end model for mandarin text-to-speech synthesis,” in Proc. of ICASSP, 2020, pp. 6689–6693

    2020

  11. [19]

    A unified accent esti- mation method based on multi-task learning for Japanese text-to- speech,

    B. Park, R. Yamamoto, and K. Tachibana, “A unified accent esti- mation method based on multi-task learning for Japanese text-to- speech,” in Proc. of Interspeech, 2022, pp. 1931–1935

    2022

  12. [20]

    Polyphone disambigua- tion and accent prediction using pre-trained language models in Japanese TTS front-end,

    R. Hida, M. Hamada, C. Kamada et al., “Polyphone disambigua- tion and accent prediction using pre-trained language models in Japanese TTS front-end,” in Proc. of ICASSP , 2022, pp. 7132– 7136

    2022

  13. [21]

    Enhancing Japanese text-to-speech ac- curacy with a novel combination Transformer-BERT-based G2P: Integrating pronunciation dictionaries and accent sandhi,

    K. Kurihara and M. Sano, “Enhancing Japanese text-to-speech ac- curacy with a novel combination Transformer-BERT-based G2P: Integrating pronunciation dictionaries and accent sandhi,” inProc. of Interspeech, 2024, pp. 2790–2794

    2024

  14. [22]

    Audio- conditioned phonemic and prosodic annotation for building text- to-speech models from unlabeled speech data,

    Y . Shirahata, B. Park, R. Yamamoto, and K. Tachibana, “Audio- conditioned phonemic and prosodic annotation for building text- to-speech models from unlabeled speech data,” in Proc. of Inter- speech, 2024, pp. 2795–2799

    2024

  15. [23]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu et al., “Robust speech recognition via large-scale weak supervision,” in Proc. of ICML , 2023, pp. 28 492–28 518

    2023

  16. [24]

    Reazonspeech: A free and massive corpus for Japanese asr,

    Y . Yin, D. Mori, and S. Fujimoto, “Reazonspeech: A free and massive corpus for Japanese asr,” in Proc. of Annual meetings of the Association for Natural Language Processing, 2023

    2023

  17. [25]

    YODAS: Youtube-oriented dataset for audio and speech,

    X. Li, S. Takamichi, T. Saeki et al., “YODAS: Youtube-oriented dataset for audio and speech,” in Proc. of ASRU, 2023, pp. 1–8

    2023

  18. [26]

    OWSM-CTC: An open encoder-only speech foundation model for speech recog- nition, translation, and language identification,

    Y . Peng, Y . Sudo, M. Shakeel, and S. Watanabe, “OWSM-CTC: An open encoder-only speech foundation model for speech recog- nition, translation, and language identification,” in Proc. of ACL, Aug. 2024, pp. 10 192–10 209

    2024

  19. [27]

    BERT: pre- training of deep bidirectional transformers for language under- standing,

    J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: pre- training of deep bidirectional transformers for language under- standing,” arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  20. [28]

    DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter,

    V . Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter,” arXiv preprint arXiv:1910.01108, 2019

    1910 arXiv

  21. [29]

    PnG BERT: Augmented BERT on phonemes and graphemes for neural TTS,

    Y . Jia, H. Zen, J. Shen et al. , “PnG BERT: Augmented BERT on phonemes and graphemes for neural TTS,” in Proc. of Inter- speech, 2021, pp. 151–155

    2021

  22. [30]

    Phoneme-level BERT for enhanced prosody of text-to-speech with grapheme pre- dictions,

    Y . A. Li, C. Han, X. Jiang, and N. Mesgarani, “Phoneme-level BERT for enhanced prosody of text-to-speech with grapheme pre- dictions,” in Proc. of ICASSP, 2023, pp. 1–5

    2023

  23. [31]

    Miipher: A robust speech restoration model integrating self-supervised speech and text rep- resentations,

    Y . Koizumi, H. Zen, S. Karita et al., “Miipher: A robust speech restoration model integrating self-supervised speech and text rep- resentations,” in Proc. of WASPAA, 2023, pp. 1–5

    2023

  24. [32]

    Japanese text-to-speech syn- thesis system: Open JTalk,

    K. Oura, S. Sako, and K. Tokuda, “Japanese text-to-speech syn- thesis system: Open JTalk,” in Proc. ASJ, 2010, pp. 343–344

    2010

  25. [33]

    Release of pre-trained mod- els for the Japanese language,

    K. Sawada, T. Zhao, M. Shing et al., “Release of pre-trained mod- els for the Japanese language,” in Proc. of LREC-COLING, May 2024, pp. 13 898–13 905

    2024

  26. [34]

    Joint ctc-attention based end- to-end speech recognition using multi-task learning,

    S. Kim, T. Hori, and S. Watanabe, “Joint ctc-attention based end- to-end speech recognition using multi-task learning,” in Proc. of ICASSP, 2017, pp. 4835–4839

    2017

  27. [35]

    Relaxing the conditional indepen- dence assumption of ctc-based asr by conditioning on interme- diate predictions,

    J. Nozaki and T. Komatsu, “Relaxing the conditional indepen- dence assumption of ctc-based asr by conditioning on interme- diate predictions,” in Proc. of Interspeech, 2021, pp. 3735–3739

    2021

  28. [36]

    BERT meets CTC: New formulation of end-to-end speech recognition with pre-trained masked language model,

    Y . Higuchi, B. Yan, S. Arora et al. , “BERT meets CTC: New formulation of end-to-end speech recognition with pre-trained masked language model,” in Findings of EMNLP, Dec. 2022, pp. 5486–5503

    2022

  29. [37]

    Improving speech recognition error prediction for modern and off-the-shelf speech recognizers,

    P. Serai, P. Wang, and E. Fosler-Lussier, “Improving speech recognition error prediction for modern and off-the-shelf speech recognizers,” in Proc. of ICASSP, 2019, pp. 7255–7259

    2019

  30. [38]

    The theory of dynamic programming,

    R. Bellman, “The theory of dynamic programming,” Bulletin of the American Mathematical Society, vol. 60, no. 6, pp. 503–515, 1954

    1954

  31. [39]

    JSUT and JVS: Free Japanese voice corpora for accelerating speech synthesis re- search,

    S. Takamichi, R. Sonobe, K. Mitsui et al. , “JSUT and JVS: Free Japanese voice corpora for accelerating speech synthesis re- search,” Acoustical Science and Technology , vol. 41, no. 5, pp. 761–768, 2020

    2020

  32. [40]

    Applying conditional random fields to Japanese morphological analysis,

    T. Kudo, K. Yamamoto, and Y . Matsumoto, “Applying conditional random fields to Japanese morphological analysis,” in Proc. of EMNLP, 2004, pp. 230–237

    2004

  33. [41]

    A proper approach to Japanese morphological analysis: Dictionary, model, and eval- uation

    Y . Den, J. Nakamura, T. Ogiso, and H. Ogura, “A proper approach to Japanese morphological analysis: Dictionary, model, and eval- uation.” in Proc. of LREC, 2008

    2008

  34. [42]

    Period VITS: Varia- tional inference with explicit pitch modeling for end-to-end emo- tional speech synthesis,

    Y . Shirahata, R. Yamamoto, E. Song et al., “Period VITS: Varia- tional inference with explicit pitch modeling for end-to-end emo- tional speech synthesis,” in Proc. of ICASSP, 2023, pp. 1–5

    2023

  35. [43]

    DEMAND: A col- lection of multi-channel recordings of acoustic noise in diverse environments,

    J. Thiemann, I. Nobutaka, and V . Emmanuel, “DEMAND: A col- lection of multi-channel recordings of acoustic noise in diverse environments,” in Proc. of ICA, 2013

    2013

  36. [44]

    The ACE challenge—corpus description and performance evaluation,

    J. Eaton, N. D. Gaubitch, A. H. Moore, and P. A. Naylor, “The ACE challenge—corpus description and performance evaluation,” in Proc. of WASPAA, 2015, pp. 1–5

    2015

  37. [45]

    End-to-end ASR to jointly predict transcriptions and linguistic annotations,

    M. Omachi, Y . Fujita, S. Watanabe, and M. Wiesner, “End-to-end ASR to jointly predict transcriptions and linguistic annotations,” in Proc. of NAACL-HLT, 2021, pp. 1861–1871

    2021

  38. [46]

    Building competitive direct acoustics-to-word models for english conversa- tional speech recognition,

    K. Audhkhasi, B. Kingsbury, B. Ramabhadran et al., “Building competitive direct acoustics-to-word models for english conversa- tional speech recognition,” in Proc. of ICASSP, 2018, pp. 4759– 4763

    2018

  39. [47]

    Joint speech recognition and speaker diarization via sequence transduction,

    L. E. Shafey, H. Soltau, and I. Shafran, “Joint speech recognition and speaker diarization via sequence transduction,” in Proc. of Interspeech, 2019, pp. 396–400

    2019

  40. [48]

    Uncon- strained many-to-many alignment for automatic pronunciation an- notation,

    K. Kubo, H. Kawanami, H. Saruwatari, and K. Shikano, “Uncon- strained many-to-many alignment for automatic pronunciation an- notation,” in Proc. of APSIPA, 2011

    2011

  41. [49]

    Evaluation of many-to-many alignment algorithm by auto- matic pronunciation annotation using web text mining,

    ——, “Evaluation of many-to-many alignment algorithm by auto- matic pronunciation annotation using web text mining,” in Proc. of Interspeech, 2012, pp. 2318–2321

    2012

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.