Pith. sign in

REVIEW 4 major objections 6 minor 43 references

ProsodyFM: Unsupervised Phrasing and Intonation Control for Intelligible Speech Synthesis

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ProsodyFM, a flow-matching TTS model, learns phrase breaks and pitch-shape tokens without prosodic labels and improves phrasing, intonation, and intelligibility.

desk verdict Solid TTS systems paper with a genuine evaluation leak in the objective phrasing metric; the human MOS evidence mostly carries the central claim, but the F1break numbers should not be trusted at face value. read the letter →

arxiv 2412.11795 v2 pith:TCLFXQAR submitted 2024-12-16 cs.CL cs.SDeess.AS

Xiangheng He , Junjie Chen , Zixing Zhang , Björn W. Schuller This is my paper · ORCID
classification cs.CLcs.SDeess.AS
keywords prosodytext-to-speechsynthesisflowmatchingphrasebreakpredictionterminalintonationpitchcontourintelligibilityunsupervisedmodeling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ProsodyFM is a text-to-speech model built on a flow-matching backbone that tries to show that phrasing and terminal intonation can be learned without prosodic labels and that getting them right directly improves intelligibility. Its two supporting ideas are a phrase-break encoder paired with a duration predictor, which locate pauses and set their lengths, and a terminal-intonation encoder that learns a bank of pitch-shape tokens from a processed pitch contour instead of regressing absolute pitch values. On held-out speech, the authors report lower pitch error, lower word error rate, and higher break-detection F1 than four comparison models, and human listeners rated its breaks, terminal intonation, and intelligibility at or above those systems. In out-of-distribution tests on longer sentences and unseen speakers, the model degrades less than its backbone, and a case study demonstrates manual insertion or removal of breaks and continuous control of rising, falling, and level terminal tones.

What carries the argument

Two coupled modules carry the argument. The Phrase Break Encoder identifies phrase-final words, and the Duration Predictor assigns each break a speaker-conditioned duration, so placement and length of pauses are both controllable. The Terminal Intonation Encoder is built around a Pitch Processor that interpolates, smooths, and randomly offsets raw pitch contours to isolate their shape, and a bank of intonation shape tokens; the last-word pitch segment is compressed into a query that attends over the token bank, and the weighted sum becomes the terminal intonation embedding. At inference, an additional attention step aligns the reference speech's intonation patterns to the target text's phrase-final words, which is what lets the model transfer prosody even when the reference transcript differs from the text being synthesized.

What would settle it

Recompute the break F1 metric on the same synthesized utterances using an independently trained or human-annotated break labeler instead of the detector that also produced the training labels; if ProsodyFM's margin over the comparison models shrinks or reverses, the measured phrasing improvement is an artifact of the shared detector.

Watch

Extended reading notes

Core claim

The paper's central claim is that a model which explicitly captures phrase-break locations, break durations, and terminal intonation shapes can make synthesized speech more intelligible than four current systems, even though it is trained with no explicit prosodic labels. The reported evidence is that ProsodyFM obtains the best or tied-best objective results on pitch error, word error rate, and break classification, and the highest human ratings for break similarity and terminal intonation similarity under both matched and mismatched reference-speech settings. The mechanism is unsupervised: speech is not annotated with break or intonation categories; instead, the model learns initial break locations from a frozen detector during training and from text at inference, adjusts break duration with a dedicated predictor, and represents terminal intonation as a weighted combination of learned shape tokens, with the pitch contour preprocessed to emphasize shape over absolute value.

Load-bearing premise

The argument depends on the phrase-break detector being a trustworthy source of break labels, because that same detector is used to create the training labels, to fine-tune the text-only break predictor, and to compute the objective break-quality score.

Editorial extensions

If this is right

  • Longer sentences with complex syntactic structure should be produced with pauses that respect phrase boundaries, reducing misparses and lowering word error rates.
  • End users get direct controls—inserting or deleting a break and changing the slope of a terminal rise or fall—without needing recorded reference prosody or labeled training data.
  • Because the model learns pitch shapes rather than absolute F0, its training should be less sensitive to pitch-tracker errors such as octave jumps and voicing misclassifications.
  • The out-of-distribution result suggests these prosody gains transfer to unseen speakers and sentence lengths, so the intelligibility improvement is not confined to the training distribution.
  • Human ratings of break and intonation similarity track intelligibility ratings, supporting the causal chain from improved phrasing and intonation to improved intelligibility.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An independent break-annotation study would provide a stronger check on the objective phrasing gain, since the detector used to create training labels is also used to score the output.
  • The intonation-shape token bank is a transferable design: it could be attached to other neural vocoders or acoustic models, and the same shape-over-value principle could be applied to prominence and rhythm rather than only terminal intonation.
  • The text-only phrase-break predictor could be used as a standalone front-end for other TTS systems, potentially improving their phrasing without retraining their speech backbones.
  • A natural next test is whether the learned shape tokens are speaker-independent; if they are, they could serve as a shared prosody prior for voice conversion or expressive speech editing.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper presents ProsodyFM, a flow-matching TTS model that augments the MatchaTTS backbone with two prosody-oriented components: a Phrase Break Encoder that obtains break locations from a frozen PSST detector during training and uses a T5-based break predictor at inference, and a Terminal Intonation Encoder that represents last-word pitch shapes through a learnable bank of intonation tokens. The authors claim that ProsodyFM improves phrasing and intonation and thereby intelligibility relative to four SOTA TTS systems, and that it transfers better to unseen long and complex sentences and speakers. Evidence includes objective RMSEf0, WER, and F1break scores (Table 1), subjective MOS judgments (Table 2), an out-of-distribution comparison (Table 3), an ablation study (Table 4), and controllability case studies.

Significance. If the phrasing and intonation improvements are real, ProsodyFM would be a valuable contribution: it is an unsupervised, controllable prosody model with a relatively simple training recipe, and the paper's release of code, model checkpoints, and demo page supports reproducibility. The subjective MOS results, especially MOSbreak and MOSintonation, and the consistency of the ablations give some directional support. However, the objective phrasing evaluation is compromised by using PSST both to create training labels and to score synthesized breaks, which by construction favors ProsodyFM over baselines; the subjective evidence is based on only 15 utterances with labels derived from reference speech. The central claim that ProsodyFM effectively improves phrasing is therefore not yet established at the level claimed.

major comments (4)
  1. [Objective Evaluation Metrics / Experimental Details] The F1break metric in Table 1 is not an independent test of phrasing. The same PSST system is used (i) as the frozen Phrase Break Detector that provides break locations during training (see 'Phrase Break Encoder'), (ii) as the source of ground-truth labels for fine-tuning the T5 Phrase Break Predictor (see 'Experimental Details' and Appendix B), and (iii) as the detector that produces phrase-break labels from both ground-truth and synthesized speech for the F1break computation. The four comparison systems were not trained against PSST labels, so ProsodyFM's higher F1break (62.76 vs 60.08 for MatchaTTS in Table 1) may reflect overfitting to PSST's boundary criteria rather than better phrasing. Please report an independent break evaluation, for example ToBI-style annotation on a held-out set or a second unsupervised break detector whose labels were not used in training, and state whether the F1break advantage survives.
  2. [Subjective Evaluation Metrics] The subjective phrasing and intonation scores are the main non-circular evidence for the headline claim, but the protocol is small and partly assumption-laden. The MOS test uses 15 utterances, labels derived from reference speech, and a non-parallel condition that assumes two sentences with similar semantics and structure share phrase-break and intonation labels (stated in 'Subjective Evaluation Metrics'); Appendix E says the labels were based on pitch contours plus perceptual judgments of two annotators, but no inter-annotator agreement or reliability statistic is reported. Please provide inter-annotator agreement, more test items, and a justification of the label-transfer assumption, or temper the phrasing and intonation claims accordingly.
  3. [Model Generalizability / Table 3] The out-of-distribution claim is supported by a comparison against only MatchaTTS, and its headline metric F1break inherits the PSST label-source leakage. The statement that 'ProsodyFM in the out-of-distribution setting achieves matching performance with the four SOTA models in the in-distribution setting' compares across different test conditions and corpora; Table 3 reports no in-distribution row for the same VCTK-trained models, so this claim is not directly supported. Please add VCTK in-distribution results for both models, or at least clearly present matched in-distribution and out-of-distribution conditions, and use an independent break evaluation.
  4. [Ablation Study / Table 4] The ablations show small objective differences (F1break 62.51 vs 61.25/61.06, WER 2.86% vs 3.13%/3.10%) with no significance tests, and the RMSEf0 differences are described as 'no substantial differences'. The conclusion that both the Phrase Break Encoder and Terminal Intonation Encoder are 'essential' for synthesizing highly intelligible speech is stronger than Table 4 supports. Please report confidence intervals or significance tests and calibrate the conclusion accordingly.
minor comments (6)
  1. [Appendix A] There is a typo in Appendix A: 'During Predictor' should be 'Duration Predictor'.
  2. [Table 2] GenerSpeech has no non-parallel MOS scores; please state this explicitly in the caption or main text rather than leaving the dash unexplained.
  3. [Ablation Study] The name 'w/o into break' is ambiguous; a clearer label such as 'w/o intonation and break' would improve readability.
  4. [Results] The claim that RMSEf0, F1break, and WER 'show a positive correlation' is not supported by a correlation statistic; please either compute it or soften the wording.
  5. [Equation (1)] Equation (1) should define T and state how DTW-mapped voiced frames are selected before the ratio is computed.
  6. [Figure 1] Figure 1 is illustrative; the statement that the proposed smoothing 'alleviates pitch tracking errors' would benefit from a quantitative comparison of the smoothed contour against the other trackers.

Circularity Check

2 steps flagged · score 4.0 of 10

F1break is scored by PSST, the same system that supplies ProsodyFM's break training labels, so the objective phrasing advantage over baselines is partly circular.

  1. fitted input called prediction [Method (Phrase Break Encoder); Experimental Details; Objective Evaluation Metrics]
    "During training, the Phrase Break Encoder uses a pre-trained, frozen Phrase Break Detector to identify phrase breaks from reference speech. ... We use the PSST (Roll, Graham, and Todd 2023) to obtain phrase breaks from the ground truth speech as labels, then apply PSST to the synthesized speech to detect phrase breaks and calculate the F1 score."

    PSST is simultaneously the teacher and the scorer for break placement. ProsodyFM's Phrase Break Encoder is trained to place breaks where the frozen PSST detector marks them, and the T5 Phrase Break Predictor is fine-tuned with PSST outputs as ground truth; the F1break metric then labels ground-truth speech with PSST and detects breaks in synthesized speech with the same PSST. The baselines were not trained against PSST, so any systematic PSST boundary bias (pause thresholds, sensitivity to speaking rate or duration) inflates ProsodyFM's F1break relative to the baselines by construction. The metric therefore does not independently establish improved phrasing; it establishes agreement with the training-label generator.

  2. fitted input called prediction [Appendix B, Table 5]
    "For the Phrase Break Predictor, we fine-tune T5 (Ni et al. 2022) independent from ProsodyFM using LoRA (Hu et al. 2022) with 16 ranks and consider the phrase breaks obtained from the PSST as the ground truth labels when fine-tuning. ... We present its performance on the LibriTTS and VCTK validation sets in Table 5."

    The reported F1, precision, and recall of the Phrase Break Predictor (e.g., 90.00% on LibriTTS) are computed against PSST-derived labels, the exact same source used as the fine-tuning ground truth. This measures how well the predictor replicates PSST, not how accurately it predicts independent human-annotated phrase breaks. Since this predictor supplies breaks during ProsodyFM inference, its benchmark is self-referential and cannot independently validate the 'unsupervised' break-location component.

full rationale

ProsodyFM's central derivation is not self-referential in the sense of a fitted parameter renamed as a prediction: the flow-matching objective, pitch-shape tokens, and inference-time conditioning are stated as independent model components with explicit losses. Nor does the paper rely on self-citation for its main architecture; the PSST detector is an external, released model (Roll, Graham, and Todd 2023). The load-bearing weakness is narrower: the objective phrasing metric F1break is computed with PSST on both ground-truth and synthesized speech, while the same PSST model supplies the break pseudo-labels used to train the Phrase Break Encoder and the T5 Phrase Break Predictor. That makes the F1break comparison against non-PSST-trained baselines partially circular. The paper does, however, provide independent evidence for the central claim: human MOSbreak ratings, WER from Whisper, MOSintelligibility, and an ablation study are all external to PSST. The MOSbreak test uses only 15 utterances and human labels derived from reference speech, so it is weaker than a dedicated independent break-annotation study, but it is not circular. Overall score 4 reflects a real, quotable train/evaluation leakage in the phrasing metric without the whole derivation reducing to its inputs.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests primarily on empirical evaluation rather than a mathematical derivation. The main unpaid-for assumptions are: PSST break annotations are valid ground truth for both training and evaluation; pitch shape is perceptually dominant over absolute pitch; and analogous sentences share prosodic labels. The free parameters are architectural and augmentation choices not derived from theory. No new physical entities are introduced; the intonation shape tokens are a learned codebook with no out-of-paper falsifiable handle.

free parameters (3)
  • Pitch perturbation offset range [fmin, fmax] = Not specified
    The Pitch Processor subtracts a random offset uniformly sampled from [fmin, fmax] from each pitch contour point (Section 'Pitch Processor'). The range is a hand-chosen hyperparameter that determines how aggressively absolute pitch level is discarded; no values or sensitivity analysis are given.
  • Intonation shape token bank size and dimension = 6 tokens, 64 dimensions
    The Terminal Intonation Encoder uses 6 64-D tokens chosen by the authors (Section 'Experimental Details'). The codebook size is a free design choice; no ablation on token count is reported.
  • Reference Encoder architecture = 4 attention heads, 128-D LSTM hidden size
    The reference encoder uses a single-layer LSTM with 128-D hidden size and 4 attention heads instead of the reference encoder in Wang et al. 2018. These hyperparameters are chosen by hand and are not derived from the problem.
assumptions (4)
  • domain assumption Pitch contour shape, rather than absolute pitch values, is the primary cue for human perception of intonational change.
    Motivates the Pitch Processor and the intonation shape token bank; supported by cited perceptual studies (Chodroff and Cole 2019; Cole, Steffman, and Tilsen 2022) but treated as a premise.
  • domain assumption Two sentences with similar semantics and structure should share the same phrasing and intonation labels.
    Stated in 'Subjective Evaluation Metrics' as the basis for the non-parallel MOS test, where labels are transferred from the reference transcript to a modified target text.
  • domain assumption PSST's break annotations are a valid proxy for ground-truth phrase breaks in both training and evaluation.
    Used for phrase break detection in training (frozen checkpoint) and as the F1break measurement on synthesized speech; also serves as pseudo-labels for fine-tuning the T5 predictor in Appendix B.
  • standard math Optimal-transport conditional flow matching and monotonic alignment search from prior work are correct and applicable.
    The training objective relies on conditional flow matching (Lipman et al. 2023) and monotonic alignment search (Kim et al. 2020), which are accepted as background.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ProsodyFM: Unsupervised Phrasing and Intonation Control for Intelligible Speech Synthesis." pith.science (2026). https://pith.science/paper/TCLFXQAR

@misc{pith2026241211795,
  author       = {Pith},
  title        = {Pith review of: ProsodyFM: Unsupervised Phrasing and Intonation Control for Intelligible Speech Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TCLFXQAR}},
  note         = {Machine review of arXiv:2412.11795}
}
read the original abstract

Prosody contains rich information beyond the literal meaning of words, which is crucial for the intelligibility of speech. Current models still fall short in phrasing and intonation; they not only miss or misplace breaks when synthesizing long sentences with complex structures but also produce unnatural intonation. We propose ProsodyFM, a prosody-aware text-to-speech synthesis (TTS) model with a flow-matching (FM) backbone that aims to enhance the phrasing and intonation aspects of prosody. ProsodyFM introduces two key components: a Phrase Break Encoder to capture initial phrase break locations, followed by a Duration Predictor for the flexible adjustment of break durations; and a Terminal Intonation Encoder which learns a bank of intonation shape tokens combined with a novel Pitch Processor for more robust modeling of human-perceived intonation change. ProsodyFM is trained with no explicit prosodic labels and yet can uncover a broad spectrum of break durations and intonation patterns. Experimental results demonstrate that ProsodyFM can effectively improve the phrasing and intonation aspects of prosody, thereby enhancing the overall intelligibility compared to four state-of-the-art (SOTA) models. Out-of-distribution experiments show that this prosody improvement can further bring ProsodyFM superior generalizability for unseen complex sentences and speakers. Our case study intuitively illustrates the powerful and fine-grained controllability of ProsodyFM over phrasing and intonation.

Figures

Figures reproduced from arXiv: 2412.11795 by the authors.

Figure 1
Figure 1. Pitch contours extracted from 5 pitch tracking [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The model architecture of the proposed ProsodyFM during training. The components outlined by the yellow shaded [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The key components of the proposed ProsodyFM in the training (a) and inference (b) phrases. The red markings [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Part of spectrograms and pitch contours (blue line) of a reference speech (1363 [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: The terminal intonation control results for the 7302 [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: The phrase break control results for the 7302 [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: The terminal intonation control results for the 2002 [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: The phrase break control results for the 2002 [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: The instruction page of our Mean Opinion Score human listening test. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: The labeled transcripts provided in the human listening test for all 15 testing samples under parallel and non-parallel [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 37 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    A.; Merritt, T.; Moinet, A.; Karlapati, S.; Muszynska, E.; Slangen, S.; Gatti, E.; and Drugman, T

    Abbas, S. A.; Merritt, T.; Moinet, A.; Karlapati, S.; Muszynska, E.; Slangen, S.; Gatti, E.; and Drugman, T. 2022. Expressive, Variable, and Controllable Duration Modelling in TTS . In 23rd Annual Conference of the International Speech Communication Association, Interspeech 2022, 4546--4550. ISCA

    work page 2022

  4. [4]

    Birkholz, P.; and Zhang, X. 2020. Accounting for microprosody in modeling intonation. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 8099--8103. IEEE

    work page 2020

  5. [5]

    Boersma, P. 2001. Praat, a system for doing phonetics by computer. Glot. Int., 5(9): 341--345

    work page 2001

  6. [6]

    Bolinger, D. 1998. Intonation in American English. Intonation systems: A survey of twenty languages, 45--55

    work page 1998

  7. [7]

    Camacho, A.; and Harris, J. G. 2008. A sawtooth waveform inspired pitch estimator for speech and music. The Journal of the Acoustical Society of America, 124(3): 1638--1652

    work page 2008

  8. [8]

    Cangemi, F. 2015. mausmooth. https://ifl.phil-fak.uni-koeln.de/sites/linguistik/Phonetik/pdf-publications/2015/cangemi2015mausmooth.pdf. Retrievable online

    work page 2015

Show all 43 references
  1. [9]

    D.; Junior, A

    Casanova, E.; Weber, J.; Shulby, C. D.; Junior, A. C.; G \"o lge, E.; and Ponti, M. A. 2022. Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone. In International Conference on Machine Learning, 2709--2720. PMLR

    2022

  2. [10]

    Chodroff, E.; and Cole, J. S. 2019. Testing the Distinctiveness of Intonational Tunes: Evidence from Imitative Productions in American English. In 20th Annual Conference of the International Speech Communication Association, Interspeech 2019, 1966--1970. ISCA

    2019

  3. [11]

    Cole, J.; Steffman, J.; and Tilsen, S. 2022. Shape matters: Machine classification and listeners perceptual discrimination of American English intonational tunes. In Proceedings of the International Conference on Speech Prosody 2022, Proceedings of the International Conference...

    2022

  4. [12]

    Futamata, K.; Park, B.; Yamamoto, R.; and Tachibana, K. 2021. Phrase Break Prediction with Bidirectional Encoder Representations in Japanese Text-to-Speech Synthesis. In 22nd Annual Conference of the International Speech Communication Association, Interspeech 2021, 3126--3130. ISCA

    2021

  5. [13]

    G.; and Dobrovolsky, M

    Grover, C.; Jamieson, D. G.; and Dobrovolsky, M. B. 1987. Intonation in English, French and German: perception and production. Language and Speech, 30(3): 277--295

    1987

  6. [14]

    J.; and de Looze, C

    Hirst, D. J.; and de Looze, C. 2021. Measuring Speech. Fundamental frequency and pitch. Cambridge Handbook of Phonetics, (1): 336--361

    2021

  7. [15]

    J.; Shen, Y.; Wallis, P.; Allen - Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W

    Hu, E. J.; Shen, Y.; Wallis, P.; Allen - Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In The Tenth International Conference on Learning Representations, ICLR 2022 . OpenReview.net

    2022

  8. [16]

    Huang, R.; Ren, Y.; Liu, J.; Cui, C.; and Zhao, Z. 2022. Generspeech: Towards style transfer for generalizable out-of-domain text-to-speech. Advances in Neural Information Processing Systems, 35: 10970--10983

    2022

  9. [17]

    Hwang, J.-S.; Lee, S.-H.; and Lee, S.-W. 2023. PauseSpeech: Natural Speech Synthesis via Pre-trained Language Model and Pause-Based Prosody Modeling. In Asian Conference on Pattern Recognition, 415--427. Springer

    2023

  10. [18]

    Kim, J.; Kim, S.; Kong, J.; and Yoon, S. 2020. Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search. In Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.; and Lin, H., eds., Advances in Neural Information Processing Systems 33: Annual Conference on ...

    2020

  11. [19]

    Kim, J.; Kong, J.; and Son, J. 2021. Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech. In International Conference on Machine Learning, 5530--5540. PMLR

    2021

  12. [20]

    Kong, J.; Kim, J.; and Bae, J. 2020. Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis. Advances in neural information processing systems, 33: 17022--17033

    2020

  13. [21]

    Lee, Y.; and Kim, T. 2019. Robust and fine-grained prosody control of end-to-end speech synthesis. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 5911--5915. IEEE

    2019

  14. [22]

    A.; Han, C.; and Mesgarani, N

    Li, Y. A.; Han, C.; and Mesgarani, N. 2022. Styletts: A style-based generative model for natural and diverse text-to-speech synthesis. arXiv preprint arXiv:2205.15439

    2022 arXiv

  15. [23]

    A.; Han, C.; Raghavan, V.; Mischler, G.; and Mesgarani, N

    Li, Y. A.; Han, C.; Raghavan, V.; Mischler, G.; and Mesgarani, N. 2024. Styletts 2: Towards human-level text-to-speech through style diffusion and adversarial training with large speech language models. Advances in Neural Information Processing Systems, 36

    2024

  16. [24]

    Liberman, M. Y. 1975. The intonational system of English. Ph.D. thesis, Massachusetts Institute of Technology

    1975

  17. [25]

    Lipman, Y.; Chen, R. T. Q.; Ben - Hamu, H.; Nickel, M.; and Le, M. 2023. Flow Matching for Generative Modeling. In The Eleventh International Conference on Learning Representations, ICLR 2023 . OpenReview.net

    2023

  18. [26]

    Mauch, M.; and Dixon, S. 2014. pYIN: A fundamental frequency estimator using probabilistic threshold distributions. In 2014 ieee international conference on acoustics, speech and signal processing (icassp), 659--663. IEEE

    2014

  19. [27]

    McAuliffe, M.; Socolof, M.; Mihuc, S.; Wagner, M.; and Sonderegger, M. 2017. Montreal forced aligner: Trainable text-speech alignment using kaldi. In Interspeech, volume 2017, 498--502

    2017

  20. [28]

    Mehta, S.; Tu, R.; Beskow, J.; Sz \'e kely, \'E .; and Henter, G. E. 2024. Matcha-TTS: A fast TTS architecture with conditional flow matching. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 11341--11345. IEEE

    2024

  21. [29]

    B.; Yang, E.; and Hwang, S

    Min, D.; Lee, D. B.; Yang, E.; and Hwang, S. J. 2021. Meta-stylespeech: Multi-speaker adaptive text-to-speech generation. In International Conference on Machine Learning, 7748--7759. PMLR

    2021

  22. [30]

    Morise, M. 2017. Harvest: A High-Performance Fundamental Frequency Estimator from Speech Signals. In 18th Annual Conference of the International Speech Communication Association, Interspeech 2017, 2321--2325. ISCA

    2017

  23. [31]

    Morise, M.; Kawahara, H.; and Katayose, H. 2009. Fast and reliable f0 estimation method based on the period extraction of vocal fold vibration of singing voice and speech. In Audio Engineering Society Conference: 35th International Conference: Audio for Games. Audio Engineerin...

    2009

  24. [32]

    Ni, J.; Hernandez Abrego, G.; Constant, N.; Ma, J.; Hall, K.; Cer, D.; and Yang, Y. 2022. Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models. In Findings of the Association for Computational Linguistics: ACL 2022, 1864--1874. ACL

    2022

  25. [33]

    W.; Xu, T.; Brockman, G.; McLeavey, C.; and Sutskever, I

    Radford, A.; Kim, J. W.; Xu, T.; Brockman, G.; McLeavey, C.; and Sutskever, I. 2023. Robust speech recognition via large-scale weak supervision. In International conference on machine learning, 28492--28518. PMLR

    2023

  26. [34]

    Ren, Y.; Hu, C.; Tan, X.; Qin, T.; Zhao, S.; Zhao, Z.; and Liu, T. 2021. FastSpeech 2: Fast and High-Quality End-to-End Text to Speech. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.net

    2021

  27. [35]

    Roll, N.; Graham, C.; and Todd, S. 2023. PSST ! Prosodic Speech Segmentation with Transformers. In Jiang, J.; Reitter, D.; and Deng, S., eds., Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL), 476--487. Singapore: Association for Computatio...

    2023

  28. [36]

    E.; Beckman, M

    Silverman, K. E.; Beckman, M. E.; Pitrelli, J. F.; Ostendorf, M.; Wightman, C. W.; Price, P.; Pierrehumbert, J. B.; and Hirschberg, J. 1992. ToBI: A standard for labeling English prosody. In ICSLP, volume 2, 867--870

    1992

  29. [37]

    Taylor, J.; and Richmond, K. 2021. Confidence Intervals for ASR-Based TTS Evaluation. In 22nd Annual Conference of the International Speech Communication Association, Interspeech 2021, 2791--2795. ISCA

    2021

  30. [38]

    Taylor, P. 2009. Text-to-speech synthesis. Cambridge university press

    2009

  31. [39]

    Wang, Y.; Stanton, D.; Zhang, Y.; Ryan, R.-S.; Battenberg, E.; Shor, J.; Xiao, Y.; Jia, Y.; Ren, F.; and Saurous, R. A. 2018. Style tokens: Unsupervised style modeling, control and transfer in end-to-end speech synthesis. In International conference on machine learning, 5180--...

    2018

  32. [40]

    Xu, Y. 2019. Prosody, tone, and intonation. In The Routledge handbook of phonetics, 314--356. Routledge

    2019

  33. [41]

    Yamagishi, J.; Veaux, C.; and MacDonald, K. 2019. CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR Voice Cloning Toolkit (version 0.92). sound

    2019

  34. [42]

    Yang, D.; Koriyama, T.; Saito, Y.; Saeki, T.; Xin, D.; and Saruwatari, H. 2023. Duration-aware pause insertion using pre-trained language model for multi-speaker text-to-speech. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASS...

    2023

  35. [43]

    J.; Jia, Y.; Chen, Z.; and Wu, Y

    Zen, H.; Dang, V.; Clark, R.; Zhang, Y.; Weiss, R. J.; Jia, Y.; Chen, Z.; and Wu, Y. 2019. LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech. In 20th Annual Conference of the International Speech Communication Association, Interspeech 2019, 1526--1530. ISCA

    2019

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.