Pith. sign in

REVIEW 4 major objections 7 minor 35 references

RepeaTTS: Towards Feature Discovery through Repeated Fine-Tuning

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read By synthesising thousands of samples with identical inputs and projecting their speech embeddings, a prompt-based TTS model can be made to reveal unlabelled prosodic features—such as emotional intensity—that a second fine-tuning round…

desk verdict A genuinely new iterative feature-discovery loop for prompt TTS, but the evidence that the discovered features are prosodically controllable is thinner than the claims. read the letter →

arxiv 2507.08012 v1 pith:AW2LIRJO submitted 2025-07-05 cs.CL cs.AIcs.SDeess.AS

Atli Sigurgeirsson , Simon King This is my paper
classification cs.CLcs.AIcs.SDeess.AS
keywords text-to-speechprosodycontrolfeaturediscoveryprincipalcomponentanalysisfine-tuningself-supervisedspeechrepresentationsexpressivesynthesiscontrollableTTS
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a way to give prompt-based text-to-speech models control over prosodic features that were never labelled during training. Instead of defining new features by hand, it generates thousands of samples for the same text, speaker, and prompt, then runs PCA on utterance-level speech embeddings to expose the main axes of uncontrolled variation. A human inspects the low-dimensional projection, names the resulting clusters or gradients, and the model is fine-tuned again with these new labels used to re-label the corpus. Applied to a model without emotion labels, the method discovers emotional-intensity and neutral-versus-emotive features that reduce output variance and align with ground-truth emotion labels; applied to a model that already sees emotion labels, it instead surfaces recording-condition variation. The paper's claim is that this pipeline can uncover genuinely controllable prosodic dimensions, with the caveat that the analysis is sensitive to any non-prosodic variation in the speech signal.

What carries the argument

The load-bearing object is the fixed-input analysis set: thousands of synthetic utterances generated with the same target text, speaker label, and description prompt, so that the only free variable is the model's internal sampling of prosodic renditions. Each utterance is condensed to the mean of its Wav2Vec2 layer-4 embeddings, and PCA projects those summaries to three dimensions for human inspection. Discrete speaking styles appear as clusters, continuous variables such as intensity appear as gradients; the human labels them, the corpus is re-labelled by cosine distance to cluster- or bin-mean embeddings, and secondary fine-tuning enrols the new label as a textual prompt. The loop is iterative: after each fine-tuning, the analysis repeats to capture residual variance.

What would settle it

Repeat the PCA analysis on several independently seeded batches of 1,000 fixed-input samples; if the emergent clusters or the ranking of acoustic features along the first principal component shifts across batches, the discovered features are sampling artefacts rather than stable latent prosodic dimensions.

Watch

Extended reading notes

Core claim

For a prompt-based TTS model that has not been told how a speaker should sound emotionally, the paper shows that the model's own uncontrolled output distribution carries learnable prosodic structure. Synthesising roughly a thousand renditions of a fixed sentence for a fixed speaker under a fixed prompt, projecting their mean Wav2Vec2 embeddings with PCA, and manually inspecting the projection reveals a first principal component tracking emotional intensity. Discretising that gradient into three bins, re-labelling the training corpus by cosine distance to bin-mean embeddings, and fine-tuning the model once more reduces the share of variance explained by the first two principal components from 42.7% to an average of 23.7%, and a second iteration separates neutral from emotive renditions with 89.3% of neutral utterances correctly clustered. The paper claims this demonstrates that unlabelled prosodic features can be recovered and enrolled as new control labels through repeated fine-tuning; it also reports that the same method applied to a model already exposed to emotion labels surfaces recording-environment loudness rather than prosody, which the paper takes as evidence of the method's sensitivity to any variation in the signal rather than a refutation of the general idea.

Load-bearing premise

The entire method rests on the assumption that, with text, speaker, and prompt held fixed, the variation in the model's output embeddings is mostly prosodic—so that a PCA projection of those embeddings isolates meaningful, controllable speaking style, rather than recording conditions, speaker leakage, or sampling noise.

Editorial extensions

If this is right

  • The pipeline can be applied to any prompt-based TTS model and any target language; the paper used Icelandic only as a test case.
  • After feature enrolment, the output distribution for a given prompt contracts: the fraction of variance explained by the first two principal components dropped from 42.7% to about 23.7%, and further to a mean of 0.353 after the second iteration.
  • Newly enrolled labels improve separation by ground-truth emotion: neutral versus emotive utterances were classified with 89.3% accuracy for the neutral cluster after the second fine-tuning.
  • The method does not require expert domain knowledge to define new control features; the features emerge from the model's own output statistics and are named by a listener inspecting the projection.
  • The failure on T3-emotion shows the method surfaces whatever accounts for the most variance in the embeddings; when recording conditions dominate, the discovered feature is acoustic environment, not prosody.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the method scales, it suggests a general recipe for 'feature discovery by data generation': any generative model with high-entropy outputs can be interrogated by sampling many outputs for fixed conditions and projecting their representations, turning uncontrolled variance into labelled control dimensions, a principle that might transfer to image diffusion or music generation models.
  • The reliance on a human listener to name the projected axes limits throughput; a natural extension would be to automate the labelling step by correlating PCA axes with interpretable acoustic functionals to propose candidate names, or by using an automatic emotion recognition model to annotate clusters.
  • The cosine-distance re-labelling of the corpus assumes the discovered clusters are stable across speakers and texts; because the analysis is per-speaker and per-text, a testable extension is whether labels learned on one speaker transfer to other speakers or to unseen texts without retraining.
  • The recording-condition correlation found in T3-emotion suggests that the representation is the bottleneck: a different utterance-level encoder that suppresses channel and room information, or that is trained specifically on prosody, might let the method work even when corpus recording conditions vary.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. RepeaTTS proposes a method to discover and enroll new control features in a prompt-based text-to-speech model. The authors generate thousands of samples with fixed text, speaker, and prompt, summarize each utterance with the mean layer-4 Wav2Vec2 embedding, and project these summaries into a 3D PCA space. By manually inspecting the projections, they identify discrete clusters or continuous gradients, which they interpret as unlabelled prosodic features; they then re-label the training corpus by cosine distance to cluster/bin means and fine-tune the model with new textual labels, iterating the process. They test this on two ParlerTTS models fine-tuned on Icelandic: T3 (no emotion labels) and T3-emotion (with emotion labels). For T3, they report a reduction in output variance across fine-tuning stages and improved alignment of the clusters with ground-truth emotion labels; for T3-emotion, the first principal component correlates with recording-condition artifacts rather than prosody, so they stop the analysis. The paper concludes that the method can uncover unlabelled prosodic features in the T3 case.

Significance. The proposed workflow is novel and reasonably presented: it turns the model's own uncontrolled variance into a source of supervision, and the negative T3-emotion result is an honest boundary condition that shows the method can fail when non-prosodic variation dominates. The release of the T3-emotion checkpoint and synthesized samples is a practical strength, and the paper is clearly written overall. However, as presented, the central claim is not yet substantiated: the evidence for T3 rests on variance reduction and on cluster agreement in the same embedding space used to define the labels, with no direct test that a user prompt actually changes the output in the intended way, and no acoustic or perceptual validation that the discovered features are prosodic. The result is therefore best read as an exploratory demonstration with promising but incomplete evidence.

major comments (4)
  1. [4.3 (controllability evaluation)] The manuscript never directly tests whether the newly enrolled labels control the model's output. After fine-tuning with the three intensity labels, the authors generate three analysis sets (one per label) but then analyze each set separately, reporting within-set variance and within-set clustering (Figure 5). They do not compare the three sets against one another; for controllability, the load-bearing test would be a between-condition comparison showing that prompting with label 1 vs. label 3 shifts the output distribution along the target axis, e.g., through embedding distances, F0/energy statistics, or a listening test. Without such a test, the observed variance reduction and cluster purity do not establish that the labels control anything; they only show that each label-conditioned set is internally variable. This is a load-bearing gap for the paper's central claim.
  2. [3.3, 4.3, 4.4] The assumption in Section 3.3 that fixed-input variation in mean Wav2Vec2 embeddings is mostly "differences in prosody" is never validated for the T3 success case. For T3, the paper reports no correlation between the discovered principal component and any acoustic prosodic correlate (F0, duration, energy, etc.), even though GeMaps features were extracted for this purpose (Section 3.2). The only such analysis, for T3-emotion in Section 4.4 (Figure 8), found that the first principal component was explained by recording-condition differences in the corpus, not by prosody. Given that Section 5 acknowledges the method's sensitivity to any variation in the speech signal, the absence of an acoustic check for T3 leaves open the possibility that the discovered "intensity" feature is an artifact like loudness or recording condition rather than a prosodic characteristic. A correlation analysis of the T3 principal component against F0, duration, and energy would directly address this.
  3. [4.3, Table 3] The label-quality evaluation is partly circular and shows only weak separation. The labels are defined as cluster means in Wav2Vec2 space, and Tables 2 and 3 report agreement with ground-truth emotion labels after assigning the same training-corpus embeddings to those cluster means by cosine distance; this measures the internal consistency of the clustering-assignment procedure, not whether the labels control the TTS model. Moreover, Table 3 shows substantial confusion: for assigned label 2, 49.1% of low-intensity and 33.3% of high-intensity utterances are assigned, while for assigned label 3, 38.6% low-intensity and 65.3% high-intensity are assigned. This is not the clear separation implied by the phrase "substantial reduction in the confusion of neutral and non-neutral utterances," and it undermines the use of these clusters as ground-truth-like labels for evaluating the method.
  4. [4.3] The variance statistics used as quantitative evidence are not interpretable as written. The text reports "42.7% of the total variance (0.513)", then "23.7% of the total variance (0.413)", then "a further reduction in mean total variance ... to 0.353". It is unclear what the parenthetical numbers denote (eigenvalue? proportion of unexplained variance? average pairwise cosine distance?), and no confidence intervals or significance tests are provided for the reductions. Since the variance-reduction claim is one of the two quantitative supports for the method's success, the metric needs to be defined precisely and reported with error bars across the three label sets.
minor comments (7)
  1. [Abstract] The phrase "the same inputs yields" should be "the same inputs yield"; also "synthesise" and "synthesized" are used inconsistently throughout the manuscript.
  2. [Figure 2] The axes in Figure 2 are labeled "t" while the text refers to temperature as "τ (tau)"; make the notation consistent.
  3. [Table 1] Table 1 contains spacing artifacts such as "0 .76" and "0 .13" that appear to be formatting errors; these should be cleaned up.
  4. [4.4] The opening sentence "We start our analysis T3-emotion of with" is garbled and should be rewritten.
  5. [5] Section 5 spells "represenations" instead of "representations".
  6. [Index terms] The index term "speech recognition" does not match the paper's content and should be replaced with something like "speech synthesis" or "prosody control".
  7. [Tables 2 and 3] Tables 2 and 3 lack explicit row and column definitions; clarify that rows are ground-truth labels and columns are assigned labels, and give counts or the total N underlying each percentage.

Circularity Check

2 steps flagged · score 5.0 of 10

The T3 controllability evidence is partly circular: labels are defined in Wav2Vec2 embedding space and success is measured in that same space, though ground-truth emotion alignment provides some independent anchor.

  1. self definitional [Section 3.3 (Finding Controllable Features) and Section 4.3 (T3 feature enrolment)]
    "We create mean bin embeddings by taking the mean of 50 random samples from each bin, to re-label the training corpus according to lowest cosine distance. ... After fine-tuning, we generate three analysis sets, corresponding to the three new control labels, comprising 1,000 samples each. These three sets are analysed separately. After enrolling the new features, we find that the first two principal components account for, on average, 23.7% of the total variance (0.413). This suggests that the method has reduced overall variance in the output."

    The control labels are defined as mean Wav2Vec2 embeddings of clusters/bins in the model's own output distribution, and the training corpus is labelled by cosine distance to those means. The post-enrolment evidence of controllability is then measured as variance reduction in PCA of the same Wav2Vec2 summary embeddings. Conditioning on a label that is literally a region of that embedding space and then measuring concentration in that space tests self-consistency: if fine-tuning succeeds, the generated samples will cluster near the target mean embedding. The variance reduction is therefore largely by construction rather than independent evidence that a perceptually or acoustically meaningful prosodic feature was discovered.

  2. fitted input called prediction [Section 4.3, T3 feature enrolment (manual binning and corpus relabelling)]
    "We evenly sample and listen to 30 generated utterances across the first principal component axis. We determine the first principal correlated with the emotional intensity of the rendition, from low to high intensity. We discretise the range into three bins and create appropriate labels for each (1: low intensity, 2: medium intensity, 3: high intensity). We create mean bin embeddings by taking the mean of 50 random samples from each bin, to re-label the training corpus according to lowest cosine distance."

    The 'discovered' intensity feature is inferred from the model's own output distribution through manual listening to 30 samples along the PCA axis, and the corpus labels are assigned by proximity to the mean embeddings of those same generated bins. The subsequent claim that enrolling these labels improves controllability is thus an evaluation of labels fitted to the model's output, not an independently measured prosodic property. No acoustic correlation (F0, duration, energy) is reported for the T3 success case, unlike the T3-emotion failure case where GeMaps correlations are shown. The ground-truth emotion alignment in Tables 2 and 3 is an external anchor not used to fit the clusters, but the low/high intensity confusion remains large, so the independent content is limited.

full rationale

The paper is not wholly circular: the learned labels are compared with ground-truth emotion labels (Tables 2 and 3) that were not used to fit the cluster means, and the T3-emotion negative result (recording-condition artifact, Figure 8) is reported honestly. However, the central T3 claim rests on a definitional loop: features are defined as PCA directions and cluster means of Wav2Vec2 summary embeddings of the model's own outputs, corpus labels are assigned by cosine distance to those means, and post-enrolment controllability is measured as variance reduction in PCA of the same embedding space. This makes the observed variance reduction largely self-consistent rather than independent evidence of prosodic control. The paper also assumes fixed-input variation is prosodic (Section 3.3) and does not report acoustic correlates for the T3 success case, unlike for the failed T3-emotion case; the only perceptual validation is manual listening to 30 samples. These issues make the discovery claim partially circular, but the ground-truth alignment and the honest negative result keep it from being fully forced.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method relies on several unverified assumptions about the representational fidelity of Wav2Vec2 mean embeddings, the validity of PCA projections for manual labeling, and the controllability of features derived from the model's own variance. No new physical or conceptual entities are introduced.

free parameters (4)
  • Temperature tau and k for k-sampling = tau=1.2, k=100
    Chosen from a hyperparameter search trading off diversity against WER and speaker similarity; directly controls the amount of output variation analyzed.
  • Number of PCA components = 3
    Projection to 3D was chosen to make manual feature search tractable, not based on variance explained or a formal criterion.
  • Number of intensity bins and clusters = 3 bins in first T3 stage, 2 clusters in second stage
    The number of bins and clusters was determined by inspecting the PCA projections and manually listening to samples, so it is a post hoc choice.
  • Wav2Vec2 layer for summary embeddings = Layer 4
    Selected based on a citation about early layers being prosodically informative and on 'initial analysis', not on a systematic evaluation.
assumptions (4)
  • domain assumption Fixed inputs (same text, speaker, prompt) isolate variation to prosody.
    Section 3.3: 'Since we limit our analysis to fixed inputs, we assume that most of the observed variation arises from differences in prosody.' This is load-bearing but unproven, and the T3-emotion result shows the assumption can fail.
  • domain assumption Mean Wav2Vec2 embeddings from layer 4 capture perceptually relevant prosodic information.
    The entire feature discovery relies on this embedding choice; no direct validation that the embeddings align with human perception of the target features.
  • domain assumption Cosine distance to cluster or bin means is a valid assignment of labels to training utterances.
    Section 3.3 describes labeling each training utterance by lowest cosine distance to cluster means; this assumes the embedding space metric matches prosodic similarity.
  • domain assumption Fine-tuning on labels derived from the model's own output will make those labels controllable.
    The method assumes that conditioning on the discovered labels during training will allow the model to reproduce the corresponding output styles, which is not directly tested.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RepeaTTS: Towards Feature Discovery through Repeated Fine-Tuning." pith.science (2026). https://pith.science/paper/AW2LIRJO

@misc{pith2026250708012,
  author       = {Pith},
  title        = {Pith review of: RepeaTTS: Towards Feature Discovery through Repeated Fine-Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AW2LIRJO}},
  note         = {Machine review of arXiv:2507.08012}
}
read the original abstract

A Prompt-based Text-To-Speech model allows a user to control different aspects of speech, such as speaking rate and perceived gender, through natural language instruction. Although user-friendly, such approaches are on one hand constrained: control is limited to acoustic features exposed to the model during training, and too flexible on the other: the same inputs yields uncontrollable variation that are reflected in the corpus statistics. We investigate a novel fine-tuning regime to address both of these issues at the same time by exploiting the uncontrollable variance of the model. Through principal component analysis of thousands of synthesised samples, we determine latent features that account for the highest proportion of the output variance and incorporate them as new labels for secondary fine-tuning. We evaluate the proposed methods on two models trained on an expressive Icelandic speech corpus, one with emotional disclosure and one without. In the case of the model without emotional disclosure, the method yields both continuous and discrete features that improve overall controllability of the model.

Figures

Figures reproduced from arXiv: 2507.08012 by the authors.

Figure 1
Figure 1. PCA of summary embeddings for synthetic speech, colored by different target texts. those checkpoints, we find the speaker identity is frequently in￾consistent. We, therefore, further fine-tune the model on an additional list of speakers, which are not included in the original training set. We choose to perform our experiments on a high-quality Icelandic emotive speech corpus, Talromur-3 [19]. We hypoth￾esise that ex… view at source ↗
Figure 3
Figure 3. Speaker similarity of synthesised Ingrid to all real speakers Ingrid Frida Astrid Leif Anders Bjorn Freya Speakers 0.0 0.2 0.4 0.6 0.8 1.0 Mean Similarity 0.82 0.80 0.80 0.80 0.80 0.79 0.79 Emotion Similarity Grouped by Speaker Emotion Threshold (0.75) Happy Sad Angry Surprised Helpful Overall [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 2
Figure 2. An overview of how the choice of temperature and k affects the output distribution 4.2. Baseline Models Before initiating feature enrolment, we prepare two baseline ParlerTTS models fine-tuned on Icelandic speech. We start with T3-emotion: our model that is exposed to emotion and emo￾tional intensity labels during training. We fine-tune the model for a total of 36 epochs. We initially validate the quality of this mo… view at source ↗
Figures from the paper (2 more)
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Correlation of GeMaps-v01b features with princi￾pal component of diverse-ingrid 5. Discussion and Conclusion Our evaluation of T3 shows that the proposed method can un￾cover unlabelled prosodic features in the training corpus by analysing thousands of samples generated…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 25 canonical work pages

  1. [1]

    Given text, TTS models pre- dict a probable speech rendition under the statistics of the train- ing data

    Introduction Modern Text-To-Speech (TTS) models can produce very natural-sounding speech by learning the complex mapping be- tween text and acoustic features. Given text, TTS models pre- dict a probable speech rendition under the statistics of the train- ing data. The model can produce countless renditions for the same inputs, which may or may not be perc...

    work page

  2. [2]

    RepeaTTS: Towards Feature Discovery through Repeated Fine-Tuning

    Background To some degree, prosody control can be achieved through data- labelling and supervised training given those labels. Training acoustic feature predictors jointly with the acoustic model, like the FastSpeech series of models [4, 2], offers fine-grained con- trol over prosodic correlates. However, controlling prosody in this manner is time-consumi...

    work page Pith review arXiv 2025

  3. [3]

    Sections 3.1-3.2 discuss which TTS architecture, training data, and utterance representations we use to support the model analysis

    Method This exploratory study can be split into three main phases: • Model and representation selection: We aim to analyse the output distribution of a prompt-based TTS model to identify features that can be enrolled as additional control inputs in a secondary fine-tuning stage. Sections 3.1-3.2 discuss which TTS architecture, training data, and utterance...

    work page

  4. [4]

    Results 4.1. Initial Hyperparameter tuning Several hyperparameters contribute to the plausible range of the model variation, and during inference, we can control how the model samples from the distribution over possible outputs. Pri- marily, we can do this by configuring the temperature ( τ) and through the choice of k for k-sampling. Increasing tempera- ...

    work page

  5. [5]

    Discussion and Conclusion Our evaluation of T3 shows that the proposed method can un- cover unlabelled prosodic features in the training corpus by analysing thousands of samples generated for the same inputs. We also demonstrate that simple classification, based on PCA projection of utterance-level embeddings, can be used to en- rol these features to the ...

    work page

  6. [6]

    We then re-label the In- grid subset again according to cosine distance to cluster means

    and two embeddings for high and low intensity utterances as before (labels 2 and 3 respectively). We then re-label the In- grid subset again according to cosine distance to cluster means. Separation based on these clusters results in a further reduc- tion in mean total variance, across the two cluster labels, to 0.353. Again we evaluate which ground-truth...

    work page

  7. [7]

    Exploring emotional prototypes in a high dimensional TTS latent space

    P. Van Rijn, S. Mertes, D. Schiller, P. Harrison, P. Larrouy- Maestri, E. André, and N. Jacoby, “Exploring emotional pro- totypes in a high dimensional tts latent space,” arXiv preprint arXiv:2105.01891, 2021

    work page Pith review arXiv 2021

  8. [8]

    Acknowledgements This work was supported in part by Huawei and the UKRI Centre for Doctoral Training in Natural Language Processing, funded by the UKRI (grant EP/S022481/1) and the University of Edinburgh, School of Informatics and School of Philosophy, Psychology & Language Sciences

    work page

Show all 35 references
  1. [9]

    To- wards end-to-end prosody transfer for expressive speech synthesis with tacotron,

    R. Skerry-Ryan, E. Battenberg, Y . Xiao, Y . Wang, D. Stan- ton, J. Shor, R. Weiss, R. Clark, and R. A. Saurous, “To- wards end-to-end prosody transfer for expressive speech synthesis with tacotron,” in international conference on machine learning . PMLR, 2018, pp. 4693–4702

    2018

  2. [10]

    Fastspeech 2: Fast and high-quality end-to-end text to speech,

    Y . Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y . Liu, “Fastspeech 2: Fast and high-quality end-to-end text to speech,” arXiv preprint arXiv:2006.04558, 2020

    2006 arXiv

  3. [11]

    Natural language guidance of high-fidelity text-to-speech with synthetic annotations,

    D. Lyth and S. King, “Natural language guidance of high-fidelity text-to-speech with synthetic annotations,” 2024

    2024

  4. [12]

    The names of those speakers can be included in the description prompt to re- call the chosen speaker identity

    were trained with a list of known speakers. The names of those speakers can be included in the description prompt to re- call the chosen speaker identity. However, without fine-tuning 2https://atlisig.github.io/SSW_supp_material/index.html Layer 1 Layer 2 Layer 3 Layer 4 Layer...

  5. [13]

    Fastspeech: Fast, robust and controllable text to speech,

    Y . Ren, Y . Ruan, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y . Liu, “Fastspeech: Fast, robust and controllable text to speech,” Advances in neural information processing systems, vol. 32, 2019

    2019

  6. [14]

    A human-in-the-loop approach to improving cross-text prosody transfer

    H. Maurya and A. Sigurgeirsson, “A human-in-the-loop approach to improving cross-text prosody transfer.”

  7. [15]

    Using previous acoustic context to improve text-to-speech synthesis,

    P. Oplustil-Gallegos and S. King, “Using previous acoustic context to improve text-to-speech synthesis,” arXiv preprint arXiv:2012.03763, 2020

    2012 arXiv

  8. [16]

    Style tokens: Unsu- pervised style modeling, control and transfer in end-to-end speech synthesis,

    Y . Wang, D. Stanton, Y . Zhang, R.-S. Ryan, E. Battenberg, J. Shor, Y . Xiao, Y . Jia, F. Ren, and R. A. Saurous, “Style tokens: Unsu- pervised style modeling, control and transfer in end-to-end speech synthesis,” in International Conference on Machine Learning . PMLR, 2018, ...

    2018

  9. [17]

    Mellotron: Mul- tispeaker expressive voice synthesis by conditioning on rhythm, pitch and global style tokens,

    R. Valle, J. Li, R. Prenger, and B. Catanzaro, “Mellotron: Mul- tispeaker expressive voice synthesis by conditioning on rhythm, pitch and global style tokens,” in ICASSP 2020-2020 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, ...

    2020

  10. [18]

    Do prosody transfer mod- els transfer prosodyƒ,

    A. T. Sigurgeirsson and S. King, “Do prosody transfer mod- els transfer prosodyƒ,” in ICASSP 2023-2023 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  11. [19]

    Hierarchical generative modeling for controllable speech synthesis,

    W.-N. Hsu, Y . Zhang, R. Weiss, H. Zen, Y . Wu, Y . Cao, and Y . Wang, “Hierarchical generative modeling for controllable speech synthesis,” in International Conference on Learning Representations, 2019. [Online]. Available: https://openreview. net/forum?id=rygkk305YQ

    2019

  12. [20]

    Parler-tts,

    Y . Lacombe, V . Srivastav, and S. Gandhi, “Parler-tts,” https: //github.com/huggingface/parler-tts, 2024

    2024

  13. [21]

    Prompttts: Con- trollable text-to-speech with text descriptions,

    Z. Guo, Y . Leng, Y . Wu, S. Zhao, and X. Tan, “Prompttts: Con- trollable text-to-speech with text descriptions,” arXiv preprint arXiv:2211.12171, 2022

    2022 arXiv

  14. [22]

    Instructtts: Modelling expressive tts in discrete latent space with natural language style prompt,

    D. Yang, S. Liu, R. Huang, G. Lei, C. Weng, H. Meng, and D. Yu, “Instructtts: Modelling expressive tts in discrete latent space with natural language style prompt,” arXiv e-prints, pp. arXiv–2301, 2023

    2023

  15. [23]

    Scaling instruction-finetuned language models,

    H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y . Tay, W. Fedus, E. Li, X. Wang, M. Dehghani, S. Brahma, A. Webson, S. S. Gu, Z. Dai, M. Suzgun, X. Chen, A. Chowdhery, S. Narang, G. Mishra, A. Yu, V . Zhao, Y . Huang, A. Dai, H. Yu, S. Petrov, E. H. Chi, J. Dean, J. Devlin, A. Rob...

    2022 arXiv

  16. [24]

    Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs,

    A. W. Rix, J. G. Beerends, M. P. Hollier, and A. P. Hekstra, “Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs,” in 2001 IEEE international conference on acoustics, speech, and signal processing. Proceed...

    2001

  17. [25]

    Sdr– half-baked or well done?

    J. Le Roux, S. Wisdom, H. Erdogan, and J. R. Hershey, “Sdr– half-baked or well done?” in ICASSP 2019-2019 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 626–630

    2019

  18. [26]

    Cross- domain neural pitch and periodicity estimation,

    M. Morrison, C. Hsieh, N. Pruyne, and B. Pardo, “Cross- domain neural pitch and periodicity estimation,” arXiv preprint arXiv:2301.12258, 2023

    2023 arXiv

  19. [27]

    Talrómur 3 v0.1 (24.09),

    G. T. Örnólfsson, A. Sigurgeirsson, A. B. Nikulásdóttir, and D. Schnell, “Talrómur 3 v0.1 (24.09),” 2024, CLARIN-IS. [Online]. Available: http://hdl.handle.net/20.500.12537/344

    2024

  20. [28]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al. , “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  21. [29]

    Mistral 7b,

    A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. de las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M.-A. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed, “Mistral 7b,”

  22. [31]

    Xls-r: Self- supervised cross-lingual speech representation learning at scale,

    A. Babu, C. Wang, A. Tjandra, K. Lakhotia, Q. Xu, N. Goyal, K. Singh, P. V on Platen, Y . Saraf, J. Pino et al. , “Xls-r: Self- supervised cross-lingual speech representation learning at scale,” arXiv preprint arXiv:2111.09296, 2021

    2021 arXiv

  23. [32]

    wav2vec 2.0: A framework for self-supervised learning of speech repre- sentations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech repre- sentations,” Advances in neural information processing systems , vol. 33, pp. 12 449–12 460, 2020

    2020

  24. [33]

    Opensmile: the mu- nich versatile and fast open-source audio feature extractor,

    F. Eyben, M. Wöllmer, and B. Schuller, “Opensmile: the mu- nich versatile and fast open-source audio feature extractor,” in Proceedings of the 18th ACM international conference on Mul- timedia, 2010, pp. 1459–1462

    2010

  25. [34]

    On the utility of self-supervised mod- els for prosody-related tasks,

    G.-T. Lin, C.-L. Feng, W.-P. Huang, Y . Tseng, T.-H. Lin, C.-A. Li, H.-y. Lee, and N. G. Ward, “On the utility of self-supervised mod- els for prosody-related tasks,” in 2022 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2023, pp. 1104–1111

    2022

  26. [35]

    Generalized end-to-end loss for speaker verification,

    L. Wan, Q. Wang, A. Papir, and I. L. Moreno, “Generalized end-to-end loss for speaker verification,” in 2018 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018, pp. 4879–4883

    2018

  27. [2023]

    Available: https://arxiv.org/abs/2310.06825

    [Online]. Available: https://arxiv.org/abs/2310.06825

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.