Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

Prompt-Unseen-Emotion: Zero-shot Expressive Speech Synthesis with Prompt-LLM Contextual Knowledge for Mixed Emotions

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A text-to-speech model trained only on single-emotion clips can synthesize never-seen mixed emotions by prompting emotion proportions.

desk verdict A testable idea with promising AB results, but the quantitative emotion-control claim needs better evidence before it can be accepted. read the letter →

arxiv 2506.02742 v1 pith:ZNQO2XLD submitted 2025-06-03 eess.AS cs.AIcs.SDeess.SP

Xiaoxue Gao , Huayun Zhang , Nancy F. Chen This is my paper
classification eess.AScs.AIcs.SDeess.SP
keywords expressivetext-to-speechzero-shotemotionsynthesismixedemotionsemotion-guidedpromptlearningLLM-TTSproportioncontrolconditioning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Prompt-Unseen-Emotion (PUE) claims that a text-to-speech model trained only on single-emotion utterances can, at inference time, synthesize emotionally mixed speech for emotion combinations it never saw in training. The trick is to rewrite the emotion label as a natural-language prompt carrying five percentage values, for example '100% surprise' or 'surprise with 90% anger,' and to hand those prompts to an LLM-based text-to-speech system during training. Because the language model sees the proportion words alongside the acoustic tokens, the paper argues, it learns a quantitative mapping from emotion weightings to speech prosody. At inference the user simply edits the percentages, and the model produces blends such as delight, outrage, and disappointment without any additional training or mixed-emotion data. If true, this gives expressive speech synthesis a cheap route from a handful of labeled emotion categories to a much larger continuous space of emotional styles.

What carries the argument

The load-bearing object is the emotion-guided prompt template of Equation (2), a fixed sentence with five named emotion categories and scalar parameters $\alpha,\beta,\gamma,\epsilon,\lambda$ for happy, sad, neutral, angry, and surprise. During training the template is instantiated one-hot (100 for the labeled emotion, 0 otherwise), and the LLM-TTS decoder is trained with an emotion-guided KL loss to predict the discrete speech tokens conditioned on the prompt and the input text. The intended work of the template is to convert the abstract idea of an emotional mixture into familiar textual form, so that the language model's instruction-following and in-context learning abilities can carry the model from the one-hot prompts it saw to the mixed-proportion prompts used at inference. A frozen flow-matching model and a vocoder then turn the predicted tokens into waveform audio.

What would settle it

Run the trained model on a fine-grained proportion grid, for example surprise mixed with 0%, 10%, 30%, 50%, 70%, 90%, and 100% anger, and have listeners or an automatic emotion classifier rank the outputs; if perceived anger does not increase monotonically with the stated percentage, or if a blend such as 47%/53% sounds identical to 50%/50%, then the zero-shot quantitative-control claim is not supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that zero-shot synthesis of unseen emotions is achievable through emotion-guided prompt learning on an LLM-TTS backbone. Each training utterance is paired with the template 'A man/woman speaks an utterance with $\alpha$ percent happy emotion, $\beta$ percent sad emotion, ...' where the dataset's labeled emotion gets 100 and all others get 0; only one-hot prompts are ever seen in training. Despite that, the authors report that at inference, prompts with mixed values such as 100% surprise plus 30, 60, or 90% anger, sadness, or happiness produce perceptually recognizable outrage, disappointment, and delight. Subjective tests show that 94.44% of listeners ranked the 90%-anger blend as expressing the strongest outrage, and the model beats both baseline systems on word error rate and preference tests across all tested blends. The conclusion drawn is that the LLM's in-context learning lets the model extrapolate emotional characteristics from the known one-hot prompts to arbitrary proportion combinations.

Load-bearing premise

The load-bearing premise is that the percentage numbers in the prompt cause a proportional, quantitative shift in the acoustic emotion of the generated speech, even though training only ever showed one-hot prompts (one emotion at 100 and the rest at 0) and the paper offers no mechanistic proof that the model truly interpolates along the proportion axis.

Editorial extensions

If this is right

  • Editing the percentage values in the prompt is sufficient to shift perceived emotional intensity: 94.44% of listeners identified 100% surprise plus 90% anger as the strongest 'outrage', while 80.56% chose 100% surprise plus 0% anger as the weakest.
  • Mixed emotions that never occurred in the training set, such as delight, outrage, and disappointment, can be synthesized without retraining or collecting mixed-emotion speech data.
  • PUE achieves lower word error rates than the prior mixed-emotion TTS system and a strong LLM-TTS baseline on both single-emotion and mixed-emotion test sets, indicating that prompt conditioning does not sacrifice intelligibility.
  • A single trained model can produce a family of blends (30%, 60%, and 90% secondary emotion) from one run, giving fine-grained control over the emotional composition of each utterance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's evidence is perceptual and limited to a few blends; the strongest test of the zero-shot claim would be a fine-grained sweep over unseen proportions, such as every 5% from 0 to 100, with an automatic emotion recognizer or forced-choice listening to check that perceived emotion moves monotonically with the stated percentages.
  • If the mechanism works as described, the same prompt-template trick could be lifted onto other LLM-TTS systems and applied to non-emotional style dimensions, such as speaking rate, energy, age, or formality, where labeled extremes exist but mixed styles do not.
  • Because the model only saw one-hot prompts, an alternative explanation is that it latches onto coarse textual cues like '90% anger' versus '30% anger' rather than genuinely understanding the numbers; generating blends with unusual values such as 47%/53% or paraphrased proportions would separate these explanations.
  • The five emotion categories are fixed by the training data, so extending to new primary emotions would require retraining on labels for those categories; the method's upper bound is tied to the breadth of the single-emotion training data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes PUE, a prompt-conditioned LLM-based TTS approach for synthesizing mixed and previously unseen emotions. During training, each utterance is paired with an emotion-guided prompt whose five emotion-relevance parameters are set one-hot (100 for the labeled emotion, 0 otherwise). At inference, the user changes these percentages to specify novel mixtures, e.g., surprise plus 30/60/90% anger, which the authors call zero-shot unseen emotion synthesis. The model is built on CosyVoice and evaluated on two speakers from the ESD dataset against a VITS-based mixed-emotion baseline and CosyVoice, using WER, AB preference, best-worst scaling, and MOS tests. The paper claims that PUE quantitatively captures emotion weightings and outperforms the baselines on mixed-emotion generation.

Significance. If the central claim were established, PUE would be a useful contribution to expressive TTS: it offers a simple prompting mechanism for generating emotion mixtures without mixed-emotion training data, and it leverages LLM instruction-following in a domain where most prior work uses fixed categorical labels. The paper's strengths include the clear prompt design, comparison against two relevant baselines, human listening studies, and a public demo page. However, the current evidence does not demonstrate the quantitative proportion-to-emotion mapping that the zero-shot claim rests on, and several technical details (the training objective, baseline prompting, and evaluation logic) need correction. The idea is promising but the manuscript is not yet convincing as a journal contribution.

major comments (5)
  1. [§II-C, Eq. (3)] The KL divergence in Eq. (3) is written inconsistently. The text says the loss is KL(P_theta || P), i.e., the divergence from the predicted distribution to the target, but the displayed equation averages log(p(s_i|EP,t_i)/p_theta(s_i|EP,t_i)) over data samples d_i~D_e, which corresponds to KL(p || p_theta), not KL(P_theta || P). Moreover, the "target emotional distribution P" is never defined; it is not clear whether P is a label-smoothed one-hot distribution, a distribution over speech tokens, or something else. Please rewrite Eq. (3) with the correct direction and explicitly define P. Without this, the training objective is ambiguous and cannot support the claim that the model quantitatively captures emotion proportions.
  2. [§II-B and §IV-C (Tables II and III)] The central zero-shot claim requires that the numerical percentages in Eq. (2) act as continuous control knobs over the acoustic emotion of the output, but the training data contain only 100/0 one-hot prompts. The reported evidence for proportional control is non-monotonic: in the BWS test (Table II), the 30% and 60% angry conditions both receive 0% "best" ratings, and the 60% condition is judged "worst" more often (11.11%) than the 30% condition (2.78%); in the MOS test (Table III), quality drops from 3.12 at 30% to 2.50 at 60% and then rises to 3.57 at 90%. These results do not establish a monotonic relationship between prompt percentage and perceived emotion intensity. Please provide a direct evaluation of emotion perception across multiple intermediate proportions (including objective acoustic emotion measures where possible), and explicitly test whether the proportion-to-emotion mapping is monotonic and reproducible.
  3. [§III-A (baseline setup)] The CosyVoice baseline prompt is not described. The paper states that a pretrained flow-matching model and HiFi-GAN vocoder are used for both CosyVoice and PUE, but it does not report what instruction or emotion information CosyVoice received. If CosyVoice was given only a categorical emotion label while PUE received the full proportion prompt, the AB/MOS comparisons in Figs. 3-5 conflate model quality with prompt/conditioning differences. Please specify the exact prompt used for each baseline and ensure that all systems receive comparable emotion information.
  4. [§IV-A and §IV-C1 (Table I)] WER is a measure of intelligibility, not emotional expressiveness. Statements such as "PUE demonstrates superior performance in mixed-emotion speech generation over the two strong baselines" and "more intelligent mixed-emotion speech" are based on Table I alone, which only reports WER. Lower WER can be achieved by a neutral, less expressive voice, so it does not support the paper's emotional-rendering claims. The subjective evaluations are the relevant evidence for emotion, but in Table III the MOS is described as "speech quality" rather than a direct rating of perceived emotion. Please separate intelligibility from emotion perception and use emotion-specific subjective ratings or objective acoustic emotion measures to support the mixed-emotion claims.
  5. [§IV-E (zero-shot evaluation)] The 90% mixing ratio used in the zero-shot unseen-emotion evaluation is chosen post hoc based on the earlier results in Tables I-III, and Fig. 5 evaluates only the 90% case. Selecting the ratio that performed best in prior tests, then reporting that ratio as the zero-shot evaluation, makes the claim weaker than stated. Please either report the unseen-emotion MOS/AB results for at least two mixing ratios (e.g., 30% and 90%), or provide a principled justification for why 90% alone is representative of arbitrary unseen mixtures.
minor comments (6)
  1. [§II-B, Eq. (1)] The symbol list after Eq. (1) mentions EP, t_i, s_i, <EOP>, and <T>, but the equation also contains <E> (the end-of-sequence token), which is not defined in the list.
  2. [§I (Introduction)] The text states that "the only prior work for mixed emotional TTS employs a VITS-based framework with a relative scheme [23]," but reference [22] (EmoMix) is cited earlier in the same section and is itself a mixed-emotion speech synthesis method based on diffusion models. Please reconcile this claim with the cited literature.
  3. [Table I and §IV-C1] "Speech Intelligence" in the Table I caption should be "Speech Intelligibility," and the phrase "more intelligent mixed-emotion speech" in Section IV-C1 is misleading; use "more intelligible" or rephrase.
  4. [§III-C] The sentence "18 listeners participate all subjective evaluation tests" should be "18 listeners participated in all subjective evaluation tests." Please also state whether the same 18 listeners took part in every test and whether they were native English speakers or had any emotional-speech expertise.
  5. [§II-B, Eq. (2)] Eq. (2) does not specify valid ranges for the percentages α, β, γ, ε, and λ. The paper should state that they are intended to be nonnegative and to sum to 100, and clarify what happens if the user supplies values outside these ranges, since the inference mechanism is otherwise undefined.
  6. [§I (Introduction)] The claim of "approximately 34,000 distinct emotions" is not supported by the cited Plutchik reference, which describes eight primary emotions; please either provide a precise citation for this number or soften the claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the zero-shot mixed-emotion claim is an empirical generalization from one-hot prompt training, not a fitted parameter renamed as a prediction.

full rationale

The paper's central claim is that a model trained on one-hot emotion prompts (one emotion set to 100, the rest to 0 in Eq. (2)) can synthesize speech for mixed prompts such as surprise + 90% anger at inference. The emotion proportions are user-supplied inputs, not parameters fitted to the evaluation data, so the evaluation does not reduce to the training objective by construction. No load-bearing argument rests on a self-citation: the authors' own EmoDPO papers [29,31] are cited as related work, while the architecture, baselines, and dataset are external (CosyVoice [37], mixed-emotion baseline [23], ESD [38]). The 'unseen emotions' are compositional blends of training categories, but that is the intended zero-shot generalization setting rather than a definitional equivalence. The reversed KL argument in Eq. (3) and the non-monotonic BWS/MOS results are empirical or formal weaknesses, not circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No new physical or conceptual entities are introduced. The method's dependence is on prompt design, the choice of mixing ratios, and the unverified KL target, all listed above.

free parameters (2)
  • Inference emotion mixing ratios = 30/60/90
    Hand-chosen percentages for the secondary emotion in mixed-emotion prompts; the 90% value for zero-shot testing is selected based on earlier results, not independently motivated.
  • One-hot training proportions = 100 for target, 0 for others
    Design choice for training prompts; assumed to teach the model the meaning of the percentage scale.
assumptions (4)
  • domain assumption Emotions are composable from a small set of primaries (emotion wheel theory)
    Invoked in Section I to justify that unseen emotions like disappointment are blends of surprise and sadness; not empirically established in this paper.
  • ad hoc to paper LLM-TTS can condition on numeric percentages in a natural language prompt
    Core behavioral hypothesis of the method; no theoretical guarantee, only empirical testing.
  • ad hoc to paper The target emotional distribution P in Eq. (3) exists and can be used in a KL loss
    The paper never defines P; the loss equation appears to invert the KL arguments and is therefore not self-contained.
  • domain assumption ESD emotion labels are accurate single-emotion ground truths
    Training and evaluation rely on ESD labels without independent verification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Prompt-Unseen-Emotion: Zero-shot Expressive Speech Synthesis with Prompt-LLM Contextual Knowledge for Mixed Emotions." pith.science (2026). https://pith.science/paper/ZNQO2XLD

@misc{pith2026250602742,
  author       = {Pith},
  title        = {Pith review of: Prompt-Unseen-Emotion: Zero-shot Expressive Speech Synthesis with Prompt-LLM Contextual Knowledge for Mixed Emotions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZNQO2XLD}},
  note         = {Machine review of arXiv:2506.02742}
}
read the original abstract

Existing expressive text-to-speech (TTS) systems primarily model a limited set of categorical emotions, whereas human conversations extend far beyond these predefined emotions, making it essential to explore more diverse emotional speech generation for more natural interactions. To bridge this gap, this paper proposes a novel prompt-unseen-emotion (PUE) approach to generate unseen emotional speech via emotion-guided prompt learning. PUE is trained utilizing an LLM-TTS architecture to ensure emotional consistency between categorical emotion-relevant prompts and emotional speech, allowing the model to quantitatively capture different emotion weightings per utterance. During inference, mixed emotional speech can be generated by flexibly adjusting emotion proportions and leveraging LLM contextual knowledge, enabling the model to quantify different emotional styles. Our proposed PUE successfully facilitates expressive speech synthesis of unseen emotions in a zero-shot setting.

Figures

Figures reproduced from arXiv: 2506.02742 by the authors.

Figure 1
Figure 1. An overview of the proposed PUE approach: (a) the PUE training process and (b) the PUE inference process. is trained using an LLM-based TTS architecture, which condi￾tions the emotional context on speech generation to maintain emotional consistency between the contextual prompts and the corresponding speech data. By incorporating various emotion style descriptions into the emotion-relevant contexts, the PUE leverage… view at source ↗
Figure 2
Figure 2. Comparison of subjective evaluation results for AB preference test [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparison of AB preference test results with 95% confidence interval [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Comparison of AB preference test results with 95% confidence interval [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Comparison of subjective evaluation results for MOS tests with 95% [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Unlocking Fine-Grained and Within-Utterance Speaking Style Control in Prompt-Based Text-to-Speech Models

    cs.CL 2026-04 conditional novelty 6.0 of 10

    Direction-vector style interpolation plus KV-cache swap and sliding-window masking unlock continuous inter- and intra-utterance style control in prompt-based autoregressive TTS without training.

Reference graph

Works this paper leans on

39 extracted references · 24 canonical work pages · cited by 1 Pith paper

  1. [1]

    Text-to-speech synthesis based on latent variable conversion using diffusion probabilistic model and variational autoencoder,

    Y . Yasuda and T. Toda, “Text-to-speech synthesis based on latent variable conversion using diffusion probabilistic model and variational autoencoder,” inIEEE ICASSP, 2023, pp. 1–5

    work page 2023

  2. [2]

    A vector quantized approach for text to speech synthesis on real-world spontaneous speech,

    L.-W. Chen, S. Watanabe, and A. Rudnicky, “A vector quantized approach for text to speech synthesis on real-world spontaneous speech,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 11, 2023, pp. 12 644–12 652

    work page 2023

  3. [3]

    Text to speech synthesis: A systematic review, deep learning based architecture and future research direction,

    F. Khanam, F. A. Munmun, N. A. Ritu, A. K. Saha, and M. Firoz, “Text to speech synthesis: A systematic review, deep learning based architecture and future research direction,”Journal of Advances in Information Technology Vol, vol. 13, no. 5, 2022

    work page 2022

  4. [4]

    A style control technique for hmm-based expressive speech synthesis,

    T. Nose, J. Yamagishi, T. Masuko, and T. Kobayashi, “A style control technique for hmm-based expressive speech synthesis,”IEICE TRANS- ACTIONS on Information and Systems, vol. 90, no. 9, pp. 1406–1413, 2007

    work page 2007

  5. [5]

    Seamless: Multilingual expressive and streaming speech translation,

    L. Barrault, Y .-A. Chung, M. C. Meglioli, D. Dale, N. Dong, M. Dup- penthaler, P.-A. Duquenne, B. Ellis, H. Elsahar, J. Haaheimet al., “Seamless: Multilingual expressive and streaming speech translation,” arXiv preprint arXiv:2312.05187, 2023

    arXiv 2023

  6. [6]

    Emotional speech synthesis with rich and granularized control,

    S.-Y . Um, S. Oh, K. Byun, I. Jang, C. Ahn, and H.-G. Kang, “Emotional speech synthesis with rich and granularized control,” inIEEE ICASSP, 2020, pp. 7254–7258

    work page 2020

  7. [7]

    Emospeech: guiding fastspeech2 towards emotional text to speech,

    D. Diatlova and V . Shutov, “Emospeech: guiding fastspeech2 towards emotional text to speech,” inSSW 2023

    work page 2023

  8. [8]

    Emotional end-to-end neural speech synthesizer,

    Y . Lee, A. Rabiee, and S.-Y . Lee, “Emotional end-to-end neural speech synthesizer,”arXiv preprint arXiv:1711.05447, 2017

    arXiv 2017

Show all 39 references
  1. [9]

    Mm-tts: A unified framework for multimodal, prompt-induced emotional text-to- speech synthesis,

    X. Li, Z.-Q. Cheng, J.-Y . He, X. Peng, and A. G. Hauptmann, “Mm-tts: A unified framework for multimodal, prompt-induced emotional text-to- speech synthesis,”arXiv preprint arXiv:2404.18398, 2024

    2024 arXiv

  2. [10]

    Emodiff: Intensity controllable emotional text-to-speech with soft-label guidance,

    Y . Guo, C. Du, X. Chen, and K. Yu, “Emodiff: Intensity controllable emotional text-to-speech with soft-label guidance,” inIEEE ICASSP, 2023, pp. 1–5

    2023

  3. [11]

    Instructtts: Modelling expressive tts in discrete latent space with natural language style prompt,

    D. Yang, S. Liu, R. Huang, C. Weng, and H. Meng, “Instructtts: Modelling expressive tts in discrete latent space with natural language style prompt,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024

    2024

  4. [12]

    Emotional voice conversion using dual supervised adversarial networks with continuous wavelet transform f0 features,

    Z. Luo, J. Chen, T. Takiguchi, and Y . Ariki, “Emotional voice conversion using dual supervised adversarial networks with continuous wavelet transform f0 features,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 27, no. 10, pp. 1535–1548, 2019

    2019

  5. [13]

    Emotion controllable speech synthesis using emotion-unlabeled dataset with the assistance of cross-domain speech emotion recognition,

    X. Cai, D. Dai, Z. Wu, X. Li, J. Li, and H. Meng, “Emotion controllable speech synthesis using emotion-unlabeled dataset with the assistance of cross-domain speech emotion recognition,” inIEEE ICASSP, 2021, pp. 5734–5738

    2021

  6. [14]

    Laugh now cry later: Controlling time-varying emotional states of flow-matching-based zero-shot text-to- speech,

    H. Wu, X. Wang, S. E. Eskimez, M. Thakker, D. Tompkins, C.-H. Tsai, C. Li, Z. Xiao, S. Zhao, J. Liet al., “Laugh now cry later: Controlling time-varying emotional states of flow-matching-based zero-shot text-to- speech,”arXiv preprint arXiv:2407.12229, 2024

    2024 arXiv

  7. [15]

    Expressive text-to-speech using style tag,

    M. Kim, S. J. Cheon, B. J. Choi, J. J. Kim, and N. S. Kim, “Expressive text-to-speech using style tag,”arXiv preprint arXiv:2104.00436, 2021

    2021 arXiv

  8. [16]

    Controllable emotion transfer for end-to-end speech synthesis,

    T. Li, S. Yang, L. Xue, and L. Xie, “Controllable emotion transfer for end-to-end speech synthesis,” inISCSLP. IEEE, 2021, pp. 1–5

    2021

  9. [17]

    emotion2vec: Self-supervised pre-training for speech emotion repre- sentation,

    Z. Ma, Z. Zheng, J. Ye, J. Li, Z. Gao, S. Zhang, and X. Chen, “emotion2vec: Self-supervised pre-training for speech emotion repre- sentation,”arXiv preprint arXiv:2312.15185, 2023

    2023 arXiv

  10. [18]

    Ttslow: Slow down text-to-speech with efficiency robustness evaluations,

    X. Gao, Y . Chen, X. Yue, Y . Tsao, and N. F. Chen, “Ttslow: Slow down text-to-speech with efficiency robustness evaluations,”IEEE Transac- tions on Audio, Speech and Language Processing, 2025

    2025

  11. [19]

    Hierarchical control of emotion rendering in speech synthesis,

    S. Inoue, K. Zhou, S. Wang, and H. Li, “Hierarchical control of emotion rendering in speech synthesis,”arXiv preprint arXiv:2412.12498, 2024

    2024 arXiv

  12. [20]

    The nature of emotions: Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice,

    R. Plutchik, “The nature of emotions: Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice,”American scientist, vol. 89, no. 4, pp. 344– 350, 2001

    2001

  13. [21]

    Mixed emotions and coping: The benefits of secondary emotions,

    A. Braniecka, E. Trzebi ´nska, A. Dowgiert, and A. Wytykowska, “Mixed emotions and coping: The benefits of secondary emotions,”PloS one, vol. 9, no. 8, p. e103940, 2014

    2014

  14. [22]

    Emomix: Emotion mixing via diffusion models for emotional speech synthesis,

    H. Tang, X. Zhang, J. Wang, N. Cheng, and J. Xiao, “Emomix: Emotion mixing via diffusion models for emotional speech synthesis,”arXiv preprint arXiv:2306.00648, 2023

    2023 arXiv

  15. [23]

    Speech synthe- sis with mixed emotions,

    K. Zhou, B. Sisman, R. Rana, B. W. Schuller, and H. Li, “Speech synthe- sis with mixed emotions,”IEEE Transactions on Affective Computing, vol. 14, no. 4, pp. 3120–3134, 2022

    2022

  16. [24]

    Style tokens: Unsupervised style modeling, control and transfer in end-to-end speech synthesis,

    Y . Wang, D. Stanton, Y . Zhang, R.-S. Ryan, E. Battenberg, J. Shor, Y . Xiao, Y . Jia, F. Ren, and R. A. Saurous, “Style tokens: Unsupervised style modeling, control and transfer in end-to-end speech synthesis,” inInternational conference on machine learning. PMLR, 2018, pp. ...

    2018

  17. [25]

    Fine-grained emotion strength transfer, control and prediction for emotional speech synthesis,

    Y . Lei, S. Yang, and L. Xie, “Fine-grained emotion strength transfer, control and prediction for emotional speech synthesis,” in2021 IEEE SLT, 2021, pp. 423–430

    2021

  18. [26]

    Emotion rendering for conversational speech synthesis with heterogeneous graph-based con- text modeling,

    R. Liu, Y . Hu, Y . Ren, X. Yin, and H. Li, “Emotion rendering for conversational speech synthesis with heterogeneous graph-based con- text modeling,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 17, 2024, pp. 18 698–18 706

    2024

  19. [27]

    An emotion speech synthesis method based on vits,

    W. Zhao and Z. Yang, “An emotion speech synthesis method based on vits,”Applied Sciences, vol. 13, no. 4, p. 2225, 2023

    2023

  20. [28]

    Emotional dimension control in language model-based text-to-speech: Spanning a broad spectrum of human emotions,

    K. Zhou, Y . Zhang, S. Zhao, H. Wang, Z. Pan, D. Ng, C. Zhang, C. Ni, Y . Ma, T. H. Nguyenet al., “Emotional dimension control in language model-based text-to-speech: Spanning a broad spectrum of human emotions,”arXiv preprint arXiv:2409.16681, 2024

    2024

  21. [29]

    Emo- dpo: Controllable emotional speech synthesis through direct preference optimization,

    X. Gao, C. Zhang, Y . Chen, H. Zhang, and N. F. Chen, “Emo- dpo: Controllable emotional speech synthesis through direct preference optimization,”arXiv preprint arXiv:2409.10157, 2024

    2024 arXiv

  22. [30]

    Llasa: Scaling train-time and inference- time compute for llama-based speech synthesis,

    Z. Ye, X. Zhu, C.-M. Chan, X. Wang, X. Tan, J. Lei, Y . Peng, H. Liu, Y . Jin, Z. DAIet al., “Llasa: Scaling train-time and inference- time compute for llama-based speech synthesis,”arXiv preprint arXiv:2502.04128, 2025

    2025 arXiv

  23. [31]

    Emo- dpo: Controllable emotional speech synthesis through direct preference optimization,

    X. Gao, C. Zhang, Y . Chen, H. Zhang, and N. F. Chen, “Emo- dpo: Controllable emotional speech synthesis through direct preference optimization,” inICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2025, pp. 1–5

    2025

  24. [32]

    Plutchik and H

    R. Plutchik and H. Kellerman,Theories of emotion. Academic press, 2013, vol. 1

    2013

  25. [33]

    Cross and C

    M. Cross and C. Hanrahan,Changing Minds: The Go-to Guide to Mental Health for Family and Friends. HarperCollins Australia, 2016

    2016

  26. [34]

    Mixed-evc: Mixed emotion synthesis and control in voice conversion,

    K. Zhou, B. Sisman, C. Busso, B. Ma, and H. Li, “Mixed-evc: Mixed emotion synthesis and control in voice conversion,”arXiv preprint arXiv:2210.13756, 2022

    2022 arXiv

  27. [35]

    GPT-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkatet al., “GPT-4 technical report,”arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  28. [36]

    Gemini: a family of highly capable multimodal models,

    G. T. Google, R. Anil, S. Borgeaud, Y . Wu, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauthet al., “Gemini: a family of highly capable multimodal models,”arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  29. [37]

    Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens,

    Z. Du, Q. Chen, S. Zhang, K. Hu, H. Lu, Y . Yang, H. Hu, S. Zheng, Y . Gu, Z. Maet al., “Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens,”arXiv preprint arXiv:2407.05407, 2024

    2024 arXiv

  30. [38]

    Emotional voice conversion: Theory, databases and esd,

    K. Zhou, B. Sisman, R. Liu, and H. Li, “Emotional voice conversion: Theory, databases and esd,”Speech Communication, vol. 137, pp. 1–18, 2022

    2022

  31. [39]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” 2022. [Online]. Available: https://arxiv.org/abs/2212.04356

    2022 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.