DialogueAgents: A Hybrid Agent-Based Speech Synthesis Framework for Multi-Party Dialogue

Bo Cheng; Duyi Pan; Hongru Xiao; Jiabao Ma; Jiale Han; Jing Tang; Wei Wang; Xiang Li

REVIEW 3 major objections 5 minor 18 references

DialogueAgents: A Hybrid Agent-Based Speech Synthesis Framework for Multi-Party Dialogue

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Three cooperating AI agents can generate multi-party dialogue speech good enough to train TTS models.

desk verdict Useful pipeline and a new dataset, but the key utility claim rests on a circular evaluation; worth serious review with a required fix. read the letter →

arxiv 2504.14482 v1 pith:MSYOB7GQ submitted 2025-04-20 cs.CL cs.SD

Xiang Li , Duyi Pan , Hongru Xiao , Jiale Han , Jing Tang , Jiabao Ma , Wei Wang , Bo Cheng This is my paper · ORCID

classification cs.CLcs.SD

keywords speechsynthesismulti-partydialoguemulti-agentframeworkdatasetparalinguistictokensiterativerefinementtext-to-speechMultiTalk

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that high-quality, multi-party, multi-turn spoken dialogue data can be produced automatically by a loop of three specialized agents—a script writer, a speech synthesizer, and a dialogue critic—rather than through costly human transcription and studio recording. If true, an automated pipeline can substitute for human-annotated dialogue data in training speech synthesis models, making customized, scalable dialogue data affordable. The authors use the framework to build MultiTalk, a bilingual English and Chinese dataset with 30 diverse characters, and report that models trained on it reach a MOS comparable to models trained on the human-recorded DailyTalk dataset, despite using less than a quarter of the data volume. The paper also argues that two iterations of critic feedback give the best quality, with a third iteration starting to degrade the output.

What carries the argument

The load-bearing mechanism is the critic-in-the-loop refinement cycle: a Script Writer agent (GPT-4o) generates a multi-party script from a pool of 30 hand-crafted character profiles, a Speech Synthesizer agent (CosyVoice2) converts it into audio, and a Dialogue Critic agent (Qwen2-Audio) listens and returns textual feedback on naturalness, clarity, and emotiveness. The writer then revises the script—inserting paralinguistic tokens like '[breath]' and '<strong>' and emotional labels like '[Agreeable]'—and the loop repeats. The character pool, with predefined social relationships, is what makes multi-party, multi-turn dialogues with consistent voices possible.

What would settle it

Have a panel of human raters score blinded dialogue scripts from the writer-only condition versus the two-loop critic condition on naturalness and emotiveness; if human ratings show no significant advantage for the critic-refined scripts, the claimed script-quality improvement collapses. Alternatively, train a TTS model on a random matched-size subset of DailyTalk and compare its MOS to a model trained on MultiTalk; if the subset-trained model matches or exceeds MultiTalk-trained MOS, the paper's central data-quality claim is falsified.

Watch

Extended reading notes

Core claim

The central claim is that an iterative three-agent loop—where a large language model writes dialogue scripts, a zero-shot speech synthesizer renders them with paralinguistic markers, and an audio-language model critic reviews the synthesized speech and feeds textual advice back to the writer—produces dialogue speech that is natural, emotionally expressive, and usable as training data. The authors demonstrate this by generating MultiTalk and evaluating it against established datasets. Their headline result is that a TTS model trained on MultiTalk achieves a mean opinion score statistically similar to one trained on DailyTalk, even though MultiTalk contains less than a quarter of the dialogues, supporting the idea that automatically synthesized dialogue data can rival human-recorded data in utility.

Load-bearing premise

The paper assumes that using GPT-4o both as the script writer and as the evaluator of script naturalness and emotiveness does not bias the reported quality gains, and that the speech critic's feedback reflects genuine auditory quality rather than artifacts of the same model family.

Editorial extensions

If this is right

Automatically generated dialogue speech can serve as a viable substitute for human-annotated dialogue data in training TTS models, lowering the cost and effort of dataset construction.
Two iterations of critic feedback appear to be the sweet spot; adding a third iteration degrades quality due to over-optimization, suggesting an optimal stopping criterion for such loops.
The framework is model-agnostic per agent, so swapping in better script writers, synthesizers, or critics should yield further quality gains without redesigning the loop.
MultiTalk's bilingual, multi-party structure covers conversational scenarios—several speakers, interruptions, emotional shifts—that existing single-speaker or two-speaker TTS datasets lack.
The reported MOS parity with DailyTalk at one-quarter the data implies that data quality, not just quantity, drives conversational TTS performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the same model (GPT-4o) both writes the scripts and evaluates their naturalness and emotiveness, the reported script-quality improvements from critic feedback could partly reflect self-serving bias; an independent evaluator or blinded human raters would give a cleaner test.
The critic's textual feedback could itself serve as a rich annotation signal—aligned with specific audio segments—potentially useful for training models on paralinguistic control beyond what MultiTalk's audio alone provides.
The iterative refine-and-synthesize loop is a general recipe that could extend to other domains, such as audiobook narration with multiple characters or interactive game dialogue, wherever a critic can listen and give actionable feedback.
A direct comparison against a human-recorded dataset of matched size, rather than the full DailyTalk, would isolate whether MultiTalk's quality claim comes from the data itself or from the TTS model's capacity to overfit a small, homogeneous set.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read it. The pipeline—script writer (GPT-4o), synthesizer (CosyVoice2), critic (Qwen2-Audio) in an iterative loop—is a sensible reuse of agent collaboration for TTS dataset construction. MultiTalk is a new bilingual, multi-party, multi-turn dataset, and the code and data are public. That is a usable contribution.

The audit is right that Table VI is circular. Models are trained and evaluated on the same distribution: for MultiTalk, both sides are CosyVoice-generated synthetic speech; for DailyTalk, both sides are human recordings. A model can get a good MOS on MultiTalk by mimicking CosyVoice's voice pool and artifacts. So the “comparable MOS despite less data” claim does not demonstrate substitutability for human-recorded dialogue data. The DailyModel rows are a better comparison, but still the test distributions differ in difficulty. This is the load-bearing weakness, not the script-evaluator issue.

The second weakness is that script naturalness and emotiveness are judged by GPT-4o, the same model that wrote the scripts. That invites self-serving bias, and the fact that self-refinement alone improves scores under that judge suggests the gains may be partly inflated. Also, no significance tests are provided, most confidence intervals overlap, and the optimal iteration count T=2 is selected after seeing the ablation. These are secondary but real.

What the paper does well: the critic-driven revision with paralinguistic tokens is a clean idea, the example shows genuine improvement, and the release of the dataset and pipeline is a useful resource for the community. The authors are straightforward about their setup.

So: worth serious peer review, not desk reject. But the revision needs to address the circular evaluation—ideally by adding a cross-dataset experiment where a model trained on MultiTalk is evaluated on natural dialogue audio, or at least by stating and contextualizing the limitation. A different LLM for script evaluation, or human ratings, would also strengthen the claims. This is a methods-and-resource paper for speech synthesis people; I'd bring it to a reading group if one of us works in TTS, otherwise it's a good cautionary example for evaluation design.

Referee Report

3 major / 5 minor

Summary. The paper proposes DialogueAgents, a hybrid agent-based framework for generating multi-party, multi-turn speech dialogue data. The framework comprises three agents: a Script Writer (GPT-4o) that generates and refines dialogue scripts, a Speech Synthesizer (CosyVoice) that converts scripts to speech, and a Dialogue Critic (Qwen2-Audio) that reviews the synthesized speech and provides textual feedback for iterative refinement. The authors use this framework to construct MultiTalk, a bilingual (Chinese and English) dataset of 4,437 utterances across 30 characters, and introduce two dialogue-level metrics, Emotional MOS (EMOS) and Turn-taking MOS (TMOS). Experiments evaluate the framework through human MOS, automated UTMOS/WER/CER, LLM-based script naturalness and emotiveness scores, and downstream training of Tacotron2, FastSpeech2, and DailyTalk-Model on the generated dataset.

Significance. If the claims are validated, the contribution is valuable: an automated pipeline that generates multi-party dialogue speech data with emotional and paralinguistic richness could substantially reduce dataset construction cost and increase diversity relative to manually recorded corpora. The framework is simple, modular, and the authors release the dataset and code, which would facilitate reproducibility and follow-up work. The introduction of EMOS and TMOS as dialogue-level evaluation metrics is also a useful addition. The paper's strengths include a concrete end-to-end system, a public dataset release, and a downstream TTS training evaluation that, while currently flawed in design, demonstrates a genuine attempt to assess practical utility.

major comments (3)

[§V-C, Table VI] The dataset evaluation is circular and does not support the claim that MultiTalk substitutes for human-recorded dialogue data. Models trained on MultiTalk are evaluated on held-out MultiTalk audio, which is synthetic speech generated by the same CosyVoice pipeline and voice pool used to create the training set; models trained on DailyTalk are evaluated on held-out DailyTalk natural human speech. A high MOS in the 'Ours' column can be achieved by reproducing CosyVoice's voice characteristics and artifacts, rather than by learning general dialogue-to-speech ability. The comparison is therefore between in-domain synthetic-to-synthetic and in-domain natural-to-natural matches, which are not commensurate. To substantiate the claim, the authors should include a cross-dataset evaluation, such as training on MultiTalk and testing on natural speech (e.g., DailyTalk) or reporting human listening tests on natural reference audio, or at minimum clarifying what the 'Ground Truth' row means for a fully synthetic dataset.
[§IV-A, Script evaluation metrics] The script naturalness and emotiveness scores in Table III are produced by GPT-4o, which is also the model that generates and refines the scripts. This self-evaluation setup risks systematic bias: GPT-4o may rate its own revised scripts higher regardless of genuine quality, and the reported improvements from the critic loop could reflect the evaluator's preference for its own output style rather than actual gains. This is load-bearing because Table III is the principal evidence that the iterative critic loop improves script quality. The authors should add human evaluation of the scripts or use an independent LLM as the evaluator, and report agreement between the automatic evaluator and human judgments.
[§IV-B, Iteration count selection] The choice of T=2 as the number of refinement iterations is made after inspecting the ablation results on the same 30 dialogues and is then presented as an 'optimal number of iterations' finding. This is a fitted hyperparameter, not a validated principle. Moreover, the differences between the 1-loop, 2-loop, and 3-loop conditions overlap within the reported confidence intervals (e.g., MOS 3.71±0.063, 3.75±0.071, 3.78±0.075 in Table II), and no significance tests are reported. The claim of an optimal iteration count requires either statistical significance testing or validation on held-out dialogues; otherwise the selection of T=2 should be described as a design choice rather than an empirical discovery.

minor comments (5)

[§IV-A, Human evaluation] The human evaluation details are incomplete: the paper states that 20 bilingual volunteers rated dialogue samples but does not specify the number of rated samples per condition, the rating instructions, the presentation order/randomization, or inter-rater agreement (e.g., Krippendorff's alpha). These details are needed to assess the reliability of the MOS, EMOS, and TMOS scores.
[Tables II and VI] The automated metrics (UTMOS, WER, CER) are reported as point estimates without confidence intervals or significance tests, and Table VI does not report test-set sizes. This makes it difficult to determine whether the observed differences are meaningful; please add uncertainty measures and test-set descriptions.
[Figure 2] Figure 2 is described as an 'Ablation of critic agent' showing that refined scripts outperform originals in 75.8% of paired human judgments, but the figure's axes, the number of paired samples, and any statistical test are not described. Please clarify what the figure displays and how the 75.8% figure was computed.
[§III-A] The notation for the character pool is inconsistent and contains formatting artifacts (e.g., 'P oola', 'P oolp' with odd spacing). Please standardize the notation, for example using P_a, P_p, and P_c.
[Footnotes and references] The CosyVoice footnote contains a line break and the URL appears to be broken across lines, and the WenetSpeech4TTS reference is cited but the character pool description says the data come from both WenetSpeech4TTS and Common Voice; please verify the citations and clean the formatting.

Circularity Check

2 steps flagged · score 6.0 of 10

Dataset-usability claim reduces to an in-distribution fit; script-quality claim is self-evaluated by the same model that writes the scripts.

fitted input called prediction [Section V-C, Table VI (Dataset Evaluation)]
"To demonstrate the usability of our synthesized dataset, MultiTalk, we compare it with existing speech datasets by evaluating their performance in training various models... Tacotron2 and FastSpeech2 are trained for 900K steps, while the DailyTalk-Model is trained on 850 dialogues and validated on 75 dialogues until convergence. As shown in Table VI, the MOS of the synthesized speech is comparable when models are trained on our dataset versus other datasets."

The claimed evidence that MultiTalk is a usable training resource is obtained by training TTS models on MultiTalk and then evaluating them on held-out audio drawn from the same MultiTalk distribution, i.e., CosyVoice-synthesized speech from the same 30-character pool. The 'Ours' column therefore measures in-distribution reconstruction, including the model's ability to memorize the synthetic voice pool and production artifacts, rather than transferable utility for natural dialogue synthesis. The comparison with DailyTalk is between synthetic-to-synthetic and natural-to-natural MOS values, so a similar MOS does not establish that MultiTalk substitutes for human-recorded dialogue data.
other [Section IV-A, Metrics and Selection for Framework Agents; Table III]
"We adopt GPT-4o as the script writer agent, leveraging its advanced text generation capabilities and strong adherence to instructions. ... For text scripts, we employ an LLM (GPT-4o) as evaluator to assess naturalness and emotiveness, which is prompted to determine whether the scripts are natural and whether the emotional content is accurate and appropriate."

The script naturalness and emotiveness improvements reported in Table III are rated by GPT-4o, the very same model that acts as the Script Writer and produces the revised scripts. Thus the conclusion in Section IV-B that introducing the critic agent improves script quality is based on a self-assessment of the generator's own output. The evaluator and the producer are identical, so the reported gains cannot independently certify that the critic loop improves scripts; an external evaluator or human rating would be required to break the self-referential loop.

full rationale

The framework's core speech-quality claim retains partial independent support: human MOS in Table II and the paired human-preference result in Figure 2 improve when the critic loop is added, so those results are not circular. The raw MultiTalk audio also receives a human ground-truth MOS of 3.76. However, the paper's headline dataset-usability claim, that MultiTalk substitutes for human-recorded dialogue data because it achieves a similar MOS with less than a quarter of the data volume, rests on Table VI, where models are trained and evaluated on the same distribution. This makes the comparison an in-distribution fit, not an independent prediction of cross-domain utility. Additionally, the script-quality gains that motivate the framework are judged by GPT-4o, the same model that writes the scripts, so those gains are self-referential. The optimal iteration count T=2 is also selected from the same 30-dialogue evaluation used to report framework superiority, and the final MultiTalk is then generated with that selected value, adding a hyperparameter-selection overfitting concern. There are no load-bearing self-citations: reference [10] involves overlapping authors but appears only in related work and does not carry the main argument. Overall, the central derivation is partly self-contained, but one of the most load-bearing claims reduces to an in-distribution evaluation, so a partial-circularity score of 6 is appropriate.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The framework depends on several black-box model capabilities (GPT-4o, CosyVoice2, Qwen2-Audio) that are treated as reliable tools. The only explicit free parameter is the iteration count, which is chosen from the data. The central evaluation also relies on self-evaluation of scripts by the same LLM, which inflates circularity burden.

free parameters (4)

Number of refinement iterations T = 2
Set to 2 because the 2-loop variant scored best in the ablation (Table II). This is a post-hoc data-driven choice rather than a theoretically predicted value.
Character pool size N = 30
Chosen as the number of distinct characters; no justification given for this size.
Paralinguistic token set = e.g., <strong>, [breath], [laughter]
The set of paralinguistic and emotional tokens added by the script writer is hand-selected; the specific vocabulary is not justified.
Critic evaluation criteria = naturalness, clarity/emotiveness
The prompt for the critic agent defines its evaluation dimensions; these are chosen by the authors.

assumptions (6)

domain assumption The large language model GPT-4o produces fluent, natural dialogue scripts and follows instructions.
The Script Writer is assumed to generate high-quality initial scripts and meaningfully incorporate critic feedback. No verification of script quality beyond the model's own ratings.
domain assumption CosyVoice2 zero-shot TTS accurately renders emotional and paralinguistic tokens (e.g., [breath], <strong>) into the synthesized speech.
The framework relies on the TTS model to convert scripted markers into acoustic expressiveness; the paper does not measure token-level fidelity.
domain assumption Qwen2-Audio provides valid and useful feedback on synthesized speech.
The critic's feedback is used to refine scripts; its reliability is not separately validated.
domain assumption UTMOS is a valid automated MOS predictor for this type of synthetic dialogue.
Used as an automated evaluation metric (Section IV-A) without comparison to human scores on MultiTalk.
ad hoc to paper GPT-4o's ratings of script naturalness and emotiveness are unbiased despite GPT-4o generating the scripts.
The same model judges scripts it wrote, creating a potential self-evaluation bias (Section IV-A).
domain assumption The 20 human raters provide reliable MOS/EMOS/TMOS scores for multi-party dialogue.
No inter-rater agreement or rater training details are reported (Section IV-A).

how reviews work

0 comments

Cite this review

Pith. "Pith review of DialogueAgents: A Hybrid Agent-Based Speech Synthesis Framework for Multi-Party Dialogue." pith.science (2026). https://pith.science/paper/MSYOB7GQ

@misc{pith2026250414482,
  author       = {Pith},
  title        = {Pith review of: DialogueAgents: A Hybrid Agent-Based Speech Synthesis Framework for Multi-Party Dialogue},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MSYOB7GQ}},
  note         = {Machine review of arXiv:2504.14482}
}

read the original abstract

Speech synthesis is crucial for human-computer interaction, enabling natural and intuitive communication. However, existing datasets involve high construction costs due to manual annotation and suffer from limited character diversity, contextual scenarios, and emotional expressiveness. To address these issues, we propose DialogueAgents, a novel hybrid agent-based speech synthesis framework, which integrates three specialized agents -- a script writer, a speech synthesizer, and a dialogue critic -- to collaboratively generate dialogues. Grounded in a diverse character pool, the framework iteratively refines dialogue scripts and synthesizes speech based on speech review, boosting emotional expressiveness and paralinguistic features of the synthesized dialogues. Using DialogueAgent, we contribute MultiTalk, a bilingual, multi-party, multi-turn speech dialogue dataset covering diverse topics. Extensive experiments demonstrate the effectiveness of our framework and the high quality of the MultiTalk dataset. We release the dataset and code https://github.com/uirlx/DialogueAgents to facilitate future research on advanced speech synthesis models and customized data generation.

Figures

Figures reproduced from arXiv: 2504.14482 by the authors.

**Figure 2.** Ablation of critic agent. (TMOS). EMOS evaluates the appropriateness and consistency of emotional and intonational changes within the dialogue, while TMOS assesses the naturalness and smoothness of speaker transitions. During testing, we invite 20 volunteers fluent in both English and Chinese to rate dialogue samples on a scale of 1 to 5. To ensure objective evaluation, we employ UTMOS [16], a MOS prediction system … view at source ↗

**Figure 3.** Distribution of the dataset by topics. B. Example Dialogue In this section, we present an example to demonstrate how our proposed dialogue synthesis framework effectively refines the multi-party, multi-turn dialogue generation process. As illustrated in Table IV, the initial dialogue script produced by the Script Writer contains only the conversational content between different speakers, without incorporating paral… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 16 canonical work pages

[1]

Chatgpt-edss: Empathetic dialogue speech synthesis trained from chatgpt-derived context word embeddings,

Yuki Saito, Shinnosuke Takamichi, Eiji Iimori, Kentaro Tachibana, and Hiroshi Saruwatari, “Chatgpt-edss: Empathetic dialogue speech synthesis trained from chatgpt-derived context word embeddings,” in Interspeech, Naomi Harte, Julie Carson-Berndsen, and Gareth Jones, Eds. 2023, ISCA

work page 2023
[2]

The voice bank corpus: Design, collection and data analysis of a large regional accent speech database,

Christophe Veaux, Junichi Yamagishi, and Simon King, “The voice bank corpus: Design, collection and data analysis of a large regional accent speech database,” in COCOSDA, 2013

work page 2013
[3]

Wenetspeech4tts: A 12,800-hour mandarin tts corpus for large speech generation model benchmark,

Linhan Ma, Dake Guo, Kun Song, Yuepeng Jiang, Shuai Wang, Li- umeng Xue, Weiming Xu, Huan Zhao, Binbin Zhang, and Lei Xie, “Wenetspeech4tts: A 12,800-hour mandarin tts corpus for large speech generation model benchmark,” arXiv preprint arXiv:2406.05763 , 2024

arXiv 2024
[4]

Dailytalk: Spoken dialogue dataset for conversational text-to-speech,

Keon Lee, Kyumin Park, and Daeyoung Kim, “Dailytalk: Spoken dialogue dataset for conversational text-to-speech,” in ICASSP, 2023

work page 2023
[5]

The fisher corpus: a resource for the next generations of speech-to-text,

Christopher Cieri, David Miller, and Kevin Walker, “The fisher corpus: a resource for the next generations of speech-to-text,” in LREC, 2004

work page 2004
[6]

Advancing speech language models by scaling supervised fine-tuning with over 60,000 hours of synthetic speech dialogue data,

Shuaijiang Zhao, Tingwei Guo, Bajian Xiang, Tongtang Wan, Qiang Niu, Wei Zou, and Xiangang Li, “Advancing speech language models by scaling supervised fine-tuning with over 60,000 hours of synthetic speech dialogue data,” arXiv preprint arXiv:2412.01078 , 2024

arXiv 2024
[7]

Naturalspeech 2: Latent diffusion models are natural and zero- shot speech and singing synthesizers,

Kai Shen, Zeqian Ju, Xu Tan, Eric Liu, Yichong Leng, Lei He, and et al., “Naturalspeech 2: Latent diffusion models are natural and zero- shot speech and singing synthesizers,” in ICLR, 2024

work page 2024
[8]

Dailydialog: A manually labelled multi-turn dialogue dataset,

Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu, “Dailydialog: A manually labelled multi-turn dialogue dataset,” in IJCNLP, 2017

work page 2017

Show all 18 references

[9]

Anygpt: Unified multimodal LLM with discrete sequence modeling,

Jun Zhan, Junqi Dai, Jiasheng Ye, Yunhua Zhou, Dong Zhang, Zhigeng Liu, and et al., “Anygpt: Unified multimodal LLM with discrete sequence modeling,” in ACL, 2024

2024
[10]

Explain-analyze-generate: A sequential multi-agent collaboration method for complex reasoning,

Wenyuan Gu, Jiale Han, Haowen Wang, Xiang Li, and Bo Cheng, “Explain-analyze-generate: A sequential multi-agent collaboration method for complex reasoning,” in COLING, 2025

2025
[11]

Encouraging divergent thinking in large language models through multi-agent debate,

Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, and et al., “Encouraging divergent thinking in large language models through multi-agent debate,” in EMNLP, 2024

2024
[12]

Chateval: Towards better llm-based evaluators through multi-agent debate,

Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, and et al., “Chateval: Towards better llm-based evaluators through multi-agent debate,” in ICLR. 2024, OpenReview.net

2024
[13]

Recon- cile: Round-table conference improves reasoning via consensus among diverse llms,

Justin Chih-Yao Chen, Swarnadeep Saha, and Mohit Bansal, “Recon- cile: Round-table conference improves reasoning via consensus among diverse llms,” in ACL, Lun-Wei Ku, Andre Martins, and Vivek Srikumar, Eds., 2024

2024
[14]

Speechagents: Human-communication simulation with multi-modal multi-agent systems,

Dong Zhang, Zhaowei Li, Pengyu Wang, Xin Zhang, Yaqian Zhou, and Xipeng Qiu, “Speechagents: Human-communication simulation with multi-modal multi-agent systems,” CoRR, 2024

2024
[15]

Common voice: A massively-multilingual speech corpus,

Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis M. Tyers, and Gregor Weber, “Common voice: A massively-multilingual speech corpus,” in LREC, 2020

2020
[16]

UTMOS: utokyo-sarulab system for voicemos challenge 2022,

Takaaki Saeki, Detai Xin, Wataru Nakata, Tomoki Koriyama, Shinno- suke Takamichi, and Hiroshi Saruwatari, “UTMOS: utokyo-sarulab system for voicemos challenge 2022,” in Interspeech, 2022

2022
[17]

Natural TTS synthesis by conditioning wavenet on MEL spectro- gram predictions,

Jonathan Shen, Ruoming Pang, Ron J. Weiss, Mike Schuster, Navdeep Jaitly, Zongheng Yang, Zhifeng Chen, Yu Zhang, Yuxuan Wang, R. J. Skerry-Ryan, Rif A. Saurous, Yannis Agiomyrgiannakis, and Yonghui Wu, “Natural TTS synthesis by conditioning wavenet on MEL spectro- gram predict...

2018
[18]

Fastspeech 2: Fast and high-quality end-to-end text to speech,

Yi Ren, Chenxu Hu, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu, “Fastspeech 2: Fast and high-quality end-to-end text to speech,” in ICLR, 2021

2021

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Chatgpt-edss: Empathetic dialogue speech synthesis trained from chatgpt-derived context word embeddings,

Yuki Saito, Shinnosuke Takamichi, Eiji Iimori, Kentaro Tachibana, and Hiroshi Saruwatari, “Chatgpt-edss: Empathetic dialogue speech synthesis trained from chatgpt-derived context word embeddings,” in Interspeech, Naomi Harte, Julie Carson-Berndsen, and Gareth Jones, Eds. 2023, ISCA

work page 2023

[2] [2]

The voice bank corpus: Design, collection and data analysis of a large regional accent speech database,

Christophe Veaux, Junichi Yamagishi, and Simon King, “The voice bank corpus: Design, collection and data analysis of a large regional accent speech database,” in COCOSDA, 2013

work page 2013

[3] [3]

Wenetspeech4tts: A 12,800-hour mandarin tts corpus for large speech generation model benchmark,

Linhan Ma, Dake Guo, Kun Song, Yuepeng Jiang, Shuai Wang, Li- umeng Xue, Weiming Xu, Huan Zhao, Binbin Zhang, and Lei Xie, “Wenetspeech4tts: A 12,800-hour mandarin tts corpus for large speech generation model benchmark,” arXiv preprint arXiv:2406.05763 , 2024

arXiv 2024

[4] [4]

Dailytalk: Spoken dialogue dataset for conversational text-to-speech,

Keon Lee, Kyumin Park, and Daeyoung Kim, “Dailytalk: Spoken dialogue dataset for conversational text-to-speech,” in ICASSP, 2023

work page 2023

[5] [5]

The fisher corpus: a resource for the next generations of speech-to-text,

Christopher Cieri, David Miller, and Kevin Walker, “The fisher corpus: a resource for the next generations of speech-to-text,” in LREC, 2004

work page 2004

[6] [6]

Advancing speech language models by scaling supervised fine-tuning with over 60,000 hours of synthetic speech dialogue data,

Shuaijiang Zhao, Tingwei Guo, Bajian Xiang, Tongtang Wan, Qiang Niu, Wei Zou, and Xiangang Li, “Advancing speech language models by scaling supervised fine-tuning with over 60,000 hours of synthetic speech dialogue data,” arXiv preprint arXiv:2412.01078 , 2024

arXiv 2024

[7] [7]

Naturalspeech 2: Latent diffusion models are natural and zero- shot speech and singing synthesizers,

Kai Shen, Zeqian Ju, Xu Tan, Eric Liu, Yichong Leng, Lei He, and et al., “Naturalspeech 2: Latent diffusion models are natural and zero- shot speech and singing synthesizers,” in ICLR, 2024

work page 2024

[8] [8]

Dailydialog: A manually labelled multi-turn dialogue dataset,

Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu, “Dailydialog: A manually labelled multi-turn dialogue dataset,” in IJCNLP, 2017

work page 2017