Pith. sign in

REVIEW 3 major objections 5 minor 24 references

Style consistency in role-play speech can be measured by the log-probability a frozen audio-language model assigns to the reference audio given the candidate generation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 06:31 UTC pith:BN25L3TV

load-bearing objection MCLP is a clever and genuinely new metric for speaking-style consistency, but it may track timbre and acoustic match as much as style; the paper is worth refereeing and needs a timbre-controlled validation. the 3 major comments →

arxiv 2601.22661 v2 pith:BN25L3TV submitted 2026-01-30 cs.SD

Evaluating and Rewarding LALMs for Expressive Role-Play TTS via Mean Continuation Log-Probability

Yong Ren , Jingbei Li , Haiyang Sun , Yujie Chen , Cheng Yi , Yechang Huang , Hao Gu , Ye Bai
show 1 more author
Xuerui Yang
This is my paper
classification cs.SD
keywords Mean Continuation Log-Probabilityrole-play TTSstyle consistency metriclarge audio language modelsreinforcement learning rewardGRPOspeech style evaluationspeech reward design
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper's central claim is that speaking style in role-play text-to-speech can be quantified by a single number: the average log-probability a frozen audio-language model assigns to ground-truth speech tokens after being given the transcript, the candidate generation, and the transcript again. This 'Mean Continuation Log-Probability' is argued to act as a proxy for stylistic continuity because the transcript is identical for both the candidate and the reference, so whatever extra likelihood the model gives must reflect style rather than content. The authors test this in two ways: MCLP predicted human MOS judgments about style, with win rates above 0.8 when the score difference exceeded 0.1, and using MCLP as a reward in a GRPO pipeline improved role-play speech both objectively and subjectively, reaching MOS 3.646 and CER 1.13%.

Core claim

Under the assumption that a pretrained large audio-language model is a near-perfect density estimator for speech, MCLP approximates the mutual information between a candidate utterance and a ground-truth reference. Because both are conditioned on the same transcript, the paper argues that the remaining likelihood signal is dominated by style attributes (emotion, paralinguistic cues, rhythm). The paper therefore treats the model's 'reverse continuation' probability — how likely the reference sounds after hearing the candidate — as a continuous, dense measure of stylistic consistency, and feeds that score back into the TTS model via reinforcement learning with a thresholded content penalty to

What carries the argument

The central object is the frozen LALM used as a conditional density model over interleaved text/audio (TA4) tokens. MCLP computes the mean per-token log-likelihood of the ground-truth audio tokens given the constructed context H=[w, z_eval, w] and previous ground-truth tokens. The information-theoretic argument in Eq. (3) — maximizing expected log-likelihood is asymptotically equivalent to minimizing conditional entropy H(Z_gt|Z_eval) — is what turns a likelihood number into a style metric.

Load-bearing premise

The load-bearing premise is that, because the transcript is identical for candidate and reference in Eq. (1), differences in MCLP are dominated by speaking style rather than by timbre, audio quality, or codec affinity of the frozen LALM.

What would settle it

Gather a set of utterances with identical transcripts and identical style but different speakers (different timbre), and a matched set with same speaker and different styles. If MCLP differences across timbre changes are comparable to or larger than differences across style changes, the metric is measuring voice identity rather than style; if MCLP correlates with human style judgments when timbre is held constant via resynthesis, the metric survives.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If MCLP is right, researchers gain a reference-based automatic metric for expressive TTS that requires no human raters and no emotion classifier, filling a gap the paper identifies.
  • Used as a reward in GRPO, MCLP gives a dense, smooth training signal that improves style alignment; the ablation shows that adding the CER constraint is necessary to prevent the model from collapsing into inaudible or gibberish output.
  • The correlation result (win rate > 0.8 at ΔMCLP>0.1) implies that MCLP can rank candidate utterances by style quality, enabling development-set selection and early stopping on style objectives.
  • The hybrid reward system produced the best objective results in the paper: CER 1.130% and Pinyin WER 0.511% with audio history, plus a subjective MOS of 3.646, approaching the ground-truth MOS of 4.411 while far exceeding the base model's 1.707.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial: If the style signal is as strong as claimed, MCLP could generalize to any conditional speech generation task (expressive TTS, dubbing, voice acting) where a reference utterance is available, not just role-play multi-turn settings.
  • Editorial: The metric's dependence on a specific tokenizer and a specific pretrained LALM is likely to matter; a portability check would test whether MCLP computed by a different LALM or resolved to a different codec still correlates with human style judgments.
  • Editorial: Because the paper's own human evaluation resynthesized audios to disentangle timbre, MCLP might be expected to correlate with style even when timbre is held fixed; verifying this directly would reduce the main confound.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Mean Continuation Log-Probability (MCLP), an objective metric for stylistic consistency in role-play TTS (RP-TTS). MCLP is defined as the average log-probability of ground-truth audio tokens under a frozen LALM, conditioned on a context of transcript, candidate audio, and repeated transcript. The authors argue that, because the transcript is shared, MCLP isolates style and can therefore serve both as an evaluation metric and as a reward signal. They integrate MCLP (with a CER penalty) into GRPO to optimize a Step-Audio-2-based RP-TTS model, and construct a large multi-turn RP-TTS dataset from drama videos. Experiments claim that MCLP is well aligned with human judgments (Fig. 5, win rate >0.8 for ΔMCLP > 0.1) and that hybrid RL improves both style (MCLP, MOS) and content (CER, WER) over strong baselines.

Significance. If the identifiability assumption underlying MCLP holds, the paper provides a dense, automatic, and interpretable style metric that fills a clear gap in expressive TTS evaluation and RL. The dataset and code release are also valuable contributions. However, the core claim that MCLP tracks style rather than low-level acoustic similarity is not yet established; the current experiments do not control for timbre, recording conditions, or codec artifacts. A resynthesis-controlled validation would substantially increase confidence. The paper also contains a formal error in the information-theoretic derivation (Eq. 3) that should be corrected. The RL results partially restate the optimized objective, so independent human assessment of the RL improvements is needed.

major comments (3)
  1. [§4.1, Eq. (1); §6.1 Table 2 footnote] The central claim that MCLP isolates style rests on the assertion that, since the transcript is shared, 'the variation in MI is dominated by style attributes.' Eq. (1) contains no term controlling for timbre, speaker identity, recording environment, or codec artifacts. The footnote in Table 2 shows the authors resynthesize stimuli for human scoring to 'disentangle style from timbre and audio quality,' but no analogous control is applied to MCLP scores in Tables 2–3 or Fig. 5. The reward-hacking result in §6.3 (style-only optimization yields high MCLP with repetitive acoustic patterns) further suggests the metric is sensitive to token-level acoustic regularities. A resynthesis-controlled experiment (e.g., recomputing MCLP after fixing timbre via a reference prompt) is needed to support the style-specificity claim.
  2. [§4.1, Eq. (3)] The information-theoretic identity in Eq. (3) has an incorrect sign. The standard relation is E_pdata[log pθ] = -H(pdata) - D_KL(pdata || pθ), so the KL divergence should be subtracted, not added. With the printed sign, the conclusion that higher MCLP implies lower conditional entropy does not follow. Although the assumption that the frozen LALM is a strong density estimator (small KL) makes the claim plausible, the derivation as written is formally incorrect and should be fixed.
  3. [§4.3 Eq. (7); §6.2 Table 3] MCLP is used both as the evaluation metric and as the style reward (Eq. 7). Consequently, the MCLP improvements in Table 3 after RL are partly a restatement of the optimized objective. The non-circular evidence is the human correlation in Fig. 5, but that figure is computed on a single small test set (31 samples, 30 valid responses). To demonstrate that the RL gains reflect genuine stylistic improvement, the authors should report human MOS or preference scores for the ablated models in Table 3 (at least SFT vs. RL). Otherwise, the perceived style benefit of the hybrid reward is not independently established.
minor comments (5)
  1. [Eq. (1)] The notation z_gt^A is used in the definition of MCLP before the set of audio tokens is defined. Please move the definition of A before Eq. (1) or add an explicit sentence.
  2. [§5.1 and Figure 3] The text refers to 'WenetSpeech' while Figure 3 labels the source as 'WenetSpeechCorpus.' Please unify the terminology.
  3. [§5.3] The 'internal style classification model' used to filter out neutral speech is not described. Please provide details or a citation; otherwise, the reproducibility of the RL dataset is limited.
  4. [Table 2] The 'Ground Truth' row reports only MOS; objective CER/WER/MCLP values would be useful as a reference for interpreting the proposed metric.
  5. [General] The paper's claims are specific to Step-Audio-2 as the scorer. Please discuss whether MCLP is expected to generalize to other LALMs or whether the metric is only meaningful with this particular tokenizer/model.

Circularity Check

1 steps flagged

RL style-score gains are partly self-referential (R_style = MCLP), but the central MCLP–human MOS alignment claim rests on independent subjective evidence.

specific steps
  1. fitted input called prediction [§4.3 Eq. (7); §6.3 'Content-Only Optimization' / Table 3]
    "We directly utilize the MCLP score derived in §4.1 with a bias term C ... Rstyle = MCLP(zroll,zgt)+C ... Content-Only Optimization: ... removing the style reward results in an extremely safe model with the lowest CER (0.783%) but a regressed style score (-4.752), which is even worse than the SFT baseline."

    Eq. (7) defines the RL style reward as MCLP (up to constant C). Table 3 then uses MCLP as the 'style score' to demonstrate reward effectiveness: the policy trained with MCLP improves MCLP, and removing the MCLP reward regresses it. These MCLP differences are restatements of the training objective by construction, not independent measurements of speaking style. The human-MOS correlation in Fig. 5 is an independent check and keeps the central metric claim from being fully circular.

full rationale

The central derivation of MCLP in Eq. (1) is direct and not circular: it defines a continuation log-probability and then tests it against human MOS in Fig. 5, an external subjective benchmark. There is no load-bearing self-citation chain or imported uniqueness theorem: Step-Audio-2 (Wu et al., 2025a) is selected as a concrete scorer/backbone, not invoked to rule out alternatives. The one construction-driven circularity is in the RL evaluation: because R_style = MCLP (Eq. 7), the MCLP improvements/regressions in Table 3 are expected consequences of the optimization objective rather than independent style evidence. I would not raise the score further because the human-alignment claim and the MOS/ CER improvements are externally grounded. Separate non-circular correctness risks: Eq. (3) has a sign inconsistency in the KL term (the standard identity uses −D_KL, not +D_KL), and the §4.1 assertion that fixing the transcript makes MI variation 'dominated by style attributes' is an untested identifiability assumption—Eq. (1) has no explicit control for timbre/audio quality, and the ⋄ resynthesis control in Table 2 is applied to human stimuli, not to MCLP scores. These are validity concerns, not constructional circularity, so they are reflected in the moderate score rather than a high one.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 1 invented entities

The metric's validity rests almost entirely on three unmeasured premises: LALMs encode style latently; the chosen LALM is a strong density estimator on the unusual [w, z_eval, w] conditioning (KL ≈ 0); and transcript equality removes all non-style confounds. The RL and data claims then add hand-set constants (C, λ, τ) and an undisclosed style filter. No genuinely new entities are introduced beyond MCLP, whose only falsifiable handle is the in-paper human study. Standard training hyperparameters (learning rates, batch size, warmup) and GRPO settings (G = 8, β = 0.001) are not counted as free parameters.

free parameters (4)
  • MCLP bias C = 15.0
    Eq. (7): shifts MCLP into a positive reward range; hand-set, no sensitivity analysis.
  • CER penalty coefficient λ = 10.0
    Eq. (8): weights the content-fidelity penalty in the hybrid reward; hand-set, no sensitivity analysis.
  • CER gating threshold τ = 0.2
    Eq. (9): rollouts with CER > 20% receive zero reward; hand-set; defines the intelligibility-before-style curriculum.
  • 'Not Neutral' style filter (internal classification model) = undisclosed
    §5.3: RL data restricted to samples an undisclosed internal model labels as stylized/emotional; threshold and model accuracy unknown — a hidden selection parameter.
axioms (6)
  • domain assumption Pretrained LALMs implicitly encode a continuous latent space of audio styles learned from large-scale speech corpora
    Foundational hypothesis stated in §1 and §4.1; the entire MCLP-as-style-proxy argument rests on it; no evidence is given in the paper beyond citing Step-Audio-2's pretraining.
  • domain assumption The frozen LALM is a strong density estimator for P(z_gt | w, z_eval, w), i.e., D_KL(P_data || P_θ) ≈ 0
    Eq. (3) needs the KL term small and roughly constant so that MCLP approximates −H(Z_gt|Z_eval); asserted in §4.1, never measured (and the printed equation has a sign inconsistency).
  • domain assumption Shared transcript isolates style: 'variation in MI is dominated by style attributes' (timbre and acoustic similarity are negligible)
    §4.1; the metric's style-specificity claim. Table 2's ⋄ footnote shows the authors believed timbre/audio quality had to be controlled — for the subjective study only, not for Eq. (1).
  • ad hoc to paper The context construction H = [w, z_eval, w] (transcript, candidate audio, repeated transcript) makes continuation likelihood a valid style comparator
    Eq. (1); a design choice with no justification for the repeated transcript and no ablation of alternative contexts (e.g., [z_eval, w] or w alone).
  • domain assumption ASR (an author-affiliated Step-Audio model) gives accurate CER for both the RL reward and content evaluation
    §4.3 and §6.1; ASR errors propagate into the gated reward and into the reported CER/WER; no error analysis or cross-ASR check.
  • standard math GRPO clipped-ratio objective with group-normalized advantages is a valid optimizer for this non-differentiable reward
    Eqs. (5)-(6), citing DeepSeekMath (Shao et al., 2024); standard RL method, accepted as-is.
invented entities (1)
  • MCLP (Mean Continuation Log-Probability) independent evidence
    purpose: Objective evaluation metric and RL reward quantifying stylistic consistency of generated speech (Eq. 1, §4.1)
    MCLP makes falsifiable predictions about human style preferences, tested in the paper's win-rate analysis (Fig. 5). The independent handle is weak: one 30-sample in-paper study, and the score is defined by an author-affiliated model (Step-Audio-2), so cross-family generalization is untested.

pith-pipeline@v1.3.0-alltime-deepseek · 13106 in / 32103 out tokens · 296434 ms · 2026-08-03T06:31:20.013315+00:00 · methodology

0 comments
read the original abstract

Recent advances in Large Audio Language Models (LALMs) have extended Text-to-Speech (TTS) to interactive role-play scenarios, which demand high expressiveness and strict adherence to role-play instructions. However, existing models struggle to maintain stylistic consistency with character profiles and scene descriptions across multi-turn dialogues. A critical bottleneck is the lack of objective metrics for quantifying speaking style. To bridge this gap, we propose Mean Continuation Log-Probability (MCLP) as both an evaluation metric and a reward signal, validated on LALM-based Role-Play TTS (RP-TTS) tasks. MCLP leverages the in-context learning capability of pretrained LALMs to measure the likelihood of ground-truth speech tokens conditioned on a contextual history consisting of the transcript, generated speech, and repeated transcript, serving as a proxy for stylistic continuity. Furthermore, we employ MCLP as a reinforcement learning reward to enhance the style alignment between generated speech and role-play instructions. To support this task, we construct a large-scale RP-TTS dataset with rich scene and character annotations. Experiments demonstrate that MCLP is well aligned with human judgments of stylistic consistency and serves as an effective reward for improving RP-TTS, leading to consistent gains in both objective metrics and subjective evaluations. Our code is publicly available at https://github.com/y-ren16/MCLP.

Figures

Figures reproduced from arXiv: 2601.22661 by Cheng Yi, Haiyang Sun, Hao Gu, Jingbei Li, Xuerui Yang, Ye Bai, Yechang Huang, Yong Ren, Yujie Chen.

Figure 1
Figure 1. Figure 1: The overall framework of our proposed method. (a) The pretraining stage of the LALM for computing MCLP. (b) The LALM is fine-tuned to generate interleaved TA4 tokens conditioned on a structured prompt containing scene descriptions, character profiles, and dialogue history. (c) The hybrid reward function for GRPO that synergizes the style-centric MCLP signal with a content-fidelity penalty (CER) to prevent … view at source ↗
Figure 2
Figure 2. Figure 2: An example for Role-Play TTS. 3. Problem Preliminaries We formulate RP-TTS as a context-aware conditional gen￾eration problem. Unlike standard TTS, which focuses on text-content fidelity, RP-TTS demands that the synthesized speech aligns with complex textual definitions regarding scene atmosphere and character persona, as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Data Curation and Annotation Pipeline. We construct the RP-TTS dataset from the WenetSpeech corpus via a multi-stage process. Hybrid Reward Formulation. A fundamental challenge in RP-TTS lies in balancing the trade-off between expres￾siveness and intelligibility. Naive optimization of a single objective is prone to reward hacking (Skalse et al., 2022): over-optimizing for style frequently compromises seman… view at source ↗
Figure 5
Figure 5. Figure 5: Win rate vs. ∆MCLP Score. The win rate (probability that a higher MCLP score predicts a higher human MOS) across different ∆MCLP bins. Content Instruction Following. We first evaluate the ca￾pability of different models to strictly adhere to textual content instructions within the RP-TTS framework. As shown in [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 14 linked inside Pith

  1. [1]

    Tts-1 technical report.arXiv preprint arXiv:2507.21138,

    Atamanenko, O., Chalova, A., Coombes, J., Cope, N., Dang, P., Deng, Z., Du, J., Ermolenko, M., Fan, F., Feng, Y ., et al. Tts-1 technical report.arXiv preprint arXiv:2507.21138,

    Pith/arXiv arXiv

  2. [5]

    Cosyvoice 2: Scalable streaming speech synthesis with large language models.arXiv preprint arXiv:2412.10117,

    Du, Z., Wang, Y ., Chen, Q., Shi, X., Lv, X., Zhao, T., Gao, Z., Yang, Y ., Gao, C., Wang, H., et al. Cosyvoice 2: Scalable streaming speech synthesis with large language models.arXiv preprint arXiv:2412.10117,

    Pith/arXiv arXiv

  3. [7]

    Differentiable reward optimization for llm based tts system.arXiv preprint arXiv:2507.05911, 2025a

    Gao, C., Du, Z., and Zhang, S. Differentiable reward optimization for llm based tts system.arXiv preprint arXiv:2507.05911, 2025a. Gao, X., Zhang, C., Chen, Y ., Zhang, H., and Chen, N. F. Emo-dpo: Controllable emotional speech synthe- sis through direct preference optimization. InICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Sig...

    Pith/arXiv arXiv 2025

  4. [8]

    Prompttts: Controllable text-to-speech with text descriptions

    Guo, Z., Leng, Y ., Wu, Y ., Zhao, S., and Tan, X. Prompttts: Controllable text-to-speech with text descriptions. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5. IEEE,

    2023

  5. [9]

    Qwen3-tts technical report.arXiv preprint arXiv:2601.15621, 2026a

    Hu, H., Zhu, X., He, T., Guo, D., Zhang, B., Wang, X., Guo, Z., Jiang, Z., Hao, H., Guo, Z., et al. Qwen3-tts technical report.arXiv preprint arXiv:2601.15621, 2026a. Hu, J., Chen, H., Ma, L., Guo, D., Zhan, Q., Li, W., Zhang, H., Xia, K., Zhang, Z., Tian, W., et al. V oic- esculptor: Your voice, designed by you.arXiv preprint arXiv:2601.10629, 2026b. Hua...

    Pith/arXiv arXiv

  6. [10]

    Speechrole: A large-scale dataset and benchmark for evaluating speech role-playing agents

    Jiang, C., Sun, J., Cao, Y ., Zhuang, J., Li, H., Fan, B., Ji, T., Gui, T., and Zhang, Q. Speechrole: A large-scale dataset and benchmark for evaluating speech role-playing agents. arXiv preprint arXiv:2508.02013,

    arXiv

  7. [11]

    Prompttts 2: Describing and generating voices with text prompt.arXiv preprint arXiv:2309.02285,

    Leng, Y ., Guo, Z., Shen, K., Tan, X., Ju, Z., Liu, Y ., Liu, Y ., Yang, D., Zhang, L., Song, K., et al. Prompttts 2: Describing and generating voices with text prompt.arXiv preprint arXiv:2309.02285,

    Pith/arXiv arXiv

  8. [12]

    Au- diorole: An audio dataset for character role-playing in large language models.arXiv preprint arXiv:2509.23435,

    Li, W., Jiao, X., Chang, Y ., Zhang, G., and Guo, Y . Au- diorole: An audio dataset for character role-playing in large language models.arXiv preprint arXiv:2509.23435,

    Pith/arXiv arXiv

  9. [13]

    Reinforcement learning for emotional text-to-speech synthesis with improved emo- tion discriminability.arXiv preprint arXiv:2104.01408,

    Liu, R., Sisman, B., and Li, H. Reinforcement learning for emotional text-to-speech synthesis with improved emo- tion discriminability.arXiv preprint arXiv:2104.01408,

    Pith/arXiv arXiv

  10. [15]

    Hybrid transform- ers for music source separation

    9 Submission and Formatting Instructions for ICML 2026 Rouard, S., Massa, F., and D´efossez, A. Hybrid transform- ers for music source separation. InICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5. IEEE,

    2026

  11. [16]

    Deepseekmath: Push- ing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300,

    Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y ., Wu, Y ., et al. Deepseekmath: Push- ing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300,

    Pith/arXiv arXiv

  12. [17]

    Speech-drame: A frame- work for human-aligned benchmarks in speech role-play

    Shi, J., Han, J., Lu, Y ., Pascual, S., Wu, P., Cui, C., Watan- abe, S., Weng, C., and Zhou, C. Speech-drame: A frame- work for human-aligned benchmarks in speech role-play. arXiv preprint arXiv:2511.01261,

    arXiv

  13. [19]

    Rrpo: Robust reward policy optimization for llm-based emotional tts.arXiv preprint arXiv:2512.04552,

    Wang, C., Gao, C., Xiang, Y ., Du, Z., An, K., Zhao, H., Chen, Q., Li, X., Gao, Y ., and Li, Y . Rrpo: Robust reward policy optimization for llm-based emotional tts.arXiv preprint arXiv:2512.04552,

    arXiv

  14. [20]

    Step-audio 2 technical report.arXiv preprint arXiv:2507.16632, 2025a

    Wu, B., Yan, C., Hu, C., Yi, C., Feng, C., Tian, F., Shen, F., Yu, G., Zhang, H., Li, J., et al. Step-audio 2 technical report.arXiv preprint arXiv:2507.16632, 2025a. Wu, W., Cao, L., Wu, X., Lin, Z., Niu, R., Li, J., and Wu, Z. V oxrole: A comprehensive benchmark for evaluat- ing speech-based role-playing agents.arXiv preprint arXiv:2509.03940, 2025b. Ya...

    Pith/arXiv arXiv

  15. [21]

    Llasa: Scaling train-time and inference-time compute for llama-based speech synthesis.arXiv preprint arXiv:2502.04128,

    Ye, Z., Zhu, X., Chan, C.-M., Wang, X., Tan, X., Lei, J., Peng, Y ., Liu, H., Jin, Y ., Dai, Z., et al. Llasa: Scaling train-time and inference-time compute for llama-based speech synthesis.arXiv preprint arXiv:2502.04128,

    Pith/arXiv arXiv

  16. [22]

    Wenetspeech: A 10000+ hours multi-domain mandarin corpus for speech recognition

    Zhang, B., Lv, H., Guo, P., Shao, Q., Yang, C., Xie, L., Xu, X., Bu, H., Chen, X., Zeng, C., et al. Wenetspeech: A 10000+ hours multi-domain mandarin corpus for speech recognition. InICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6182–6186. IEEE,

    2022

  17. [23]

    Mimo-audio: Audio language models are few-shot learners.arXiv preprint arXiv:2512.23808, 2025a

    Zhang, D., Wang, G., Xue, J., Fang, K., Zhao, L., Ma, R., Ren, S., Liu, S., Guo, T., Zhuang, W., et al. Mimo-audio: Audio language models are few-shot learners.arXiv preprint arXiv:2512.23808, 2025a. Zhang, H., Luo, R., Liu, X., Wu, Y ., Lin, T.-E., Zeng, P., Qu, Q., Fang, F., Yang, M., Gao, L., et al. Omnicharacter: Towards immersive role-playing agents ...

    arXiv

  18. [24]

    drama_name

    10 Submission and Formatting Instructions for ICML 2026 A. Prompts Used in Data Construction In this section, we provide the specific prompts used in our data construction pipeline, including metadata extraction, scene description generation, and character profile inference. A.1. Metadata Extraction Prompt We employ DeepSeek-R1 to infer the drama series n...

    2026

  19. [2021]

    Ov-instructtts: Towards open-vocabulary instruct text-to-speech.arXiv preprint arXiv:2601.01459,

    Ren, Y ., Yi, J., Tao, J., Sun, H., Wen, Z., Gu, H., Xu, L., and Bai, Y . Ov-instructtts: Towards open-vocabulary instruct text-to-speech.arXiv preprint arXiv:2601.01459,

    arXiv

  20. [2022]

    Au- diobox: Unified audio generation with natural language prompts.arXiv preprint arXiv:2312.15821,

    Vyas, A., Shi, B., Le, M., Tjandra, A., Wu, Y .-C., Guo, B., Zhang, J., Zhang, X., Adkins, R., Ngan, W., et al. Au- diobox: Unified audio generation with natural language prompts.arXiv preprint arXiv:2312.15821,

    Pith/arXiv arXiv

  21. [2023]

    Flexivoice: Enabling flexible style control in zero- shot tts with natural language instructions.arXiv preprint arXiv:2601.04656,

    Chen, D., Zhang, X., Wang, Y ., Dai, K., Ma, L., and Wu, Z. Flexivoice: Enabling flexible style control in zero- shot tts with natural language instructions.arXiv preprint arXiv:2601.04656,

    arXiv

  22. [2024]

    Cosyvoice 3: Towards in-the-wild speech generation via scaling-up and post-training.arXiv preprint arXiv:2505.17589,

    Du, Z., Gao, C., Wang, Y ., Yu, F., Zhao, T., Wang, H., Lv, X., Wang, H., Ni, C., Shi, X., et al. Cosyvoice 3: Towards in-the-wild speech generation via scaling-up and post-training.arXiv preprint arXiv:2505.17589,

    Pith/arXiv arXiv

  23. [2025]

    org/abs/2511.21631

    URL https://arxiv. org/abs/2511.21631. Bredin, H. pyannote.audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe. InProc. INTER- SPEECH 2023,

    Pith/arXiv arXiv 2023

  24. [2026]

    Vall-e 2: Neural codec lan- guage models are human parity zero-shot text to speech synthesizers.arXiv preprint arXiv:2406.05370,

    Chen, S., Liu, S., Zhou, L., Liu, Y ., Tan, X., Li, J., Zhao, S., Qian, Y ., and Wei, F. Vall-e 2: Neural codec lan- guage models are human parity zero-shot text to speech synthesizers.arXiv preprint arXiv:2406.05370,

    Pith/arXiv arXiv