Pith. sign in

REVIEW 3 major objections 5 minor 45 references

A speech LLM learns when to respond, listen, or ignore in noisy multi-speaker scenes.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 01:52 UTC pith:A4B7K4S5

load-bearing objection Cocktail-Talker is a solid system paper with a genuinely new task formulation and a strong synthetic-data pipeline; the 0.93 accuracy is real but measures agreement with Gemini-generated labels, so treat 'socially appropriate' as an assumption rather than a demonstrated fact. the 3 major comments →

arxiv 2607.27756 v1 pith:A4B7K4S5 submitted 2026-07-30 cs.SD cs.CLcs.MMeess.AS

Cocktail-Talker: Multi-Speaker Dialog Modeling in Noisy Social Environments with Turn Action GRPO

Xilin Jiang , Riki Shimizu , Sukru Samet Dindar , Junkai Wu , Zhongweiyang Xu , Nima Mesgarani This is my paper
classification cs.SD cs.CLcs.MMeess.AS
keywords spoken dialog systemsmulti-speaker conversationcocktail party problemturn-takingaction tokensreinforcement learningspeech large language modelnoisy environments
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Cocktail-Talker aims to show that a spoken-dialog assistant can do more than answer; it can decide whether to answer at all. The paper trains a speech-based large language model to emit one of three turn-action tokens — respond, listen, or ignore — before each turn, given a raw mixture of overlapping speech and noise. With supervised finetuning plus group-relative policy optimization, the model reaches about 0.93 binary respond/silent accuracy on both seen and unseen synthetic environments, outperforming existing speech LLMs that lack an explicit silent mechanism. If this holds, spoken assistants could participate selectively in real social scenes rather than assuming every utterance is directed at them.

Core claim

Cocktail-Talker is built on a speech-based large language model that receives the continuous mixed audio of all preceding turns, plus optional metadata about who is present, and must generate one of three turn-action tokens before any response. Only the <|respond|> token is followed by a synthesized spoken reply; <|listen|> and <|ignore|> both produce silence. The paper reports that supervised finetuning with these tokens gives a modest improvement over a version without them, while adding GRPO — which scores each output for action correctness and format — raises binary respond/silent accuracy to about 0.93 on both seen and unseen synthetic environments and lifts respond recall from roughly

What carries the argument

The central object is a small action-token vocabulary — <|respond|>, <|listen|>, <|ignore|> — prepended to the model's output and used as targets during supervised finetuning and reinforcement learning. The reward in GRPO is the sum of an action-accuracy score (1 for a correct action, 0 otherwise) and a format-integrity score (1 if the output starts with exactly one action token and follows the structural rule), and because the action tokens are skipped when routing to the speech synthesizer, spoken quality is preserved. This token-based formulation converts an open-ended behavioral question — 'should I speak now?' — into a discrete classification the LLM can be optimized for.

Load-bearing premise

The load-bearing premise is that the LLM-generated action labels used as ground truth for both training and evaluation correctly capture when a real person would want the assistant to speak, listen, or ignore; if those labels do not match human social judgment, the reported accuracy measures consistency with the data generator rather than the quality of the assistant's behavior.

What would settle it

Have human annotators listen to a sample of the synthetic scenes and mark whether the assistant should respond, remain silent but attentive, or ignore; compare their labels with the model's actions. If human agreement with the model is near chance, or if retraining on human labels produces similar accuracy numbers, the current results reflect label self-consistency rather than socially appropriate turn-taking.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A spoken assistant can make turn-taking decisions directly from raw mixed audio, without explicit speaker separation or diarization, because the model is trained on the full unprocessed mixture.
  • GRPO-style reinforcement learning on action accuracy is an effective way to improve turn decisions: the gain comes mainly from recovering missed responses, not from changing what is said.
  • Noise robustness holds across the tested SNR range (down to 0–6 dB), with only a slight accuracy drop relative to clean audio.
  • The largest semantic degradation occurs when names and roles are removed, indicating that inferring the addressee from context and voice is the key bottleneck for deciding whether to respond.
  • Transfer to unseen environments is nearly as strong as to seen ones, suggesting the learned turn-taking behavior is not tied to the specific background sounds or topics seen in training.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • As an editorial extension: if the LLM-generated action labels do not match human judgments about appropriate silence, the reported accuracy is an internal-consistency measure; the natural next experiment is a human listening study comparing model actions with what a person would want in the same scene.
  • The current system assumes fixed turn boundaries and is not streaming, so it does not yet handle barge-in or overlapping speech; adapting the action-token idea to a streaming, overlap-tolerant model is a concrete testable direction.
  • Because the action-token formulation is model-agnostic, the same respond/listen/ignore training recipe could be applied to other speech LLMs; the paper itself notes the idea is not limited to its base model.
  • The finding that anonymity hurts more than noise suggests practical systems in unknown social settings need better addressee inference — perhaps by conditioning on voice characteristics or head orientation — before turn-taking will feel natural.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Cocktail-Talker, a speech LLM built on Qwen2.5-Omni-7B that is fine-tuned with supervised finetuning and GRPO to emit one of three turn-action tokens (respond, listen, ignore) before generating a spoken response. Training data are produced by Cocktail-DialogGen, a pipeline in which Gemini 3 Pro composes multi-speaker dialog logs with action labels, Qwen3-TTS synthesizes speech, and a deterministic assembler mixes in background noise from Freesound/DEMAND. On the paper's synthetic test sets, Cocktail-Talker reports approximately 0.93 macro F1 for turn action accuracy, outperforming several strong speech LLMs. Ablations show the action-token vocabulary and GRPO improve the decision, and robustness analyses cover SNR, number of speakers, anonymity of names/roles, conversation vibe, and context length.

Significance. If the action labels are trusted, this is a useful contribution: it addresses the under-explored 'speaking side' of multi-speaker conversation for speech LLMs, introduces a scalable annotated data pipeline, and demonstrates a practical training recipe with a clear code release. The ablations are informative — GRPO mainly recovers respond recall, and the model degrades gracefully in noise and transfers to unseen environment categories. The central uncertainty is external validity: the evaluation target is the same Gemini-generated label distribution used for training, and no human validation anchors the construct of socially appropriate turn-taking.

major comments (3)
  1. [Sections III and V] The ground-truth action labels used for training (Section IV, reward definition) and for evaluation (Table II, Fig. 4 'Oracle Action') are all produced by the same Gemini 3 Pro dialog composer in Cocktail-DialogGen. The paper claims socially appropriate turn-taking in the Abstract and Conclusion, but no human validation of these labels is reported. The only human step in Section III is verifying Freesound background recordings, not the turn-action annotations. Please add a human rater study on a sample of turns (e.g., 300–500) that measures agreement with the Gemini labels, or evaluate on an existing multi-party conversation corpus with human-annotated addressee/turn-taking behavior. Without such evidence, the 0.93 macro F1 measures internal consistency with the data generator rather than assistant quality, which is load-bearing for the central claim.
  2. [Section V, Tables II and III] No confidence intervals, bootstrap estimates, or significance tests are reported. Several key comparisons are based on point estimates from a single run. For example, the difference between SFT without action tokens and SFT with action tokens is 0.903 vs 0.901 (seen) and 0.907 vs 0.903 (unseen), well within likely sampling noise. Even the GRPO gain (+2.5 pp) may be seed-dependent. Please report means and standard deviations over at least three training seeds or bootstrap CIs over test items, and state whether reported differences are statistically reliable. This is needed to support the ablation and the claimed improvement from GRPO.
  3. [Section V, unseen environments] The 'unseen' environments are generated by the same Cocktail-DialogGen pipeline — same Gemini dialog composer, same Qwen3-TTS voices, same assembler and noise-mixing procedures. Thus the generalization result is limited to new environment categories within the same synthetic distribution; it does not provide evidence of transfer to real conversational audio. Please qualify the claim in Section V (Analysis, Fig. 5g) that the model 'generalizes well beyond the specific environments,' or supplement with a small real-recording evaluation if available.
minor comments (5)
  1. [Section III, Table I] Typo: 'V oice' should be 'Voice'. Also clarify how the 100 Freesound recordings per category were 'verified by humans' — what criterion was applied?
  2. [Section V, Table III] The header 'penalized / conditionalSCORES' is missing a space; this makes the table harder to read. Consider separating the columns more clearly.
  3. [Section IV, Input and Output Format] The text prompt includes a JSON snippet with a trailing comma (after 'role' in speaker_c) and an extra closing brace; ensure the example is syntactically valid as shown.
  4. [Section V, baselines] The exact prompts used for the four instruction-following baselines are not given. Since the comparison depends on how well the task is expressed, include the full prompt templates in an appendix.
  5. [General] Figure 5 has many subplots sharing one legend; some labels (e.g., 'CafeterTraffic' in panel g) are truncated. Increase font sizes and fix abbreviations for readability.

Circularity Check

0 steps flagged

No significant circularity; the Gemini-label benchmark limits external validity but the derivation is not circular.

full rationale

The paper's training and evaluation loop does not reduce to its own inputs by construction. The supervised targets are Gemini-generated action labels from Cocktail-DialogGen (Section III, Fig. 3), and the GRPO reward is an exact-match action-accuracy term (Section IV, reward description). The reported test numbers in Table II and Fig. 4 compare model outputs to held-out 'Oracle Action' labels from the same synthetic pipeline. This is a standard train/held-out evaluation on a synthetic distribution: the test labels are not the training instances, no parameter is fitted to the test labels, and the model's outputs are not used to redefine the labels. The fact that the oracle labels come from Gemini rather than human raters is a genuine external-validity concern about whether 'socially appropriate turn-taking' is well grounded, but it is not circularity under the definitions used here. The paper does not claim the action labels were independently human-validated; the reported accuracy is honestly an accuracy against its own generator's annotations. The self-citations ([13], [14], [25], etc.) appear only as related-work motivation and are not load-bearing in the training or evaluation argument, so there is no self-citation chain forcing the conclusion. The limitation section acknowledges fixed turn boundaries and missing non-audio cues but does not assert or rely on a circular step. Verdict: no significant circularity; score 0.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The central result rests on treating LLM-generated action labels and synthetic audio mixtures as valid proxies for real social behavior; neither has external grounding, so the correctness of the task itself is assumed rather than demonstrated.

free parameters (6)
  • GRPO group size G = 16
    Rollout count for advantage normalization; no sensitivity analysis reported.
  • LoRA rank / alpha = r=128, alpha=256
    Chosen for capacity on all linear layers; no sweep reported.
  • Metadata drop probability p = 0.5
    Robustness augmentation for missing metadata; no ablations reported.
  • Speaker energy levels = agent -20 dB RMS; others -20±5 dB RMS
    Manual mixing choices that shape acoustic difficulty.
  • SNR ranges = 0-3, 3-6, 6-9, 9-12 dB and clean
    Training distribution of five noise levels; only a coarse analysis in Fig. 5a.
  • Reward weights for action accuracy and format integrity = 1.0 / 1.0
    Sum to r in [0,2]; no weight tuning shown.
axioms (4)
  • domain assumption Gemini 3 Pro-generated action labels are correct ground truth for whether to respond, listen, or ignore.
    Introduced in Section III (Fig. 3) and used as SFT target and GRPO reward (Section IV); no human validation of action labels.
  • domain assumption Synthetic mixtures of LibriTTS/Qwen3-TTS speech, Freesound/DailyTalk background, and random SNR faithfully emulate real noisy multi-speaker social scenes.
    Section III; underlies all generalization claims, including the 10 unseen environments.
  • domain assumption Fixed turn boundaries and non-streaming input are acceptable scope; the mixture is trimmed to the end of the current utterance.
    Section IV input format and Section VI limitations; system cannot handle streaming or overlapping turn-taking.
  • domain assumption Qwen2.5-Omni-7B's frozen audio encoder and talker remain adequate for the new task after LoRA on the thinker.
    Section IV trainable components; no ablation of unfreezing alternatives.

pith-pipeline@v1.3.0-daily-deepseek · 12417 in / 11959 out tokens · 129987 ms · 2026-08-01T01:52:39.838906+00:00 · methodology

0 comments
read the original abstract

Spoken dialog systems are typically designed for clean, dyadic interactions in which a single user and an assistant take turns speaking. Real-world social conversations, however, are often more ambiguous: multiple speakers may participate in the same conversation amid irrelevant speech and background noise. Each utterance may be directed to the assistant, addressed to another speaker, or completely irrelevant. In such settings, the assistant must decide not only what to say, but also whether to speak at all. In this paper, we introduce Cocktail-Talker, a speech LLM framework for multi-speaker spoken dialog modeling in noisy social environments. We model the assistant's behavior with three action tokens: <|respond|>, <|listen|>, and <|ignore|>, placed before a response or silence. Cocktail-Talker is trained via supervised finetuning and reinforcement learning to generate the appropriate action token and, only in <|respond|> mode, a speech response. To prepare the training data, we develop Cocktail-DialogGen, an LLM-based data pipeline that simulates realistic multi-speaker dialogs with speaker roles across diverse social settings. Together, these components take a step toward spoken dialog systems that interact more naturally and selectively in complex social environments.

Figures

Figures reproduced from arXiv: 2607.27756 by Junkai Wu, Nima Mesgarani, Riki Shimizu, Sukru Samet Dindar, Xilin Jiang, Zhongweiyang Xu.

Figure 1
Figure 1. Figure 1: A three-speaker conversation example illustrating Cocktail-Talker’s [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The complete workflow of Cocktail-DialogGen, our simulation pipeline of multi-speaker conversations. Users specify the environment; Dialog composer [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: An example three-speaker dialog with speakers’ metadata, content [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Cocktail-Talker’s confusion matrices on seen (a,b) and unseen (c,d) en [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Cocktail-Talker (SFT+GRPO) performance by (a) SNR level, (b) number of speakers, (c) names/roles anonymity, (d) conversation vibe, (e) number [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 15 linked inside Pith

  1. [1]

    GPT-4o System Card,

    OpenAI, “GPT-4o System Card,” https://openai.com/index/gpt-4o- system-card/, Aug. 2024, accessed: 2026-06-22

    2024

  2. [2]

    Moshi: A speech-text foundation model for real-time dialogue,

    A. Defossez, L. Mazare, M. Orsini, A. Royer, P. Perez, H. Jegou, E. Grave, and N. Zeghidour, “Moshi: A speech-text foundation model for real-time dialogue,”arXiv preprint arXiv:2410.00037, 2024

    Pith/arXiv arXiv 2024

  3. [3]

    Personaplex: V oice and role control for full duplex conversational speech models,

    R. Roy, J. Raiman, S.-g. Lee, T.-D. Ene, R. Kirby, S. Kim, J. Kim, and B. Catanzaro, “Personaplex: V oice and role control for full duplex conversational speech models,”arXiv preprint arXiv:2602.06053, 2026

    arXiv 2026

  4. [4]

    Qwen2.5-omni technical report,

    J. Xu, Z. Guo, J. He, H. Hu, T. He, S. Bai, K. Chen, J. Wang, Y . Fan, K. Dang, B. Zhang, X. Wang, Y . Chu, J. Linet al., “Qwen2.5-omni technical report,”arXiv preprint arXiv:2503.20215, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Qwen3-omni technical report,

    J. Xu, Z. Guo, H. Hu, Y . Chu, X. Wang, J. Heet al., “Qwen3-omni technical report,”arXiv preprint arXiv:2509.17765, 2025

    Pith/arXiv arXiv 2025

  6. [6]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models,

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . Li, Y . Wuet al., “Deepseekmath: Pushing the limits of mathematical reasoning in open language models,”arXiv preprint arXiv:2402.03300, 2024

    Pith/arXiv arXiv 2024

  7. [7]

    Mini-omni: Language models can hear, talk while thinking in streaming,

    Z. Xie and C. Wu, “Mini-omni: Language models can hear, talk while thinking in streaming,”arXiv preprint arXiv:2408.16725, 2024

    Pith/arXiv arXiv 2024

  8. [8]

    Llama-omni: Seamless speech interaction with large language models,

    Q. Fang, S. Guo, Y . Zhou, Z. Ma, S. Zhang, and Y . Feng, “Llama-omni: Seamless speech interaction with large language models,”arXiv preprint arXiv:2409.06666, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    Qwen2-audio technical report,

    Y . Chu, J. Xu, Q. Yang, H. Wei, X. Wei, Z. Guo, Y . Leng, Y . Lv, J. He, J. Lin, C. Zhou, and J. Zhou, “Qwen2-audio technical report,”arXiv preprint arXiv:2407.10759, 2024

    Pith/arXiv arXiv 2024

  10. [10]

    Mini-omni2: Towards open-source gpt-4o with vi- sion, speech and duplex capabilities,

    Z. Xie and C. Wu, “Mini-omni2: Towards open-source gpt-4o with vi- sion, speech and duplex capabilities,”arXiv preprint arXiv:2410.11190, 2024

    Pith/arXiv arXiv 2024

  11. [11]

    Some experiments on the recognition of speech, with one and with two ears,

    E. C. Cherry, “Some experiments on the recognition of speech, with one and with two ears,”Journal of the acoustical society of America, vol. 25, pp. 975–979, 1953

    1953

  12. [12]

    The cocktail party problem: what is it? how can it be solved? and why should animal behaviorists study it?

    M. A. Bee and C. Micheyl, “The cocktail party problem: what is it? how can it be solved? and why should animal behaviorists study it?” Journal of comparative psychology, vol. 122, no. 3, p. 235, 2008

    2008

  13. [13]

    Just ASR + LLM? a study on speech large language models’ ability to identify and understand speaker in spoken dialogue,

    J. Wu, X. Fan, B.-R. Lu, X. Jiang, N. Mesgarani, M. Hasegawa-Johnson, and M. Ostendorf, “Just ASR + LLM? a study on speech large language models’ ability to identify and understand speaker in spoken dialogue,” in2024 IEEE Spoken Language Technology Workshop (SLT), 2024, pp. 1137–1143

    2024

  14. [14]

    Avmeme exam: A multimodal multilingual multicultural benchmark for llms’ contextual and cultural knowledge and thinking,

    X. Jiang, Q. Wang, J. Wu, X. He, Z. Xu, Y . Ma, M. Piao, K. Yang, X. Zheng, R. Shimizuet al., “Avmeme exam: A multimodal multilingual multicultural benchmark for llms’ contextual and cultural knowledge and thinking,”arXiv preprint arXiv:2601.17645, 2026

    arXiv 2026

  15. [15]

    MSU-Bench: Towards understanding the conversational multi-talker scenarios,

    S. Wang, Z. Sun, Z. Lin, C. Wang, Z. Pan, and L. Xie, “MSU-Bench: Towards understanding the conversational multi-talker scenarios,”arXiv preprint arXiv:2508.08155, 2025

    Pith/arXiv arXiv 2025

  16. [16]

    M3-slu: Evaluating speaker- attributed reasoning in multimodal large language models,

    Y . Kwon, T. Kang, H. Yoon, and C. Kim, “M3-slu: Evaluating speaker- attributed reasoning in multimodal large language models,”arXiv preprint arXiv:2510.19358, 2025

    arXiv 2025

  17. [17]

    Full-duplex-bench: A benchmark to evaluate full-duplex spoken dialogue models on turn-taking capabilities,

    G.-T. Lin, J. Lian, T. Li, Q. Wang, G. Anumanchipalli, A. H. Liu, and H.-y. Lee, “Full-duplex-bench: A benchmark to evaluate full-duplex spoken dialogue models on turn-taking capabilities,”arXiv preprint arXiv:2503.04721, 2025

    Pith/arXiv arXiv 2025

  18. [18]

    Full-duplex-bench v1. 5: Evaluating overlap handling for full-duplex speech models,

    G.-T. Lin, S.-Y . S. Kuan, Q. Wang, J. Lian, T. Li, S. Watanabe, and H.-y. Lee, “Full-duplex-bench v1. 5: Evaluating overlap handling for full-duplex speech models,” inICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2026, pp. 19 447–19 451

    2026

  19. [19]

    V oiceFilter: Targeted voice separation by speaker-conditioned spectrogram masking,

    Q. Wang, H. Muckenhirn, K. Wilson, P. Sridhar, Z. Wu, J. R. Hershey, R. A. Saurous, R. J. Weiss, Y . Jia, and I. Lopez Moreno, “V oiceFilter: Targeted voice separation by speaker-conditioned spectrogram masking,” inProceedings of Interspeech, 2019, pp. 2728–2732

    2019

  20. [20]

    Listen, chat, and remix: Text-guided soundscape remixing for enhanced auditory experience,

    X. Jiang, C. Han, Y . A. Li, and N. Mesgarani, “Listen, chat, and remix: Text-guided soundscape remixing for enhanced auditory experience,” IEEE Journal of Selected Topics in Signal Processing, 2025

    2025

  21. [21]

    Meanflow-tse: One-step generative target speaker extraction with mean flow,

    R. Shimizu, X. Jiang, and N. Mesgarani, “Meanflow-tse: One-step generative target speaker extraction with mean flow,”arXiv preprint arXiv:2512.18572, 2025

    arXiv 2025

  22. [22]

    Auxiliary interference speaker loss for target-speaker speech recognition,

    N. Kanda, S. Horiguchi, R. Takashima, Y . Fujita, K. Nagamatsu, and S. Watanabe, “Auxiliary interference speaker loss for target-speaker speech recognition,” inProceedings of Interspeech, 2019, pp. 236–240

    2019

  23. [23]

    End-to-end multi-speaker speech recognition using speaker embeddings and transfer learning,

    P. Denisov and N. T. Vu, “End-to-end multi-speaker speech recognition using speaker embeddings and transfer learning,” inProceedings of Interspeech, 2019, pp. 4425–4429

    2019

  24. [24]

    Conformer- based target-speaker automatic speech recognition for single-channel audio,

    Y . Zhang, K. C. Puvvada, V . Lavrukhin, and B. Ginsburg, “Conformer- based target-speaker automatic speech recognition for single-channel audio,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023, pp. 1–5

    2023

  25. [25]

    AAD-LLM: Neural attention-driven auditory scene understanding,

    X. Jiang, S. S. Dindar, V . Choudhari, S. Bickel, A. Mehta, G. M. McKhann, D. Friedman, A. Flinker, and N. Mesgarani, “AAD-LLM: Neural attention-driven auditory scene understanding,” inProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025, pp. 25 887–25 909

    2025

  26. [26]

    Focus then listen: An empirical study of plug-and-play audio enhancer for noise-robust large audio language models,

    H. Yin, Y . Xiao, Y . Kwon, T. Dang, and J.-W. Choi, “Focus then listen: An empirical study of plug-and-play audio enhancer for noise-robust large audio language models,”arXiv preprint arXiv:2603.04862, 2026

    Pith/arXiv arXiv 2026

  27. [27]

    LibriMix: An open-source dataset for generalizable speech separation,

    J. Cosentino, M. Pariente, S. Cornell, A. Deleforge, and E. Vincent, “LibriMix: An open-source dataset for generalizable speech separation,” arXiv preprint arXiv:2005.11262, 2020

    Pith/arXiv arXiv 2005

  28. [28]

    CHiME-6 challenge: Tackling multispeaker speech recognition for unsegmented recordings,

    S. Watanabe, M. Mandel, J. Barker, E. Vincent, A. Arora, X. Chang, S. Khudanpur, V . Manohar, D. Povey, D. Rajet al., “CHiME-6 challenge: Tackling multispeaker speech recognition for unsegmented recordings,” in6th International Workshop on Speech Processing in Everyday Environments (CHiME), 2020

    2020

  29. [29]

    The AMI meeting corpus: A pre-announcement,

    J. Carletta, S. Ashby, S. Bourban, M. Flynn, M. Guillemot, T. Hain, J. Kadlec, V . Karaiskos, W. Kraaij, M. Kronenthal, G. Lathoud, M. Lin- coln, A. Lisowska, I. McCowan, W. Post, D. Reidsma, and P. Wellner, “The AMI meeting corpus: A pre-announcement,” inMachine Learning for Multimodal Interaction, 2006, pp. 28–39

    2006

  30. [30]

    Speak or stay silent: Context-aware turn-taking in multi-party dialogue,

    K. Bhagtani, M. Anand, Y . C. Xu, and A. K. S. Yadav, “Speak or stay silent: Context-aware turn-taking in multi-party dialogue,”arXiv preprint arXiv:2603.11409, 2026

    arXiv 2026

  31. [31]

    Demand: a collection of multi- channel recordings of acoustic noise in diverse environments,

    J. Thiemann, N. Ito, and E. Vincent, “Demand: a collection of multi- channel recordings of acoustic noise in diverse environments,”(No Title), 2013

    2013

  32. [32]

    Dailytalk: Spoken dialogue dataset for conversational text-to-speech,

    K. Lee, K. Park, and D. Kim, “Dailytalk: Spoken dialogue dataset for conversational text-to-speech,” inICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  33. [33]

    Gemini 3 Pro Model Card,

    Google DeepMind, “Gemini 3 Pro Model Card,” https://deepmind.google/models/model-cards/gemini-3-pro, Nov. 2025, model released November 2025; model card last updated May 2026

    2025

  34. [34]

    Qwen3-tts technical report,

    H. Hu, X. Zhu, T. He, D. Guo, B. Zhang, X. Wang, Z. Guo, Z. Jiang, H. Hao, Z. Guoet al., “Qwen3-tts technical report,”arXiv preprint arXiv:2601.15621, 2026

    Pith/arXiv arXiv 2026

  35. [35]

    Libritts: A corpus derived from librispeech for text-to-speech,

    H. Zen, V . Dang, R. Clark, Y . Zhang, R. J. Weiss, Y . Jia, Z. Chen, and Y . Wu, “Libritts: A corpus derived from librispeech for text-to-speech,” inProc. Interspeech 2019, 2019, pp. 1526–1530

    2019

  36. [36]

    Lora: Low-rank adaptation of large language models

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chenet al., “Lora: Low-rank adaptation of large language models.” ICLR, vol. 1, no. 2, p. 3, 2022

    2022

  37. [37]

    Llamafactory: Unified efficient fine-tuning of 100+ language models,

    Y . Zheng, R. Zhang, J. Zhang, Y . Ye, and Z. Luo, “Llamafactory: Unified efficient fine-tuning of 100+ language models,” inProceedings of the 62nd annual meeting of the association for computational linguistics (volume 3: system demonstrations), 2024, pp. 400–410

    2024

  38. [38]

    Swift: a scalable lightweight infrastructure for fine-tuning,

    Y . Zhao, J. Huang, J. Hu, X. Wang, Y . Mao, D. Zhang, Z. Jiang, Z. Wu, B. Ai, A. Wanget al., “Swift: a scalable lightweight infrastructure for fine-tuning,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 28, 2025, pp. 29 733–29 735

    2025

  39. [39]

    Efficient memory management for large language model serving with pagedattention,

    W. Kwon, Z. Li, S. Zhuang, Y . Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica, “Efficient memory management for large language model serving with pagedattention,” inProceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023

    2023

  40. [40]

    Step-audio 2 technical report,

    B. Wu, C. Yan, C. Hu, C. Yi, C. Feng, F. Tian, F. Shen, G. Yu, H. Zhang, J. Liet al., “Step-audio 2 technical report,”arXiv preprint arXiv:2507.16632, 2025

    Pith/arXiv arXiv 2025

  41. [41]

    Kimi-audio technical report,

    D. Ding, Z. Ju, Y . Leng, S. Liu, T. Liu, Z. Shang, K. Shen, W. Song, X. Tan, H. Tanget al., “Kimi-audio technical report,”arXiv preprint arXiv:2504.18425, 2025

    Pith/arXiv arXiv 2025

  42. [42]

    METEOR: An automatic metric for MT evaluation with improved correlation with human judgments,

    S. Banerjee and A. Lavie, “METEOR: An automatic metric for MT evaluation with improved correlation with human judgments,” inProceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, J. Goldstein, A. Lavie, C.-Y . Lin, and C. V oss, Eds. Ann Arbor, Michigan: Association for Computational Li...

    2005

  43. [43]

    ROUGE: A package for automatic evaluation of summaries,

    C.-Y . Lin, “ROUGE: A package for automatic evaluation of summaries,” inText Summarization Branches Out. Barcelona, Spain: Association for Computational Linguistics, Jul. 2004, pp. 74–81. [Online]. Available: https://aclanthology.org/W04-1013/

    2004

  44. [44]

    Bertscore: Evaluating text generation with bert,

    T. Zhang, V . Kishore, F. Wu, K. Q. Weinberger, and Y . Artzi, “Bertscore: Evaluating text generation with bert,” inInternational Conference on Learning Representations

  45. [45]

    Sentence-bert: Sentence embeddings using siamese bert-networks,

    N. Reimers and I. Gurevych, “Sentence-bert: Sentence embeddings using siamese bert-networks,” inProceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP- IJCNLP), 2019, pp. 3982–3992

    2019