Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

Leveraging Chain of Thought towards Empathetic Spoken Dialogue without Corresponding Question-Answering Data

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A frozen text LLM, trained only on ASR transcripts and emotion labels through two small adapters, can be prompted to generate empathetic spoken-dialogue responses with no question-answer supervision.

desk verdict A sensible two-stage alignment pipeline plus CoT prompting for speech-conditioned empathetic responses, but the headline superiority is undercut by test-set prompt tuning and GPT-4o-only evaluation. read the letter →

arxiv 2501.10937 v1 pith:7RYI3W2D submitted 2025-01-19 cs.CL cs.SDeess.AS

Jingran Xie , Shun Lei , Yue Yu , Yang Xiang , Hui Wang , Xixin Wu , Zhiyong Wu This is my paper · ORCID
classification cs.CLcs.SDeess.AS
keywords empatheticdialoguespokensystemchain-of-thoughtpromptingspeechemotionrecognitionautomaticmultitasklearningfrozenlanguagemodelspeech-textalignment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that empathetic spoken dialogue does not require spoken question-answer training data. The proposed LPE method decomposes the task into listening (ASR), perceiving (SER), and expressing (empathetic response), trains a frozen LLaMA2 decoder on existing speech transcription and emotion datasets, and then uses a hand-written Chain-of-Thought prompt to activate the expression step. If correct, this removes the main data bottleneck for empathetic spoken dialogue and gives a low-cost recipe: two adapters plus prompting on top of a frozen text LLM. The experiments report competitive ASR and SER accuracy, better generated-response scores than SALMONN and Qwen-Audio-Chat, and win rates above a cascaded Whisper + emotion2vec + LLaMA2 baseline.

What carries the argument

The central mechanism is the LPE (Listen, Perceive, Express) framework: a frozen LLaMA2-7B-chat decoder is fed speech content embeddings from a WavLM encoder through a subsampler adapter, plus a pooled emotion embedding from a multiscale adapter. Two-stage training first aligns content with ASR transcripts, then aligns emotion labels through multitask ASR/SER training. At inference, a hand-crafted zero-shot Chain-of-Thought prompt explicitly lists the steps transcribe, identify emotion, and respond empathetically, and this step decomposition is what unlocks response generation without QA supervision.

What would settle it

Corrupt the emotion embedding at inference while keeping the transcript identical; if generated responses and their empathy scores stay unchanged, the perceive stage is not load-bearing and the claimed role of emotion perception is unsupported.

Watch

Extended reading notes

Core claim

The paper claims that a spoken dialogue system can generate empathetic text responses without ever being trained on question-answer pairs. A frozen LLaMA2-7B-chat decoder is aligned to speech content through a subsampler adapter trained on ASR data, then aligned to emotional cues through a multiscale adapter trained with multitask ASR and SER data. At inference, a manually designed zero-shot Chain-of-Thought prompt instructs the model to first transcribe the speech, then identify the emotion, and finally produce an empathetic response. The authors report that without this prompt the model merely outputs transcriptions or emotion labels, and that with it LPE outperforms existing speech-based LLMs on objective and GPT-4o-based subjective metrics while matching a state-of-the-art cascaded pipeline.

Load-bearing premise

The load-bearing premise is that a hand-written step-by-step prompt can flip a language model with locked weights from repeating the transcript or emotion label it was trained to emit into producing a new empathetic response, even though the model never saw an example of that final response during training.

Editorial extensions

If this is right

  • Spoken empathetic dialogue can be built from a frozen text LLM plus two trainable speech adapters using only existing ASR and SER datasets, so the need for expensive spoken question-answer corpora disappears.
  • Zero-shot CoT with manually defined listen-perceive-express steps is the prompt configuration that unlocks empathetic generation, while longer reasoning prompts and few-shot rationales hurt content quality in this setup.
  • The second training stage is essential: without emotion alignment via multitask ASR/SER training, the model falls back to transcribing or labeling instead of responding.
  • The single-model LPE matches or slightly exceeds a cascaded Whisper + emotion2vec + LLaMA2 system on content and empathy scores, suggesting the end-to-end adapter approach does not sacrifice quality for lower latency and simplicity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the CoT prompt is the true switch that turns transcription behavior into response behavior, the same two-stage recipe may transfer to other modalities where paired instruction data is scarce, such as visual emotion perception; the bottleneck becomes designing intermediate steps rather than collecting new supervised pairs.
  • The poor few-shot CoT results suggest the frozen LLM may be copying the style of the provided rationale instead of conditioning on the speech input; a follow-up could measure n-gram overlap between the generated response and the few-shot example to test this copying hypothesis.
  • Because the current evaluation uses GPT-4o-generated references and GPT-4o scoring, a human listening study would test whether the reported empathy win rates hold when people, rather than an automated judge, rate the responses.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes LPE (Listen, Perceive, and Express), a framework for empathetic spoken dialogue that avoids the need for spoken question-answering (QA) training data. LPE uses a WavLM speech encoder with two adapters feeding a frozen LLaMA2-7B-chat decoder. A two-stage training procedure first aligns speech content with text via ASR data, then aligns emotional information via SER data using multitask learning and an emotion classification loss. At inference, a manually designed Chain-of-Thought (CoT) prompt guides the model to first transcribe, then identify emotion, and finally generate an empathetic response. The authors construct test sets from IEMOCAP session 5 and a TTS-augmented Alpaca set, evaluate ASR/SER performance, and use BLEU, BERTScore, and GPT-4o-based subjective scoring to claim that LPE outperforms other speech-based LLMs. Ablations show that stage 2 and the CoT prompt steps are important for the reported gains.

Significance. If validated, the LPE approach is a practically valuable contribution: it shows a frozen text LLM can be adapted to empathetic spoken dialogue using only ASR and SER supervision, which are more readily available than spoken QA datasets. The paper's strengths include the use of natural speech data, a small number of trainable parameters (79.7M, 1.1% of the model), and a clear decomposition of listening, perceiving, and expressing via CoT. The ablations are informative and consistent with the proposed mechanism. However, the significance is tempered by the evaluation design: the headline empathy results rely on GPT-4o as both reference generator and scorer, the CoT prompt is selected on the test set, and no human evaluation is reported. These issues directly affect the central claim that LPE outperforms other speech-based LLMs in empathetic response quality.

major comments (3)
  1. [II.C and IV.C] The final CoT prompt is selected by iterative refinement on the test set. Section II.C states: 'We evaluate each prompt on several samples of test set using our two-stage trained model. By iteratively refining the prompts, we identify the most effective prompt for each CoT type, leading to the results in Section IV-C.' This same prompt is then used for the headline comparisons in Table III and the objective results in Table II. Since Table IV shows that without the selected prompt the model largely emits transcriptions or emotion labels (No CoT: Content 2.14, Empathy 1.35), the entire empathetic-expression capability is carried by the prompt. Selecting the prompt on the evaluation test set therefore means the reported advantage may reflect test-set overfitting in prompt selection rather than a general capability of LPE. Please validate the prompt on a held-out split or otherwise demonstrate that the chosen prompt generalizes to unseen data.
  2. [III.B and III.D] The ground truth empathetic responses are generated by GPT-4o (Section III.B) and the subjective quality scores are also produced by GPT-4o (Section III.D), with no human evaluation reported. This creates a circularity: the references and the scoring model share the same model, so the objective metrics (BLEU, BERTScore) and the subjective scores may systematically favor GPT-4o-like responses. The statement 'we manually verify all responses to ensure the quality of GT responses' verifies the references, not the generated responses. Please add a human evaluation on a sample of generated responses, or at minimum report the correlation between GPT-4o scores and human judgments on this task.
  3. [III.C and Table III] The cascaded baseline is given the same CoT prompt optimized for LPE, rather than a prompt tuned separately for the cascade. Section III.C states: 'As for the cascade model, we use the same CoT prompt as our LPE.' Since the paper itself demonstrates that prompt choice strongly affects response quality (Table IV), this design biases the comparison in Table III and Fig. 2: the LPE prompt was selected for LPE's decoding behavior, not for the cascade's. Please tune the cascade's prompt separately, or report the cascade with its official recommended prompt as well as with the LPE prompt.
minor comments (4)
  1. [Abstract and Section I] There are several grammatical errors that should be corrected, including 'listen the content' and 'an novel' in the Abstract and Introduction.
  2. [Fig. 1 caption] The bracket notation used to denote transcriptions, emotions, and CoT steps is only described in the caption; please define it in the main text as well.
  3. [Table II] The row label '-w/o stage 2' has inconsistent spacing; also consider reporting confidence intervals or significance tests for the objective metrics, since the differences in BERTScore are small.
  4. [Table IV] The prompt-ablation results in Table IV are reported without standard deviations, unlike Table III; adding them would clarify whether the differences between prompt variants are meaningful.

Circularity Check

2 steps flagged · score 6.0 of 10

Test-set prompt selection makes the reported LPE superiority a fitted rather than predicted result; GPT-4o both generates references and scores outputs.

  1. fitted input called prediction [Section II-C (Chain of Thought); results reported in Section IV-C, Tables II-IV]
    "We evaluate each prompt on several samples of test set using our two-stage trained model. By iteratively refining the prompts, we identify the most effective prompt for each CoT type, leading to the results in Section IV-C."

    The CoT prompt is the component that creates the empathetic response behavior: Table IV shows that without CoT the model emits transcriptions or emotion labels (Content 2.14, Empathy 1.35), while the chosen 'steps' prompt scores 3.99/3.72. The prompt was selected by iterative evaluation on the test set, and the same test set is then used to report LPE's superiority over baselines in Tables II and III. The reported 'prediction' that LPE outperforms is therefore partly a report of test-set prompt fitting rather than an independent generalization result; no held-out prompt validation is described.

  2. other [Section III.B (Test Setup) and Section III.D (Generation Evaluation Metrics)]
    "Following main stream evaluation metric for natural language generation (NLG), we provide question and emotion labels in text form as conditions, with generated text response as input to GPT-4o ... to score the generated responses from 1-5 based on empathetic quality, content helpful and generate clarity."

    Section III.B states 'We use GPT-4o to generate ground truth empathetic responses based on the question and emotion label.' Thus GPT-4o both defines the reference answers and serves as the subjective judge. The subjective scores therefore measure agreement with GPT-4o's own response distribution and preferences, not an independent human standard. Combined with the test-set prompt selection in Section II-C, the reported empathy and content superiority is a self-referential evaluation loop: GPT-4o helps select the prompt and then evaluates the result.

full rationale

The paper's two-stage ASR/SER adaptation is a genuine training contribution with independent data sources, and the frozen-LLM generation is not mathematically derived from the evaluation metrics. However, the central claim that LPE outperforms other speech-based LLMs rests on the CoT prompt, which the authors explicitly selected by testing candidate prompts on the test set before reporting results on that same test set. Table IV shows the prompt is load-bearing: without it the model only outputs transcriptions or emotion labels. Therefore the headline comparison is, in part, a fitted input reported as a prediction. The evaluation is further self-referential because GPT-4o generates the ground-truth empathetic responses and also provides the subjective scores, so the reported empathy and content gains reflect agreement with GPT-4o's own preferences. No self-citation chain is involved, and the underlying speech-alignment method is not itself circular, but the empirical superiority claim is not independent of the test-set prompt-tuning loop. This is partial circularity rather than full equivalence, so the score is 6.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new physical or conceptual entities are postulated. The central claim rests on hyperparameters (lambda, multitask ratio, replay ratio, decoding parameters), on the domain assumptions about WavLM and frozen LLaMA2, and on the ad hoc use of GPT-4o as ground-truth generator and judge.

free parameters (4)
  • stage-2 emotion loss weight lambda = 0.1
    Set based on preliminary experiments (Section II.B); controls the balance between decoder loss and emotion classification loss.
  • multitask sampling ratio = ASR 0.2, SER 0.3, ASR+SER 0.5
    Predefined in Section III.A; hand-chosen scheduling for stage 2 to balance tasks.
  • ASR replay ratio = 20% of LibriSpeech
    Continual-learning replay fraction chosen to avoid catastrophic forgetting (Section II.B).
  • decoding temperature and top-p = temperature 0.7, top-p 0.85
    Inference hyperparameters in Section III.A; affect generation quality and the reported empathy scores.
assumptions (4)
  • domain assumption WavLM upper layers capture speech content and lower layers capture paralinguistic or emotional features
    Invoked in Section II.A to justify extracting content embedding from the last layer and emotion embedding from a weighted sum over all layers; relies on [22].
  • domain assumption A frozen LLaMA2-7B-chat can produce empathetic responses when given a transcription and an emotion label via a CoT prompt
    The method does not train the LLM on QA pairs; the entire expressing step depends on the base model's pretrained conversational and empathetic abilities (Section II.C).
  • ad hoc to paper GPT-4o-generated ground-truth empathetic responses and GPT-4o scores are valid proxies for human empathetic quality
    No human evaluation is reported; both the reference responses (Section III.B) and the subjective scoring (Section III.D) use GPT-4o, so this assumption is introduced by the paper rather than established externally.
  • domain assumption Multitask learning with template completion unifies ASR, SER, and combined tasks into text generation
    The paper assumes that filling GPT-4o-generated templates with transcriptions and emotion labels teaches the LLM to listen and perceive (Section II.B).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Leveraging Chain of Thought towards Empathetic Spoken Dialogue without Corresponding Question-Answering Data." pith.science (2026). https://pith.science/paper/7RYI3W2D

@misc{pith2026250110937,
  author       = {Pith},
  title        = {Pith review of: Leveraging Chain of Thought towards Empathetic Spoken Dialogue without Corresponding Question-Answering Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7RYI3W2D}},
  note         = {Machine review of arXiv:2501.10937}
}
read the original abstract

Empathetic dialogue is crucial for natural human-computer interaction, allowing the dialogue system to respond in a more personalized and emotionally aware manner, improving user satisfaction and engagement. The emergence of large language models (LLMs) has revolutionized dialogue generation by harnessing their powerful capabilities and shown its potential in multimodal domains. Many studies have integrated speech with text-based LLMs to take speech question as input and output text response. However, the lack of spoken question-answering datasets that include speech style information to supervised fine-tuning (SFT) limits the performance of these systems. As a result, while these systems excel at understanding speech content, they often struggle to generate empathetic responses. In response, we propose a novel approach that circumvents the need for question-answering data, called Listen, Perceive, and Express (LPE). Our method employs a two-stage training process, initially guiding the LLM to listen the content and perceive the emotional aspects of speech. Subsequently, we utilize Chain-of-Thought (CoT) prompting to unlock the model's potential for expressing empathetic responses based on listened spoken content and perceived emotional cues. We employ experiments to prove the effectiveness of proposed method. To our knowledge, this is the first attempt to leverage CoT for speech-based dialogue.

Figures

Figures reproduced from arXiv: 2501.10937 by the authors.

Figure 1
Figure 1. Architecture of the proposed model is shown on left side. On the right is the real sample showing how it listens, perceives, and expresses. () denotes [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Win rate compared with cascade model. TABLE III SUBJECTIVE RESULTS ON GENERATED RESPONSE Content Empathy Clarity LPE 3.99 ± 0.54 3.72 ± 0.90 4.97 ± 0.17 SALMONN 2.12 ± 0.32 1.34 ± 0.56 4.94 ± 0.23 Qwen-Audio-Chat 2.00 ± 0.46 1.63 ± 0.96 4.98 ± 0.14 Whisper+LLaMA2 3.97 ± 0.46 2.87 ± 0.96 4.95 ± 0.20 + w/ emotion2vec 4.01 ± 0.34 3.64 ± 0.52 4.98 ± 0.14 metrics are limited in assessing generation quality, so we conduct… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Enhancing Generalization of Speech Large Language Models with Multi-Task Behavior Imitation and Speech-Text Interleaving

    eess.AS 2025-05 conditional novelty 5.0 of 10

    Multi-task behavior imitation with speech-text interleaving improves speech LLM generalization on prompts and zero-shot tasks using only paired speech and transcripts.

  2. Empowering Multimodal LLMs with External Tools: A Comprehensive Survey

    cs.CV 2025-08 unverdicted novelty 2.0 of 10

    A survey paper maps how external tools are used to augment multimodal large language models across data, tasks, evaluation, and future directions.

Reference graph

Works this paper leans on

39 extracted references · 3 canonical work pages · cited by 2 Pith papers

  1. [1]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al. , “Language models are unsupervised multitask learners,” OpenAI blog , vol. 1, no. 8, p. 9, 2019

    2019

  2. [2]

    Language models are few-shot learners,

    T. B. Brown, “Language models are few-shot learners,” arXiv preprint arXiv:2005.14165, 2020

    arXiv 2005

  3. [3]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar et al. , “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023

    arXiv 2023

  4. [4]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al. , “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023

    arXiv 2023

  5. [5]

    Language is not all you need: Aligning perception with language models,

    S. Huang, L. Dong, W. Wang, Y . Hao, S. Singhal, S. Ma, T. Lv, L. Cui, O. K. Mohammed, B. Patra et al. , “Language is not all you need: Aligning perception with language models,” Advances in Neural Information Processing Systems , vol. 36, pp. 72 096–72 109, 2023

    2023

  6. [6]

    Visual instruction tuning,

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,”Advances in neural information processing systems , vol. 36, 2024

    2024

  7. [7]

    Llava-plus: Learning to use tools for creating multimodal agents,

    S. Liu, H. Cheng, H. Liu, H. Zhang, F. Li, T. Ren, X. Zou, J. Yang, H. Su, J. Zhu et al. , “Llava-plus: Learning to use tools for creating multimodal agents,” arXiv preprint arXiv:2311.05437 , 2023

    arXiv 2023

  8. [8]

    Speechgpt: Empowering large language models with intrinsic cross- modal conversational abilities,

    D. Zhang, S. Li, X. Zhang, J. Zhan, P. Wang, Y . Zhou, and X. Qiu, “Speechgpt: Empowering large language models with intrinsic cross- modal conversational abilities,” arXiv preprint arXiv:2305.11000 , 2023

    arXiv 2023

Show all 39 references
  1. [9]

    Llasm: Large language and speech model,

    Y . Shu, S. Dong, G. Chen, W. Huang, R. Zhang, D. Shi, Q. Xiang, and Y . Shi, “Llasm: Large language and speech model,” arXiv preprint arXiv:2308.15930, 2023

    2023 arXiv

  2. [10]

    Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models,

    Y . Chu, J. Xu, X. Zhou, Q. Yang, S. Zhang, Z. Yan, C. Zhou, and J. Zhou, “Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models,” arXiv preprint arXiv:2311.07919 , 2023

    2023 arXiv

  3. [11]

    Salmonn: Towards generic hearing abilities for large language models,

    C. Tang, W. Yu, G. Sun, X. Chen, T. Tan, W. Li, L. Lu, Z. Ma, and C. Zhang, “Salmonn: Towards generic hearing abilities for large language models,” arXiv preprint arXiv:2310.13289 , 2023

    2023 arXiv

  4. [12]

    Blsp: Bootstrapping language-speech pre-training via behavior align- ment of continuation writing,

    C. Wang, M. Liao, Z. Huang, J. Lu, J. Wu, Y . Liu, C. Zong, and J. Zhang, “Blsp: Bootstrapping language-speech pre-training via behavior align- ment of continuation writing,” arXiv preprint arXiv:2309.00916 , 2023

    2023 arXiv

  5. [13]

    Blsp- emo: Towards empathetic large speech-language models,

    C. Wang, M. Liao, Z. Huang, J. Wu, C. Zong, and J. Zhang, “Blsp- emo: Towards empathetic large speech-language models,” arXiv preprint arXiv:2406.03872, 2024

    2024 arXiv

  6. [14]

    Hello gpt-4o,

    OpenAI, “Hello gpt-4o,” OpenAI, Tech. Rep., 2024. [Online]. Available: https://openai.com/index/hello-gpt-4o

    2024

  7. [15]

    E-chat: Emotion-sensitive spoken dialogue system with large language models,

    H. Xue, Y . Liang, B. Mu, S. Zhang, Q. Chen, and L. Xie, “E-chat: Emotion-sensitive spoken dialogue system with large language models,” arXiv preprint arXiv:2401.00475 , 2023

    2023 arXiv

  8. [16]

    Advancing large language models to capture varied speaking styles and respond properly in spoken conversations,

    G.-T. Lin, C.-H. Chiang, and H.-y. Lee, “Advancing large language models to capture varied speaking styles and respond properly in spoken conversations,” arXiv preprint arXiv:2402.12786 , 2024

    2024 arXiv

  9. [17]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhou et al. , “Chain-of-thought prompting elicits reasoning in large language models,” Advances in neural information processing systems , vol. 35, pp. 24 824–24 837, 2022

    2022

  10. [18]

    Soulchat: Improving llms’ empathy, listening, and comfort abilities through fine-tuning with multi-turn empathy conversations,

    Y . Chen, X. Xing, J. Lin, H. Zheng, Z. Wang, Q. Liu, and X. Xu, “Soulchat: Improving llms’ empathy, listening, and comfort abilities through fine-tuning with multi-turn empathy conversations,” in Findings of the Association for Computational Linguistics: EMNLP 2023 , 2023, pp...

    2023

  11. [19]

    Cause-aware empathetic response generation via chain-of- thought fine-tuning,

    X. Chen, C. Yang, M. Lan, L. Cai, Y . Chen, T. Hu, X. Zhuang, and A. Zhou, “Cause-aware empathetic response generation via chain-of- thought fine-tuning,” arXiv preprint arXiv:2408.11599 , 2024

    2024 arXiv

  12. [20]

    Cot-st: Enhancing llm-based speech translation with multimodal chain-of-thought,

    Y . Du, Z. Ma, Y . Yang, K. Deng, X. Chen, B. Yang, Y . Xiang, M. Liu, and B. Qin, “Cot-st: Enhancing llm-based speech translation with multimodal chain-of-thought,” arXiv preprint arXiv:2409.19510 , 2024

    2024 arXiv

  13. [21]

    Seamlessexpressivelm: Speech language model for expressive speech-to-speech translation with chain-of-thought,

    H. Gong and B. Veluri, “Seamlessexpressivelm: Speech language model for expressive speech-to-speech translation with chain-of-thought,” arXiv preprint arXiv:2405.20410, 2024

    2024 arXiv

  14. [22]

    Wavlm: Large-scale self-supervised pre- training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao et al. , “Wavlm: Large-scale self-supervised pre- training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  15. [23]

    Multi-task learning in natural language processing: An overview,

    S. Chen, Y . Zhang, and Q. Yang, “Multi-task learning in natural language processing: An overview,” ACM Computing Surveys, vol. 56, no. 12, pp. 1–32, 2024

    2024

  16. [24]

    Automatic chain of thought prompting in large language models,

    Z. Zhang, A. Zhang, M. Li, and A. Smola, “Automatic chain of thought prompting in large language models,” arXiv preprint arXiv:2210.03493 , 2022

    2022 arXiv

  17. [25]

    Least-to-most prompting enables complex reasoning in large language models,

    D. Zhou, N. Sch ¨arli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schu- urmans, C. Cui, O. Bousquet, Q. Le et al. , “Least-to-most prompting enables complex reasoning in large language models,” arXiv preprint arXiv:2205.10625, 2022

    2022 arXiv

  18. [26]

    Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models,

    L. Wang, W. Xu, Y . Lan, Z. Hu, Y . Lan, R. K.-W. Lee, and E.-P. Lim, “Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models,” arXiv preprint arXiv:2305.04091 , 2023

    2023 arXiv

  19. [27]

    Librispeech: an asr corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: an asr corpus based on public domain audio books,” in 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP) . IEEE, 2015, pp. 5206–5210

    2015

  20. [28]

    Iemocap: Interactive emotional dyadic motion capture database,

    C. Busso, M. Bulut, C.-C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. N. Chang, S. Lee, and S. S. Narayanan, “Iemocap: Interactive emotional dyadic motion capture database,” Language resources and evaluation , vol. 42, pp. 335–359, 2008

    2008

  21. [29]

    Mead: A large-scale audio-visual dataset for emotional talking-face generation,

    K. Wang, Q. Wu, L. Song, Z. Yang, W. Wu, C. Qian, R. He, Y . Qiao, and C. C. Loy, “Mead: A large-scale audio-visual dataset for emotional talking-face generation,” in European Conference on Computer Vision . Springer, 2020, pp. 700–717

    2020

  22. [30]

    Stanford alpaca: An instruction-following llama model,

    R. Taori, I. Gulrajani, T. Zhang, Y . Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto, “Stanford alpaca: An instruction-following llama model,” https://github.com/tatsu-lab/stanford alpaca, 2023

    2023

  23. [31]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” 2022. [Online]. Available: https://arxiv.org/abs/2212.04356

    2022 arXiv

  24. [32]

    emotion2vec: Self-supervised pre-training for speech emotion repre- sentation,

    Z. Ma, Z. Zheng, J. Ye, J. Li, Z. Gao, S. Zhang, and X. Chen, “emotion2vec: Self-supervised pre-training for speech emotion repre- sentation,” arXiv preprint arXiv:2312.15185 , 2023

    2023 arXiv

  25. [33]

    Bleu: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th annual meeting of the Association for Computational Linguistics , 2002, pp. 311–318

    2002

  26. [34]

    Bertscore: Evaluating text generation with bert,

    T. Zhang, V . Kishore, F. Wu, K. Q. Weinberger, and Y . Artzi, “Bertscore: Evaluating text generation with bert,” arXiv preprint arXiv:1904.09675 , 2019

    1904 arXiv

  27. [35]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  28. [36]

    G-eval: Nlg evaluation using gpt-4 with better human alignment,

    Y . Liu, D. Iter, Y . Xu, S. Wang, R. Xu, and C. Zhu, “G-eval: Nlg evaluation using gpt-4 with better human alignment,” arXiv preprint arXiv:2303.16634, 2023

    2023 arXiv

  29. [37]

    Gptscore: Evaluate as you desire,

    J. Fu, S.-K. Ng, Z. Jiang, and P. Liu, “Gptscore: Evaluate as you desire,” arXiv preprint arXiv:2302.04166 , 2023

    2023 arXiv

  30. [38]

    Llm-based nlg evaluation: Current status and challenges,

    M. Gao, X. Hu, J. Ruan, X. Pu, and X. Wan, “Llm-based nlg evaluation: Current status and challenges,” arXiv preprint arXiv:2402.01383 , 2024

    2024 arXiv

  31. [39]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al. , “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023

    2023 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.