Pith. sign in

REVIEW 3 major objections 5 minor 44 references

Expressive Speech Retrieval using Natural Language Descriptions of Speaking Style

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces expressive speech retrieval, showing that speech and text encoders trained with contrastive and adversarial objectives can retrieve speech segments by natural-language style descriptions across 22 styles.

desk verdict New task framing and reproducible baselines for expressive speech retrieval, but the headline numbers are an ensemble-query, in-distribution estimate, not the single free-form query scenario the abstract promises. read the letter →

arxiv 2508.11187 v1 pith:X6R4EMZ6 submitted 2025-08-15 eess.AS cs.CLcs.SD

Wonjune Kang , Deb Roy This is my paper · ORCID
classification eess.AScs.CLcs.SD
keywords expressivespeechretrievalspokencontentspeakingstylecross-modalcontrastivelearningrepresentationnaturallanguagepromptspromptaugmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper attempts to establish a new task: retrieving speech by how it is said, using open-ended natural language descriptions of emotion or speaking style as queries. It claims that a contrastively trained pair of speech and text encoders, aided by an adversarial modality alignment term, an auxiliary style classification loss, and LLM-based prompt augmentation, can embed audio and style descriptions into a shared space where cosine similarity ranks matching utterances accurately. Evidence is reported as Recall@k on three datasets spanning 22 styles, and the paper positions this as the first general, open-ended setting for expressive speech retrieval, beyond earlier fixed-emotion work. If correct, it would let users search large speech corpora with prompts like 'sarcastic' or 'calm and empathetic' without predefined labels.

What carries the argument

The central object is a joint latent embedding space for speech and text, produced by a CLAP-style symmetric contrastive loss between paired speech segments and style prompts, an adversarial modality discriminator (with a gradient reversal layer) that makes the two modalities' embeddings harder to tell apart, and an auxiliary style classification loss on speech embeddings that the authors find critical to prevent retrieval collapse. Prompt augmentation with GPT-4o-generated templates and synonyms widens the query space. The mechanism's job is to make cosine similarity between a text query and speech embeddings a reliable ranking signal for speaking style.

What would settle it

Take the trained RoBERTa + emotion2vec model and prompt it with descriptions hand-written by humans that avoid the 11 generated templates and the generated synonym list. If Recall@1 on the same Expresso test items drops to near chance (about 1 in 401, the chance rate with 400 distractors) while performance on generated prompts stays high, the claim of open-ended generalization is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that how something is said can be made retrievable by what someone says about how it is said. It trains a speech encoder and a text encoder so that a speech segment and a free-form text description of its emotion or style land close together in a shared embedding space. At test time, a user prompt is embedded and compared by cosine similarity to cached speech embeddings; the top-ranked utterances are the retrieved matches. The authors report strong Recall@k on three datasets covering 22 emotion and style classes, and argue this is the first open-ended, general setting for expressive speech retrieval, going beyond previous work limited to a fixed set of emotion la

Load-bearing premise

The open-ended generalization claim rests on the assumption that the GPT-4o-generated prompt templates and synonyms used in training are representative of arbitrary user queries, because the evaluation prompts are drawn from the same pool.

Editorial extensions

If this is right

  • Retrieving expressive audio no longer requires a fixed label set: users can query with paraphrases and descriptive phrases, and the same index supports many open-ended styles.
  • Starting from a paralinguistics-specific speech model (emotion2vec) improves retrieval over a general-purpose model (WavLM), especially for nuanced styles on Expresso.
  • The language model chosen to encode prompts matters: RoBERTa and Flan-T5 consistently beat BERT and T5, even for short style descriptions.
  • Retrieval quality grows with target utterance duration; for utterances above roughly six seconds, top-5 retrieval is essentially perfect in the reported setting.
  • The auxiliary style classification loss is load-bearing: without it, the joint embedding space fails to learn and retrieval collapses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because training and evaluation share the same GPT-4o template and synonym pool, the reported Recall@k is probably an in-distribution estimate; genuinely out-of-distribution user phrasings could show lower performance until prompt diversity is increased.
  • The same joint embedding could serve as a reusable style representation for neighboring tasks, such as zero-shot style classification or ranking style captions, not just retrieval.
  • A natural stress test is to evaluate on prompt phrasings written by humans independent of the generated pool, and to measure whether a similarity threshold can separate relevant from irrelevant styles without knowing the class labels.
  • The method's dependence on labeled style datasets suggests a next boundary: learning style-text alignment from weakly labeled or unlabeled expressive audio, where prompt augmentation and adversarial alignment may matter even more.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces the task of expressive speech retrieval: retrieving speech utterances that match a natural-language description of their speaking style (e.g., "sounds bored" or "whispered, sympathetic"). The authors propose a CLAP-style framework with speech and text encoders trained via contrastive learning, an adversarial modality discriminator, an auxiliary style classification loss, and GPT-4o-generated prompt templates/synonyms for data augmentation. The system is evaluated on ESD and Expresso test sets, covering 22 unique style classes, with Recall@k against classification-based retrieval baselines, plus ablations showing the importance of each training component. The paper also analyzes performance across utterance durations and different encoder backbones.

Significance. If the headline claim were fully supported, this would be a useful new task and a reproducible baseline for cross-modal speech-style retrieval. The authors release code, data, and checkpoints, and the controlled trials with one positive vs. 400 distractors, together with systematic ablations, are strengths. The central weakness is that the evaluation does not actually measure the single-prompt, open-ended inference scenario described in the paper, and the prompts used in evaluation come from the same GPT-4o template/synonym pool used in training. The generalization claim is therefore not yet established. The framework and empirical inventory are nonetheless valuable and the issue is addressable by a revised evaluation protocol.

major comments (3)
  1. [§IV-C, §III-D, Table II] The evaluation protocol does not measure the inference scenario described in §III-D. In each trial, candidates are ranked by the average cosine similarity over all 11 templates × 5 synonyms for the target style—i.e., over all 55 ground-truth paraphrases. A real user submits one prompt, and a single-query retrieval accuracy is never reported. This ensemble-query protocol is likely to inflate Recall@k and cannot support the "free-form," "arbitrary text query" claim. Please report per-query Recall@k by sampling one prompt per trial, or by averaging Recall@k over the individual prompts, and present these numbers in Table II.
  2. [§V-A, Table II] The statement that the best models are "on par with, or even slightly exceeding, that of the classification-based baselines" is contradicted at Recall@1. On Expresso, the best proposed model (BERT + emotion2vec) achieves R@1 = 0.8317 versus 0.9467 for the emotion2vec classifier; on ESD, the best proposed model (T5 + emotion2vec) achieves 0.6438 versus 0.6814. The R@5/R@10/R@20 numbers are close, but the R@1 gap is substantial. If the contribution is flexibility rather than parity, the comparison should be framed accordingly and should report the trade-off explicitly.
  3. [§V-C, Table III] The prompt-augmentation ablation reinforces the concern that the model is tuned to the GPT-4o template/synonym distribution. Removing prompt augmentation drops Expresso R@1 from 0.8159 to 0.5901 when evaluation uses the same diverse prompt pool. Since the same 55 prompts used in training are also used in evaluation, the paper never tests generalization to genuinely novel phrasings. Please add an evaluation on a separate, held-out prompt set (e.g., human-written descriptions or LLM paraphrases generated from a different seed/template family) and report single-query Recall@k for those prompts. This is necessary to support the paper's open-ended retrieval claim.
minor comments (5)
  1. [§IV-A1] The set of 22 unique styles is obtained after excluding 7 Expresso categories (e.g., 'animal directed', 'child directed', 'narration'). The exclusion is reasonable, but the paper should explicitly list the excluded categories and note that the '22-style' claim depends on this filtering, since '22' is not simply the union of all annotated classes in the three datasets.
  2. [§IV-C] It would help to state explicitly that each trial uses 55 prompt embeddings per style (11 templates × 5 synonyms) and that the final ranking is an ensemble average. The current wording is clear but does not give the reader the 55 number, which makes the protocol's nature easy to miss.
  3. [Fig. 3] There is a minor typo in the caption: "k = 1and k = 5" should be "k = 1 and k = 5." Also, the figure would benefit from confidence intervals or error bars, since the duration-binned results are based on a small number of test utterances.
  4. [Table III] The row "w/o aux. style classification loss" is shown with dashes. The text explains that performance collapses, but it would be clearer to state in the table caption or a footnote that the model fails to learn a meaningful joint embedding, so no recall values are reported.
  5. [§II-B] Reference [9] is described as the most closely related work, but it is not included as a direct baseline. The ablation table does include a "w/o prompt augmentation" condition that might serve a similar purpose, but a direct reimplementation or an explicit discussion of why [9] is not compared would strengthen the paper.

Circularity Check

1 steps flagged · score 4.0 of 10

Prompt generalization is evaluated on the same GPT-4o template/synonym pool used for training, and recall is averaged over all 55 paraphrase combinations, so the open-ended single-query claim is not actually measured; the core speech-retrieval learning is not equation-level circular.

  1. fitted input called prediction [Section IV-A2 (prompt generation), IV-C (evaluation), V-A (results claim)]
    "During training, we dynamically sample a template and synonym based on an utterance’s class label to construct a random style description. ... we consider all possible combinations of prompt templates and synonyms for the target style as potential queries ... The candidate utterances are ranked by their average similarity across all possible query prompts."

    The 11 templates and 5 synonyms per style generated in Section IV-A2 are the training prompt distribution, and Section IV-C reuses that exact pool as the evaluation query set. Reported Recall@k is therefore an in-distribution score, not a held-out test of generalization to arbitrary user phrasings. The claim in Section V-A that the framework enables retrieval based on arbitrary natural language queries that need not match training classes is thus an extrapolation, not a measured result. Additionally, ranking by average similarity over all 55 combinations evaluates an ensemble of all ground-truth paraphrases, not the single-query inference described in Section III-D, so the abstract's 'strong retrieval performance' for free-form prompts is not directly evidenced.

full rationale

No self-citation chain or imported uniqueness theorem is load-bearing: the training objectives (contrastive CLAP loss, adversarial modality discriminator, auxiliary style classification) are standard and externally motivated, and the speech-encoder/backbone comparisons are evaluated on held-out speech segments. The central ranking computation is not circular in the equation-level sense: test speech utterances are not used to fit the model, and the ablation results (Table III) provide independent evidence about which training components matter. The principal circularity concern is the prompt-generalization claim: the GPT-4o template/synonym pool sampled during training is the same pool used to define evaluation queries, and evaluation averages over all prompt combinations rather than using one user prompt. This makes the open-ended 'arbitrary text queries' portion of the central claim partially self-referential—measured performance is on the training prompt distribution—while leaving the style-retrieval core with genuine independent content. A moderate score of 4 reflects that the problem is concentrated in the evaluation of the free-form query claim, not in the underlying learned representation.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method rests on standard contrastive learning plus a hand-tuned auxiliary classification loss that is load-bearing; the prompt distribution is generated ad hoc; no new physical or conceptual entities are introduced.

free parameters (4)
  • adversarial loss weight lambda_adv = 0.1
    Set by hand in the total loss (Eq. 5, Section III-C4); no sensitivity analysis reported.
  • auxiliary classification loss weight lambda_cls = 0.5
    Set by hand in the total loss; critical to training since ablation without it collapses (Table III).
  • prompt templates and synonyms per style = 11 templates, 5 synonyms
    Chosen by hand and generated via GPT-4o; defines the training and evaluation prompt distribution (Section IV-A2).
  • latent dimension d = 512
    Chosen in Section IV-B; standard design choice.
assumptions (4)
  • domain assumption Style labels are comparable across IEMOCAP, ESD, and Expresso after filtering
    The datasets are combined into 22 shared classes with no stated label-mapping procedure (Section IV-A1); cross-dataset style equivalence is assumed.
  • domain assumption Mean-pooled backbone representations retain sufficient style information after fine-tuning
    Speech encoder uses temporal mean pooling of WavLM/emotion2vec outputs (Section III-A); no analysis of pooling choice.
  • standard math Contrastive loss with in-batch negatives gives a valid alignment signal
    Standard CLAP-style objective (Section III-C1).
  • ad hoc to paper Excluding Expresso categories such as animal directed and child directed does not bias the 22-style claim
    Categories deemed less relevant are filtered (Section IV-A1), which narrows the task and may inflate average performance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Expressive Speech Retrieval using Natural Language Descriptions of Speaking Style." pith.science (2026). https://pith.science/paper/X6R4EMZ6

@misc{pith2026250811187,
  author       = {Pith},
  title        = {Pith review of: Expressive Speech Retrieval using Natural Language Descriptions of Speaking Style},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X6R4EMZ6}},
  note         = {Machine review of arXiv:2508.11187}
}
read the original abstract

We introduce the task of expressive speech retrieval, where the goal is to retrieve speech utterances spoken in a given style based on a natural language description of that style. While prior work has primarily focused on performing speech retrieval based on what was said in an utterance, we aim to do so based on how something was said. We train speech and text encoders to embed speech and text descriptions of speaking styles into a joint latent space, which enables using free-form text prompts describing emotions or styles as queries to retrieve matching expressive speech segments. We perform detailed analyses of various aspects of our proposed framework, including encoder architectures, training criteria for effective cross-modal alignment, and prompt augmentation for improved generalization to arbitrary text queries. Experiments on multiple datasets encompassing 22 speaking styles demonstrate that our approach achieves strong retrieval performance as measured by Recall@k.

Figures

Figures reproduced from arXiv: 2508.11187 by the authors.

Figure 1
Figure 1. Proposed training framework for expressive speech retrieval. Using a contrastive loss, speech and matching text prompts [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Expressive speech retrieval at inference time. Given a [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Retrieval performance on Expresso (Recall@1 and [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: t-SNE plots of speech and text embeddings computed [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 39 canonical work pages

  1. [9]

    Prompting audios using acoustic properties for emotion representation,

    H. Dhamyal, B. Elizalde, S. Deshmukh, H. Wang, B. Raj, and R. Singh, “Prompting audios using acoustic properties for emotion representation,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 11 936–11 940

    work page 2024

  2. [1]

    Retrieval and browsing of spoken content,

    C. Chelba, T. J. Hazen, and M. Saraclar, “Retrieval and browsing of spoken content,” IEEE Signal Processing Magazine , vol. 25, no. 3, pp. 39–49, 2008

    work page 2008

  3. [2]

    V oice-based information retrieval—how far are we from the text-based information retrieval?

    L.-s. Lee and Y .-c. Pan, “V oice-based information retrieval—how far are we from the text-based information retrieval?” in IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) . IEEE, 2009, pp. 26–43

    work page 2009

  4. [3]

    Spoken content retrieval: A survey of techniques and technologies,

    M. Larson, G. J. Jones et al. , “Spoken content retrieval: A survey of techniques and technologies,” Foundations and Trends in Information Retrieval, vol. 5, no. 4–5, pp. 235–422, 2012

    work page 2012

  5. [4]

    Spoken content retrieval—beyond cascading speech recognition with text retrieval,

    L.-s. Lee, J. Glass, H.-y. Lee, and C.-a. Chan, “Spoken content retrieval—beyond cascading speech recognition with text retrieval,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 23, no. 9, pp. 1389–1420, 2015

    work page 2015

  6. [5]

    SpeechDPR: End-to-end spoken passage retrieval for open-domain spoken question answering,

    C.-J. Lin, G.-T. Lin, Y .-S. Chuang, W.-L. Wu, S.-W. Li, A. Mohamed, H.- y. Lee, and L.-S. Lee, “SpeechDPR: End-to-end spoken passage retrieval for open-domain spoken question answering,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 12 476–12 480

    work page 2024

  7. [6]

    Retrieval augmented end-to-end spoken dialog models,

    M. Wang, I. Shafran, H. Soltau, W. Han, Y . Cao, D. Yu, and L. El Shafey, “Retrieval augmented end-to-end spoken dialog models,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 12 056–12 060

    work page 2024

  8. [7]

    WavRAG: Audio-integrated retrieval augmented generation for spoken dialogue models,

    Y . Chen, S. Ji, H. Wang, Z. Wang, S. Chen, J. He, J. Xu, and Z. Zhao, “WavRAG: Audio-integrated retrieval augmented generation for spoken dialogue models,” arXiv preprint arXiv:2502.14727 , 2025

    arXiv 2025

Show all 44 references
  1. [8]

    Speech retrieval-augmented generation without automatic speech recognition,

    D. J. Min, K. Mundnich, A. Lapastora, E. Soltanmohammadi, S. Ronanki, and K. Han, “Speech retrieval-augmented generation without automatic speech recognition,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2025, pp. 1–5

    2025

  2. [10]

    Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,

    Y . Wu, K. Chen, T. Zhang, Y . Hui, T. Berg-Kirkpatrick, and S. Dubnov, “Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 20...

    2023

  3. [11]

    IEMOCAP: Interactive emotional dyadic motion capture database,

    C. Busso, M. Bulut, C.-C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. N. Chang, S. Lee, and S. S. Narayanan, “IEMOCAP: Interactive emotional dyadic motion capture database,” Language Resources and Evaluation , vol. 42, pp. 335–359, 2008

    2008

  4. [12]

    Emotional voice conversion: Theory, databases and ESD,

    K. Zhou, B. Sisman, R. Liu, and H. Li, “Emotional voice conversion: Theory, databases and ESD,” Speech Communication, vol. 137, pp. 1–18, 2022

    2022

  5. [13]

    Expresso: A benchmark and analysis of discrete expressive speech resynthesis,

    T. A. Nguyen, W.-N. Hsu, A. D’Avirro, B. Shi, I. Gat, M. Fazel-Zarani, T. Remez, J. Copet, G. Synnaeve, M. Hassid, F. Kreuk, Y . Adi, and E. Dupoux, “Expresso: A benchmark and analysis of discrete expressive speech resynthesis,” in Proc. Interspeech, 2023, pp. 4823–4827

    2023

  6. [14]

    PromptTTS: Controllable text-to-speech with text descriptions,

    Z. Guo, Y . Leng, Y . Wu, S. Zhao, and X. Tan, “PromptTTS: Controllable text-to-speech with text descriptions,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2023, pp. 1–5

    2023

  7. [15]

    PromptStyle: Controllable style transfer for text-to-speech with natural language descriptions,

    G. Liu, Y . Zhang, Y . Lei, Y . Chen, R. Wang, L. Xie, and Z. Li, “PromptStyle: Controllable style transfer for text-to-speech with natural language descriptions,” in Proc. Interspeech, 2023, pp. 4888–4892

    2023

  8. [16]

    PromptTTS 2: Describing and generating voices with text prompt,

    Y . Leng, Z. Guo, K. Shen, X. Tan, Z. Ju, Y . Liu, Y . Liu, D. Yang, L. Zhang, K. Song et al., “PromptTTS 2: Describing and generating voices with text prompt,” in International Conference on Learning Representations , 2024

    2024

  9. [17]

    DreamV oice: Text-guided voice conversion,

    J. Hai, K. Thakkar, H. Wang, Z. Qin, and M. Elhilali, “DreamV oice: Text-guided voice conversion,” in Proc. Interspeech, 2024, pp. 4373– 4377

    2024

  10. [18]

    StyleCap: Automatic speaking-style captioning from speech based on speech and language self-supervised learning models,

    K. Yamauchi, Y . Ijima, and Y . Saito, “StyleCap: Automatic speaking-style captioning from speech based on speech and language self-supervised learning models,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2024, pp. 11 261–11 265

    2024

  11. [19]

    LibriTTS-P: A corpus with speaking style and speaker identity prompts for text-to-speech and style captioning,

    M. Kawamura, R. Yamamoto, Y . Shirahata, T. Hasumi, and K. Tachibana, “LibriTTS-P: A corpus with speaking style and speaker identity prompts for text-to-speech and style captioning,” in Proc. Interspeech, 2024, pp. 1850–1854

    2024

  12. [20]

    V ocabulary independent spoken term detection,

    J. Mamou, B. Ramabhadran, and O. Siohan, “V ocabulary independent spoken term detection,” in Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2007, pp. 615–622

    2007

  13. [21]

    Statistical lattice-based spoken document retrieval,

    T. K. Chia, K. C. Sim, H. Li, and H. T. Ng, “Statistical lattice-based spoken document retrieval,” ACM Transactions on Information Systems (TOIS), vol. 28, no. 1, pp. 1–30, 2010

    2010

  14. [22]

    Improved semantic retrieval of spoken content by document/query expansion with random walk over acoustic similarity graphs,

    H.-Y . Lee and L.-S. Lee, “Improved semantic retrieval of spoken content by document/query expansion with random walk over acoustic similarity graphs,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 22, no. 1, pp. 80–94, 2013

    2013

  15. [23]

    Evaluating ASR output for information retrieval,

    L. van der Werff and W. Heeren, “Evaluating ASR output for information retrieval,” Searching Spontaneous Conversational Speech , p. 13, 2007

    2007

  16. [24]

    Investigating the global semantic impact of speech recognition error on spoken content collections,

    M. Larson, M. Tsagkias, J. He, and M. de Rijke, “Investigating the global semantic impact of speech recognition error on spoken content collections,” in European Conference on Information Retrieval. Springer, 2009, pp. 755–760

    2009

  17. [25]

    Speech-centric information processing: An optimization-oriented approach,

    X. He and L. Deng, “Speech-centric information processing: An optimization-oriented approach,” Proceedings of the IEEE , vol. 101, no. 5, pp. 1116–1135, 2013

    2013

  18. [26]

    Unsupervised spoken-term detection with spoken queries using segment-based dynamic time warping

    C.-a. Chan and L.-s. Lee, “Unsupervised spoken-term detection with spoken queries using segment-based dynamic time warping.” in Proc. Interspeech, 2010, pp. 693–696

    2010

  19. [27]

    Memory efficient subsequence dtw for query-by-example spoken term detection,

    X. Anguera and M. Ferrarons, “Memory efficient subsequence dtw for query-by-example spoken term detection,” in IEEE International Conference on Multimedia and Expo (ICME) . IEEE, 2013, pp. 1–6

    2013

  20. [28]

    The spoken web search task at MediaEval 2012,

    F. Metze, X. Anguera, E. Barnard, M. Davel, and G. Gravier, “The spoken web search task at MediaEval 2012,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2013, pp. 8121–8125

    2012

  21. [29]

    RECAP: Retrieval-augmented audio captioning,

    S. Ghosh, S. Kumar, C. K. R. Evuru, R. Duraiswami, and D. Manocha, “RECAP: Retrieval-augmented audio captioning,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 1161–1165

    2024

  22. [30]

    Beyond speaker identity: Text guided target speech extraction,

    M. Huo, A. Jain, C. P. Huynh, F. Kong, P. Wang, Z. Liu, and V . Bhat, “Beyond speaker identity: Text guided target speech extraction,” in International Conference on Automatic Face and Gesture Recognition , no. 1, 2023

    2023

  23. [31]

    WavLM: Large-scale self-supervised pre- training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao et al., “WavLM: Large-scale self-supervised pre- training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  24. [32]

    emo- tion2vec: Self-supervised pre-training for speech emotion representation,

    Z. Ma, Z. Zheng, J. Ye, J. Li, Z. Gao, S. Zhang, and X. Chen, “emo- tion2vec: Self-supervised pre-training for speech emotion representation,” in Findings of the Association for Computational Linguistics: ACL , 2024, pp. 15 747–15 760

    2024

  25. [33]

    BERT: Pre- training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre- training of deep bidirectional transformers for language understanding,” in Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, ...

    2019

  26. [34]

    RoBERTa: A robustly optimized BERT pretraining approach,

    Y . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V . Stoyanov, “RoBERTa: A robustly optimized BERT pretraining approach,” arXiv preprint arXiv:1907.11692 , 2019

    1907 arXiv

  27. [35]

    Exploring the limits of transfer learning with a unified text-to-text transformer,

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of Machine Learning Research , vol. 21, no. 140, pp. 1–67, 2020

    2020

  28. [36]

    The flan collection: Designing data and methods for effective instruction tuning,

    S. Longpre, L. Hou, T. Vu, A. Webson, H. W. Chung, Y . Tay, D. Zhou, Q. V . Le, B. Zoph, J. Wei et al., “The flan collection: Designing data and methods for effective instruction tuning,” in International Conference on Machine Learning. PMLR, 2023, pp. 22 631–22 648

    2023

  29. [37]

    Sentence-T5: Scalable sentence encoders from pre-trained text-to-text models,

    J. Ni, G. H. Abrego, N. Constant, J. Ma, K. Hall, D. Cer, and Y . Yang, “Sentence-T5: Scalable sentence encoders from pre-trained text-to-text models,” in Findings of the Association for Computational Linguistics: ACL, 2022, pp. 1864–1874

    2022

  30. [38]

    Domain-adversarial training of neural networks,

    Y . Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. March, and V . Lempitsky, “Domain-adversarial training of neural networks,” Journal of Machine Learning Research , vol. 17, no. 59, pp. 1–35, 2016

    2016

  31. [39]

    Unsupervised domain adaptation by backpropagation,

    Y . Ganin and V . Lempitsky, “Unsupervised domain adaptation by backpropagation,” in International Conference on Machine Learning . PMLR, 2015, pp. 1180–1189

    2015

  32. [40]

    GPT-4o system card,

    A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford et al., “GPT-4o system card,” arXiv preprint arXiv:2410.21276 , 2024

    2024 arXiv

  33. [41]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in International Conference on Learning Representations , 2019

    2019

  34. [42]

    PromptTTS++: Controlling speaker identity in prompt-based text-to-speech using natural language descriptions,

    R. Shimizu, R. Yamamoto, M. Kawamura, Y . Shirahata, H. Doi, T. Ko- matsu, and K. Tachibana, “PromptTTS++: Controlling speaker identity in prompt-based text-to-speech using natural language descriptions,” inIEEE International Conference on Acoustics, Speech and Signal Processi...

    2024

  35. [43]

    PromotiCon: Prompt- based emotion controllable text-to-speech via prompt generation and matching,

    J.-E. Lee, S.-B. Kim, D.-H. Cho, and S.-W. Lee, “PromotiCon: Prompt- based emotion controllable text-to-speech via prompt generation and matching,” in IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2024, pp. 1151–1156

    2024

  36. [44]

    Visualizing data using t-SNE,

    L. Van der Maaten and G. Hinton, “Visualizing data using t-SNE,”Journal of Machine Learning Research , vol. 9, no. 11, 2008

    2008

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.