Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Speech Retrieval-Augmented Generation without Automatic Speech Recognition

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SpeechRAG claims that a speech adapter aligned to a frozen LLM text retriever can retrieve audio passages from text queries as accurately as ground-truth transcripts, while avoiding ASR error propagation.

desk verdict Promising ASR-free retrieval recipe, but the missing train/eval split undermines the headline parity claim. read the letter →

arxiv 2412.16500 v3 pith:JREFRNA5 submitted 2024-12-21 eess.AS cs.AIcs.CL

Do June Min , Karel Mundnich , Andy Lapastora , Erfan Soltanmohammadi , Srikanth Ronanki , Kyu Han This is my paper · ORCID
classification eess.AScs.AIcs.CL
keywords speechretrieval-augmentedgenerationspokencontentretrievalcross-modalmultimodalopenquestionansweringaudiolanguagemodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SpeechRAG aims to show that question answering over spoken audio does not need a speech-to-text step. The paper trains a lightweight speech adapter that projects audio into the embedding space of a frozen, LLM-based text retriever, so text queries can retrieve audio passages directly. In experiments on two spoken question-answering datasets, this direct retriever matches the ground-truth transcript baseline (Recall@5 of 0.9702 vs 0.9707 on SpokenSQuAD, 0.9952 vs 0.9942 on VoxPopuli) and clearly beats cascaded ASR-based retrieval when transcripts are noisy. The same idea carries through generation: a pre-trained speech language model conditioned on retrieved audio, not transcripts, outperforms cascaded text-only generation when ASR word error rate is high. The contribution is a way to remove ASR error propagation from both retrieval and generation without training a cross-modal retriever from scratch.

What carries the argument

The load-bearing object is the speech adapter: a trainable projection inserted between a pre-trained speech encoder (HuBERT-large) and a frozen LLM-based text retriever (E5-Mistral-7B-Instruct). Speech is encoded into frame-level features, downsampled by average pooling, and projected into the retriever's token-embedding dimension; the same frozen retriever then processes both text queries and speech passages. The adapter is trained by cosine-embedding distillation, $L = 1 - \cos(\mathbf{e}_s, \mathbf{e}_t)$, pulling the speech embedding $\mathbf{e}_s$ toward the text embedding $\mathbf{e}_t$ of the passage's ground-truth transcript. This alignment lets a single frozen text retriever do cross-modal search without contrastive learning, and the same frozen retriever's capacity is what makes parity with text retrieval plausible.

What would settle it

Train the speech adapter on one spoken QA dataset, such as SpokenSQuAD, and evaluate it zero-shot on another, such as VoxPopuli; if Recall@5 falls far below the ground-truth text baseline on the unseen dataset, the method's parity is an artifact of training on the evaluation distribution rather than a general alignment. A second check is to hold out a random split of one dataset and compare the speech adapter against the text retriever on the same split, since the paper does not state whether such a split was used.

Watch

Extended reading notes

Core claim

The paper's central claim is that direct speech retrieval does not degrade relative to text retrieval on ground-truth transcripts, and that it outperforms cascaded systems that rely on ASR. The evidence is the retrieval table: on SpokenSQuAD the speech retriever reaches Recall@5 of 0.9702 against 0.9707 for the ground-truth text baseline and 0.8768 for the high-WER cascade; on VoxPopuli it reaches 0.9952 against 0.9942 and 0.7106 respectively. The paper also claims that conditioning a speech language model on the retrieved audio, rather than on ASR transcripts, yields better generated answers than cascaded text generators when the transcripts have high word error rate, with exact-match scores of 0.3522 vs 0.2684 on SpokenSQuAD and 0.8045 vs 0.4511 on VoxPopuli. The reason this is significant is that ASR errors, especially on named entities, are avoided entirely in the retrieval-and-generation path.

Load-bearing premise

The load-bearing premise is that the speech adapter is trained on ground-truth transcripts from the same datasets it is later evaluated on, so the parity with text retrieval may be an in-domain distillation effect rather than evidence that spoken passages in general can be embedded into the text retriever's space.

Editorial extensions

If this is right

  • Retrieval over spoken archives can be built without an ASR component, eliminating transcription-error propagation at the indexing and search stages.
  • The speech retriever's Recall@k is on par with ground-truth text retrieval, so for retrieval purposes audio can substitute for clean transcripts.
  • Under added Gaussian noise, the direct speech retriever holds up better than a cascade using the same speech encoder, except at very high noise on the TTS-based SpokenSQuAD set.
  • In high-WER settings, audio-conditioned generation with a speech language model answers more accurately than text-generation pipelines that read noisy transcripts.
  • The gap against low-WER cascades in generation remains, with SpeechRAG performing worse than both fully- and semi-cascaded systems when transcripts are clean.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the absence of a stated train/test split means the headline parity could be in-domain distillation; a held-out-corpus test would tell whether the adapter learns general text-to-speech alignment.
  • Editorial inference: the same frozen-retriever-plus-adapter pattern could be applied to other modalities, such as image or video, wherever paired text descriptions and a pre-trained encoder exist.
  • Editorial inference: if long-context audio is what limits the speech language model, retrieving shorter passages or adapting the SLM to multi-passage audio would likely narrow the low-WER generation gap.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes SpeechRAG, a framework for open-domain question answering over spoken passages that bypasses ASR at both retrieval and generation time. A HuBERT encoder plus a lightweight adapter is trained with cosine distillation (Eq. 1) to map speech into the embedding space of a frozen E5-Mistral text retriever; retrieval is done by cosine similarity between text queries and audio passage embeddings. Generation is performed by a frozen speech language model (Qwen-Audio-Chat) conditioned on retrieved audio passages. On SpokenSQuAD and a self-built VoxPopuli QA benchmark, the speech retriever achieves Recall@5 close to the ground-truth text baseline (0.9702 vs 0.9707 and 0.9952 vs 0.9942) and outperforms high-WER cascaded ASR baselines. The generation experiments show SpeechRAG outperforms high-WER cascades but underperforms low-WER cascades and the ground-truth text baseline, a result the paper reports honestly.

Significance. If the retrieval-parity claim is robust, the paper makes a useful contribution: it shows that a lightweight speech adapter plus a frozen LLM-based text retriever can avoid ASR error propagation in spoken retrieval, and it provides a concrete recipe that avoids expensive cross-modal contrastive pretraining. The noise-robustness experiment is a valuable additional result, and the paper is transparent about the generation gap against low-WER cascades. The main limitation is evidential: because the training/evaluation split is not specified, the core parity result may reflect in-domain distillation rather than generalizable cross-modal alignment, and the reported differences are presented without error bars or significance checks. The architecture and the honest reporting of the generation results are strengths, but the central retrieval claim needs stronger experimental support.

major comments (3)
  1. [Section IV-A and IV-B] The paper never states the train/validation/test partition for SpokenSQuAD or VoxPopuli. The adapter is trained with the cosine distillation loss in Eq. (1) using ground-truth transcript embeddings of passages from the same two datasets on which retrieval is evaluated, and Section IV-B mentions early stopping on validation loss but does not say whether the evaluation passages were excluded from training. If the evaluation passages overlap with the training set, the Recall@5 values in Table II (0.9702 on SpokenSQuAD, 0.9952 on VoxPopuli) can be achieved by memorizing the text embeddings of those exact passages, and the headline claim of parity with ground-truth text retrieval is not established as a generalization result. Please specify the split, ensure passage-disjoint training/evaluation, and report results on a held-out split.
  2. [Section V-A, Table II] The retrieval results are reported as single point estimates with no error bars, confidence intervals, or significance tests. The differences at issue are very small: the Speech Retriever is 0.0005 below the GT text baseline on SpokenSQuAD Recall@5 and 0.0010 above it on VoxPopuli Recall@5, while both methods are within 0.001 on most rows. Without knowing the number of queries, the variance across random seeds or dataset subsamples, and the significance of these differences, the claim that direct speech retrieval 'does not degrade' over the text baseline is not statistically supported. Please report uncertainty estimates and, ideally, significance tests for the key comparisons.
  3. [Section V-C and Table III] The generation experiments support the claim that SpeechRAG beats high-WER cascades, but the paper should also state explicitly in the abstract and conclusions that SpeechRAG underperforms low-WER cascades by large margins (e.g., on SpokenSQuAD, EM 0.3522 vs 0.5019 for Low WER fully-cascaded; on VoxPopuli, EM 0.8045 vs 0.7473 for Low WER fully-cascaded is closer, but LLM Correctness is lower). The current abstract says the framework 'outperforms cascaded text-based models when there is high WER,' which is accurate, but the broader framing should not imply that the proposed system is generally superior to cascades.
minor comments (4)
  1. [Section V-B] The noise robustness experiment in Fig. 2 is described as comparing 'text-based' versus the end-to-end retriever, but the text says the comparison is with 'the cascaded baseline that uses the same speech encoder as our end-to-end retriever.' Please clarify the exact baselines in the figure caption and text.
  2. [Section IV-D] The description of the Low WER cascaded baseline is vague: it says 'the case uses the same audio encoder used for the speech adapter,' but it is not clear which ASR model or decoding strategy produces the reported ~17-20% WER. Please specify the ASR model and how its outputs were obtained.
  3. [Section IV-B] The training configuration reports learning rate, batch size, gradient accumulation, and early stopping, but the number of training examples and the number of validation queries are not given. Please add dataset sizes and the number of query-passage pairs used for evaluation.
  4. [Section V-C] The sentence 'Table 3 shows how SpeechRAG avoids the corruption of context information' should reference Table III consistently (the text uses 'Table 3' while the table is numbered III).

Circularity Check

0 steps flagged · score 0.0 of 10

No circular steps found; SpeechRAG's retrieval parity is an empirical distillation-generalization result, and the unstated train/eval split is a reporting gap rather than a definitional reduction.

full rationale

I walked the paper's derivation chain. The speech adapter is trained with Eq. (1), L = 1 - cos(es, et), minimizing cosine distance between the speech embedding and the frozen text retriever's embedding of the ground-truth transcript. The headline result in Table II, that direct speech retrieval matches the GT text baseline, measures how well this learned adapter generalizes to the evaluation queries; it is an empirical outcome, not a logical consequence of the loss, unless the evaluation passages were used in training. The paper does not state the train/validation/test partition in Section IV-B, so the parity claim cannot be verified to be passage-disjoint; this is a completeness or experimental-design concern, not a circularity under the defined patterns. The comparisons against low- and high-WER cascaded baselines and the injected-noise experiment are anchored to external ASR transcripts and externally defined noise conditions, so they do not reduce to the paper's own fitted values. The only overlapping-author citation, [23], is used to motivate the adapter design, but the paper states the architecture and loss explicitly, so the citation is not load-bearing. No step in the claimed derivation chain is equivalent to its input by construction.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The adapter introduces no new physical or conceptual entities; the only free parameters are standard training hyperparameters. The core assumptions are the suitability of the frozen text retriever's embedding space and the pretrained speech and audio-language models.

free parameters (4)
  • learning_rate = 5e-5
    Adam optimizer learning rate for adapter fine-tuning; a standard hyperparameter, not load-bearing for the conceptual claim.
  • batch_size = 4 with gradient accumulation 16
    Effective batch size 64; standard training choice.
  • downsampling_factor = 4 (80ms frames)
    Temporal average pooling in the adapter, chosen to match transcript length.
  • training_epochs = 20 with patience 3
    Early stopping criterion with validation loss.
assumptions (5)
  • domain assumption Fine-tuning HuBERT-Large with the cosine distillation loss can extract the semantic content of speech needed for retrieval.
    The whole retrieval pipeline depends on the speech encoder being adaptable to semantic text embedding space; no formal guarantee is given.
  • domain assumption E5-Mistral-7B-Instruct's text embedding space is an appropriate shared space for cross-modal retrieval.
    The method projects speech into this pre-existing space and assumes it is semantically rich and alignable.
  • domain assumption Qwen-Audio-Chat can answer questions from retrieved audio passages without fine-tuning.
    The generator is used as-is; the paper relies on the model's pretrained audio understanding.
  • standard math Cosine embedding loss minimization aligns speech and text embeddings.
    Standard metric learning objective; not in dispute.
  • domain assumption The VoxPopuli QA pairs generated by LLM prompting are valid for evaluating open question answering.
    Queries are generated by Claude 3.5 from extracted named entities; no human validation is described.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Speech Retrieval-Augmented Generation without Automatic Speech Recognition." pith.science (2026). https://pith.science/paper/JREFRNA5

@misc{pith2026241216500,
  author       = {Pith},
  title        = {Pith review of: Speech Retrieval-Augmented Generation without Automatic Speech Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JREFRNA5}},
  note         = {Machine review of arXiv:2412.16500}
}
read the original abstract

One common approach for question answering over speech data is to first transcribe speech using automatic speech recognition (ASR) and then employ text-based retrieval-augmented generation (RAG) on the transcriptions. While this cascaded pipeline has proven effective in many practical settings, ASR errors can propagate to the retrieval and generation steps. To overcome this limitation, we introduce SpeechRAG, a novel framework designed for open-question answering over spoken data. Our proposed approach fine-tunes a pre-trained speech encoder into a speech adapter fed into a frozen large language model (LLM)--based retrieval model. By aligning the embedding spaces of text and speech, our speech retriever directly retrieves audio passages from text-based queries, leveraging the retrieval capacity of the frozen text retriever. Our retrieval experiments on spoken question answering datasets show that direct speech retrieval does not degrade over the text-based baseline, and outperforms the cascaded systems using ASR. For generation, we use a speech language model (SLM) as a generator, conditioned on audio passages rather than transcripts. Without fine-tuning of the SLM, this approach outperforms cascaded text-based models when there is high WER in the transcripts.

Figures

Figures reproduced from arXiv: 2412.16500 by the authors.

Figure 1
Figure 1. (a) Our speech retriever consists of an adapter that projects speech into the embedding space of the text retrieval model. During training, we use distillation from the text embedding of the transcript of the speech to refine our speech embedding. This allows us to leverage the frozen text retriever’s capacity during a similarity search with the query. During testing, we use the text branch (Fig 1a, left) to embed t… view at source ↗
Figure 2
Figure 2. Retrieval performance comparison with injected noise. We [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. VoxRAG: A Step Toward Transcription-Free RAG Systems in Spoken Question Answering

    cs.IR 2025-05 conditional novelty 5.0 of 10

    VoxRAG shows that a spoken query can retrieve topically relevant podcast segments via CLAP audio embeddings and FAISS search, with Recall@10 of 0.60 for somewhat relevant segments, though precise answers remain rare.

Reference graph

Works this paper leans on

34 extracted references · 28 canonical work pages · cited by 1 Pith paper

  1. [1]

    Retrieval- augmented generation for knowledge-intensive nlp tasks,

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela, “Retrieval- augmented generation for knowledge-intensive nlp tasks,” in Advances in Neural Information Processing Systems , H. Larochelle, M. Ranzato, R. Hadsell, M.F....

    work page 2020

  2. [2]

    MuRAG: Multimodal retrieval-augmented generator for open question answering over images and text,

    Wenhu Chen, Hexiang Hu, Xi Chen, Pat Verga, and William Cohen, “MuRAG: Multimodal retrieval-augmented generator for open question answering over images and text,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, Eds., Abu Dhabi, United Arab Emirates, Dec. 2022, pp....

    work page 2022

  3. [3]

    Robust multi model rag pipeline for documents containing text, table & images,

    Pankaj Joshi, Aditya Gupta, Pankaj Kumar, and Manas Sisodia, “Robust multi model rag pipeline for documents containing text, table & images,” 2024 3rd International Conference on Applied Artificial Intelligence and Computing (ICAAIC), pp. 993–999, 2024

    work page 2024

  4. [4]

    Spo- ken content retrieval—beyond cascading speech recognition with text retrieval,

    Lin-shan Lee, James Glass, Hung-yi Lee, and Chun-an Chan, “Spo- ken content retrieval—beyond cascading speech recognition with text retrieval,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 23, no. 9, pp. 1389–1420, 2015

    work page 2015

  5. [5]

    Retrieval and browsing of spoken content,

    Ciprian Chelba, Timothy J. Hazen, and Murat Saraclar, “Retrieval and browsing of spoken content,” IEEE Signal Processing Magazine , vol. 25, no. 3, pp. 39–49, 2008

    work page 2008

  6. [6]

    Robust speech recognition via large- scale weak supervision,

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever, “Robust speech recognition via large- scale weak supervision,” 2022

    2022

  7. [7]

    MTEB: Massive text embedding benchmark,

    Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers, “MTEB: Massive text embedding benchmark,” in Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics , Dubrovnik, Croatia, May 2023, pp. 2014– 2037, Association for Computational Linguistics

    work page 2023

  8. [8]

    OLISIA: a cascade system for spoken dialogue state tracking,

    Léo Jacqmin, Lucas Druart, Yannick Estève, Benoît Favre, Lina M Ro- jas, and Valentin Vielzeuf, “OLISIA: a cascade system for spoken dialogue state tracking,” in Proceedings of The Eleventh Dialog System Technology Challenge, Prague, Czech Republic, Sept. 2023, pp. 95–104, Association for Computational Linguistics

    work page 2023

Show all 34 references
  1. [9]

    Towards end-to-end spoken language understanding,

    Dmitriy Serdyuk, Yongqiang Wang, Christian Fuegen, Anuj Kumar, Baiyang Liu, and Yoshua Bengio, “Towards end-to-end spoken language understanding,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2018, pp. 5754–5758

    2018

  2. [10]

    Why aren’t we NER yet? artifacts of ASR errors in named entity recognition in spontaneous speech transcripts,

    Piotr Szyma ´nski, Lukasz Augustyniak, Mikolaj Morzy, Adrian Szym- czak, Krzysztof Surdyk, and Piotr ˙Zelasko, “Why aren’t we NER yet? artifacts of ASR errors in named entity recognition in spontaneous speech transcripts,” in Proceedings of the 61st Annual Meeting of the Assoc...

    2023

  3. [11]

    Learning transferable visual models from natural language supervision,

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever, “Learning transferable visual models from natural language supervision,” in International C...

    2021

  4. [12]

    Large-scale contrastive language- audio pretraining with feature fusion and keyword-to-caption augmen- tation,

    Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg- Kirkpatrick, and Shlomo Dubnov, “Large-scale contrastive language- audio pretraining with feature fusion and keyword-to-caption augmen- tation,” in ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech ...

    2023

  5. [13]

    Clap learning audio concepts from natural language supervision,

    Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huam- ing Wang, “Clap learning audio concepts from natural language supervision,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2023, pp. 1–5

    2023

  6. [14]

    Contrastive learning with hard negative samples,

    Joshua David Robinson, Ching-Yao Chuang, Suvrit Sra, and Stefanie Jegelka, “Contrastive learning with hard negative samples,” in Interna- tional Conference on Learning Representations , 2021

    2021

  7. [15]

    Why do we need large batchsizes in contrastive learning? a gradient-bias perspective,

    Changyou Chen, Jianyi Zhang, Yi Xu, Liqun Chen, Jiali Duan, Yiran Chen, S. Tran, Belinda Zeng, and Trishul M. Chilimbi, “Why do we need large batchsizes in contrastive learning? a gradient-bias perspective,” in Neural Information Processing Systems , 2022

    2022

  8. [16]

    SONAR: sentence-level multimodal and language-agnostic represen- tations,

    Paul-Ambroise Duquenne, Holger Schwenk, and Benoit Sagot, “SONAR: sentence-level multimodal and language-agnostic represen- tations,” 2023

    2023

  9. [17]

    Audio retrieval with natural language queries,

    Andreea-Maria Oncescu, A. Sophia Koepke, João F. Henriques, Zeynep Akata, and Samuel Albanie, “Audio retrieval with natural language queries,” 2021

    2021

  10. [18]

    Speechbert: Cross-modal pre-trained language model for end-to-end spoken question answering,

    Yung-Sung Chuang, Chi-Liang Liu, and Hung yi Lee, “Speechbert: Cross-modal pre-trained language model for end-to-end spoken question answering,” ArXiv, vol. abs/1910.11559, 2019

    1910 arXiv

  11. [19]

    Recap: Retrieval-augmented audio captioning,

    Sreyan Ghosh, Sonal Kumar, Chandra Kiran Reddy Evuru, Ramani Duraiswami, and Dinesh Manocha, “Recap: Retrieval-augmented audio captioning,” in ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2024, pp. 1161– 1165

    2024

  12. [20]

    Retrieval augmented end-to-end spoken dialog models,

    Mingqiu Wang, Izhak Shafran, Hagen Soltau, Wei Han, Yuan Cao, Dian Yu, and Laurent El Shafey, “Retrieval augmented end-to-end spoken dialog models,” ArXiv, vol. abs/2402.01828, 2024

    2024 arXiv

  13. [21]

    Speechdpr: End-to-end spoken passage retrieval for open-domain spoken question answering,

    Chyi-Jiunn Lin, Guan-Ting Lin, Yung-Sung Chuang, Wei-Lun Wu, Shang-Wen Li, Abdelrahman Mohamed, Hung yi Lee, and Lin shan Lee, “Speechdpr: End-to-end spoken passage retrieval for open-domain spoken question answering,” 2024

    2024

  14. [22]

    Retrieval augmented end-to-end spoken dialog models,

    Mingqiu Wang, Izhak Shafran, Hagen Soltau, Wei Han, Yuan Cao, Dian Yu, and Laurent El Shafey, “Retrieval augmented end-to-end spoken dialog models,” 2024

    2024

  15. [23]

    Speechverse: A large-scale generalizable audio language model,

    Nilaksh Das, Saket Dingliwal, Srikanth Ronanki, Rohit Paturi, Zhaocheng Huang, Prashant Mathur, Jie Yuan, Dhanush Bekal, Xing Niu, Sai Muralidhar Jayanthi, Xilai Li, Karel Mundnich, Monica Sunkara, Sundararajan Srinivasan, Kyu J Han, and Katrin Kirchhoff, “Speechverse: A large...

    2024

  16. [24]

    Hubert: Self- supervised speech representation learning by masked prediction of hidden units,

    Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakho- tia, Ruslan Salakhutdinov, and Abdelrahman Mohamed, “Hubert: Self- supervised speech representation learning by masked prediction of hidden units,” 2021

    2021

  17. [25]

    Prompting large language models with audio for general-purpose speech summarization,

    Wonjune Kang and Deb Roy, “Prompting large language models with audio for general-purpose speech summarization,” 2024

    2024

  18. [26]

    Sentence-bert: Sentence embeddings using siamese bert-networks,

    Nils Reimers and Iryna Gurevych, “Sentence-bert: Sentence embeddings using siamese bert-networks,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing . 11 2019, Association for Computational Linguistics

    2019

  19. [27]

    Improving text embeddings with large language models,

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Ma- jumder, and Furu Wei, “Improving text embeddings with large language models,” 2024

    2024

  20. [28]

    Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models,

    Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou, “Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models,” arXiv preprint arXiv:2311.07919 , 2023

    2023 arXiv

  21. [29]

    Spoken squad: A study of mitigating the impact of speech recognition errors on listening comprehension,

    Chia-Hsuan Lee, Szu-Lin Wu, Chi-Liang Liu, and Hung-yi Lee, “Spoken squad: A study of mitigating the impact of speech recognition errors on listening comprehension,” Proc. Interspeech 2018, pp. 3459–3463, 2018

    2018

  22. [30]

    V oxPopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation,

    Changhan Wang, Morgane Riviere, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, and Emmanuel Dupoux, “V oxPopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation,” in Proceedings...

    2021

  23. [31]

    SQuAD: 100,000+ questions for machine comprehension of text,

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang, “SQuAD: 100,000+ questions for machine comprehension of text,” in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, Nov. 2016, pp. 2383–2392, Asso- ciation f...

    2016

  24. [32]

    Introduction to the CoNLL-2003 shared task: Language-independent named entity recogni- tion,

    Erik F. Tjong Kim Sang and Fien De Meulder, “Introduction to the CoNLL-2003 shared task: Language-independent named entity recogni- tion,” in Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003 , 2003, pp. 142–147

    2003

  25. [33]

    Evaluation of rag metrics for question answering in the telecom domain,

    Sujoy Roychowdhury, Sumit Soman, H G Ranjani, Neeraj Gunda, Vansh Chhabra, and Sai Krishna Bala, “Evaluation of rag metrics for question answering in the telecom domain,” 2024

    2024

  26. [34]

    Librispeech: An asr corpus based on public domain audio books,

    Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur, “Librispeech: An asr corpus based on public domain audio books,” in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp. 5206–5210

    2015

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.