Text-based Audio Retrieval by Learning from Similarities between Audio Captions

Huang Xie; Khazar Khorrami; Okko R\"as\"anen; Tuomas Virtanen

REVIEW 3 major objections 4 minor 28 references

Text-based Audio Retrieval by Learning from Similarities between Audio Captions

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that graded audio-caption relevance scores computed from the textual similarity of captions, trained with a listwise ranking objective, improve text-based audio retrieval over binary-relevance InfoNCE training.

desk verdict A solid, honest paper showing a modest but real gain from soft relevance labels in audio retrieval, with one load-bearing proxy assumption left unvalidated. read the letter →

arxiv 2412.01356 v1 pith:5JC7MTSO submitted 2024-12-02 eess.AS

Huang Xie , Khazar Khorrami , Okko Räsänen , Tuomas Virtanen This is my paper · ORCID

classification eess.AS

keywords text-basedaudioretrievalaudio-captionrelevancenon-binarytextualsimilaritylistwiserankingsentenceembeddingscontrastivelearningaudio-language

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that text-based audio retrieval is best trained with graded, non-binary relevance scores rather than the binary matched/not-matched labels used by contrastive methods such as CLAP. It derives these scores from caption text: captions with high Sentence-BERT cosine similarity are assumed to describe similar audio, so each audio sample is treated as partially relevant to captions other than its own. A dual-encoder model is trained with a listwise ranking objective, ListNet, that compares the full predicted relevance list to these computed scores. On Clotho this raises mAP@10 from 28.2 to 30.4, and after pretraining on WavCaps the gains on both Clotho and AudioCaps are statistically significant; the AudioCaps gain without pretraining is small and not significant.

What carries the argument

The central mechanism is caption-to-caption textual similarity used as a proxy for audio-to-caption relevance. Sentence-BERT maps each caption to an embedding, cosine similarity gives $h(x_i,x_j)$, and the logistic function in Eq. (5) converts that similarity into a graded relevance score $g(x_i,y_j)$ in $[0,1]$, so an audio sample inherits partial relevance to every caption whose text resembles its own caption. The ListNet objective then turns the target scores and the model's predicted cosine-similarity scores into two softmax distributions over audio samples and minimizes their cross-entropy, replacing InfoNCE's binary positive/negative sampling with a full-list ranking comparison.

What would settle it

A reader could test this by taking a held-out set of audio-caption pairs with human graded relevance, computing the paper's text-similarity-based scores, and checking whether they predict the human ratings; if they do not, or if replacing the computed scores with random soft labels leaves retrieval performance unchanged, the central claim would be contradicted.

Watch

Extended reading notes

Core claim

The central claim is that relevance between audio and arbitrary captions can be inferred from caption text alone, without additional human annotation. For a caption pair $(x_i,x_j)$, the paper computes the cosine similarity $h(x_i,x_j)$ of their Sentence-BERT embeddings and maps it through the logistic function $g(x_i,y_j)=1/(1+e^{2.73-4.58\,h(x_i,x_j)})$, giving each audio sample a graded relevance to every caption in the training set. The dual encoder is then trained by minimizing the cross-entropy between the softmax distribution of these target scores and the softmax distribution of the model's predicted scores, a listwise ranking objective. In experiments this yields mAP@10 gains on Clotho (30.4 vs. 28.2) and AudioCaps (55.0 vs. 54.5) without pretraining, and significant gains on both datasets when the model is first pretrained on WavCaps; the AudioCaps-only gain without pretraining is not statistically significant.

Load-bearing premise

The load-bearing premise is that captions that are textually similar describe audio content similar enough to make each caption relevant to the other's audio; this transitivity through text is assumed, not validated against audio content.

Editorial extensions

If this is right

Existing audio-caption datasets can be used to train retrieval with graded relevance without collecting new human relevance judgments, because the relevance scores are derived from caption text.
The same training recipe improves audio-based text retrieval, ranking captions for a given audio query, on both AudioCaps and Clotho.
The method is model-agnostic and can be applied to any captioning dataset, including image-caption data, for cross-modal retrieval.
After large-scale pretraining on WavCaps, graded-relevance training with ListNet yields statistically significant gains over the InfoNCE baseline on both AudioCaps and Clotho.
InfoNCE can be seen as a special case of the listwise objective in which the target distribution puts all probability mass on one sample, which helps explain why the graded target distribution is the source of the improvement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the transitivity-through-text assumption holds at scale, soft relevance labels could be mined cheaply from weakly labeled or synthetic caption sets, reducing the annotation cost of audio-language pretraining.
The lack of a significant AudioCaps gain without pretraining hints that the benefit depends on dataset properties such as caption redundancy and audio diversity; a controlled study varying those properties could test that.
A symmetric test using audio-side similarity from a pretrained audio encoder to generate graded relevance labels would isolate where the textual transitivity assumption starts to fail.
One practical extension is to calibrate the logistic mapping separately for each dataset or caption distribution rather than transferring a curve fitted on a Clotho subset.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Xie et al. propose replacing binary audio-caption relevance with soft labels computed from caption-caption textual similarity: g(xi,yj) is a logistic transform of Sentence-BERT cosine similarity between captions, and training uses a ListNet ranking objective. That specific estimation route is new for audio retrieval, and the paper is clear about what it does. The experiments are properly repeated (five runs, paired t-tests), and the main comparisons are meaningful: ListNet audio beats InfoNCE on Clotho text-based retrieval (mAP@10 30.4 vs 28.2, p<0.001), and after WavCaps pretraining it beats InfoNCE on both AudioCaps and Clotho. The gains are modest, one to two mAP@10 points, but they are credible and consistent with the motivation about false negatives in binary contrastive training.

Credit where due: the paper states its proxy assumption explicitly, uses a principled listwise objective, and reports the nonsignificant AudioCaps result rather than hiding it in the main table. The ablation against min-max scaling for f is also decent.

The soft spots are real, and the main one is load-bearing. Section III-A says captions with high textual similarity can describe each other's audio content. All computed relevances rest on that transitivity, and the paper never validates it by checking, say, whether high-similarity captions from different audio samples actually share acoustic content. The logistic mapping in Eq. (5) comes from the authors' earlier crowdsourced Clotho ratings, and its transfer to AudioCaps and WavCaps is assumed; caption-similarity distributions may differ. If the premise is wrong, the training targets are text-text similarities, not audio-caption relevances, and the gain could be soft-label regularization rather than recovered relevance. That would not make the empirical result useless, but it would change what the paper claims.

Two smaller issues. The AudioCaps gain without pretraining is not significant (t=0.532, p>0.05), and the abstract does not qualify this; read strictly, the abstract overstates. And the paper does not compare against [16], the estimated-correspondence baseline from the same DCASE line, which is the natural competing way to get non-binary relevance; omitting it weakens the claim that caption similarity is the useful signal. No code or data are provided, which matters more because the relevance labels are cheap to recompute.

Who is this for: people working on audio-text retrieval and cross-modal soft-label training. It is a solid workshop-to-journal-level contribution, not a breakthrough. I would send it to peer review, conditional on a same-audio caption-similarity validation and a comparison with [16]. The central idea deserves referee time even though the effect size is modest.

Referee Report

3 major / 4 minor

Summary. The paper proposes a method for text-based audio retrieval in which training relevance labels are computed as non-binary scores from the textual similarity between captions. Sentence-BERT embeddings are used to measure caption similarity, a logistic function (Eq. 5) converts these similarities into audio-caption relevance scores, and a dual-encoder model is trained with a ListNet listwise ranking objective (Eqs. 2-4). Experiments on AudioCaps and Clotho, with and without WavCaps pretraining, compare the proposed 'ListNet audio' approach against an InfoNCE binary-relevance baseline. The paper reports significant gains on Clotho (mAP@10 30.4 vs 28.2) and significant gains on both datasets after pretraining, while the AudioCaps gain without pretraining is not significant. The method is also adapted to audio-based text retrieval.

Significance. If the central proxy assumption is valid, the method is significant because it provides a cheap, scalable way to generate soft relevance labels for audio-text retrieval without expensive human annotation. The experimental protocol is a strength: five independent runs, paired t-tests, and a pretraining/fine-tuning scenario are all included, and the supplementary details of the dual-encoder architecture and training configuration are clearly stated. The main weakness is that the load-bearing premise—that high textual similarity between captions implies cross-relevance of their paired audio clips—is asserted in Section III-A but never validated against audio content. The paper also over-generalizes its conclusions given the non-significant AudioCaps result without pretraining. With additional validation and more careful claims, the method would be a useful contribution to audio-language learning.

major comments (3)

[Section III-A, Eq. (1), Eq. (5)] The entire method rests on the premise stated in Section III-A: 'audio captions exhibiting high mutual textual similarity can be appropriate descriptions of each other's audio contents.' This premise is never tested against audio content. The computed relevance g(xi, yj) is purely a function of the text-text similarity h(xi, xj), so if two captions are textually similar but describe different audio events, the training labels are wrong. I ask the authors to validate the premise directly: for example, on AudioCaps and Clotho, which have multiple captions per audio clip, one can test whether captions of the same audio have significantly higher h(xi, xj) than captions of different audio clips, or have human annotators judge a sample of the computed relevance scores. Without such evidence, the observed retrieval gains could be due to soft-label regularization rather than to the recovery of true audio-caption relevance, which is the paper's claimed mechanism.
[Section V-B, Table II, Section VI] The paper's conclusions overstate the empirical support. The paired t-test reported in Section V-B shows that ListNetaudio improves significantly on Clotho (t(5224)=11.658, p<0.001), but on AudioCaps the improvement is not significant (t(4699)=0.532, p>0.05). Nevertheless, Section VI states that 'experimental results validated the effectiveness of the proposed method in both text-based audio retrieval and audio-based text retrieval.' The audio-based text retrieval results in Table II are not significance-tested at all, and the ListNetaudio+text comparison against InfoNCE on AudioCaps (54.7 vs 54.5) is not significance-tested. The supported claims are limited to Clotho without pretraining and to both datasets with pretraining; the conclusions should be revised accordingly, and significance tests should be provided for all central comparisons.
[Section V-A, Eq. (5)] The logistic function in Eq. (5) was fitted in the authors' prior crowdsourcing study [14] on a limited Clotho subset, and the paper transfers this mapping to AudioCaps and WavCaps without checking whether the distribution of caption similarities h(xi, xj) is similar across datasets. If the similarity distribution differs, the relevance scores will be miscalibrated. The paper should report the distribution of h(xi, xj) for each dataset or otherwise justify the transferability of the fitted parameters. This is load-bearing for the claim that the method generalizes beyond the dataset on which the mapping was calibrated.

minor comments (4)

[Section III-B, Eqs. (2)-(4)] The notation uses Y as if it were the full training set, but the experimental setup in Section IV-B uses mini-batches of 32 audio-caption pairs. Please clarify that Y in Eqs. (2)-(4) denotes the mini-batch during training.
[Section V-B] The sentence 'ListNetaudio+text utilizing computed non-binary relevances outperforms InfoNCE' is not supported by significance testing; on AudioCaps the difference is 54.7 vs 54.5 with overlapping standard deviations. Please either add significance tests or soften the claim.
[Section V-A, Eq. (5)] The logistic mapping gives g(xi, yi) = 0.864 for the annotated pair (since h(xi, xi)=1), so the diagonal is not assigned relevance 1.0. This is presumably intentional because human relevance ratings may be below 1, but it should be explicitly stated and justified.
[Section IV-D] The evaluation metrics are described in terms of 'relevant items' but it is not explicitly stated that, at test time, the ground truth is binary relevance defined by the annotated caption(s). Please make this explicit.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: soft relevance targets are constructed from caption similarity, but retrieval claims are tested on held-out human annotations.

full rationale

The paper's central claim is that training with non-binary audio-caption relevance scores derived from caption-text similarity improves text-based audio retrieval over binary InfoNCE training. This claim is evaluated on AudioCaps and Clotho using standard retrieval metrics whose ground truth is the human-annotated audio-caption pairing, not the soft labels computed by the method. The soft targets g(xi,yj)=f(h(xi,xj)) in Eq. (1) are defined from caption similarity by construction, but the evaluation does not consist of predicting those same soft targets: it ranks held-out test audio against held-out test captions. The logistic mapping in Eq. (5) is imported from the authors' prior crowdsourced relevance study [14], which is external human-judgment data rather than a fitted parameter of the present retrieval experiments. Even if that mapping were set aside, the parameter-free min-max scaling variant also improves over InfoNCE on Clotho (29.0 vs 28.2 mAP@10), so the central improvement claim does not reduce to the fitted logistic parameters. The transitivity premise in Section III-A, that textually similar captions describe similar audio content, is an assumption about label validity rather than a circular derivation: the paper never claims to derive caption similarity from relevance, and the empirical gains are not definitionally forced. The self-citation [14] is load-bearing for choosing the calibration curve but is independently grounded in crowdsourced ratings, so it does not constitute circular evidence. Overall, the derivation chain runs from caption similarity to soft relevance targets to a listwise training objective, and the reported retrieval improvements are measured against independent held-out annotations.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a text-based proxy for audio relevance: caption similarity is assumed to transfer to audio content similarity. The logistic calibration is data-fit from prior crowdsourcing, and the training objective is a standard listwise ranking loss. No new physical or conceptual entities are introduced.

free parameters (2)

logistic function coefficients a and b in Eq. (5) = 2.73, 4.58
Fitted via beta regression to crowdsourced audio-caption relevance ratings from the authors' previous work [14]; used to map caption-text similarity to relevance.
ListNet temperature parameters omega and tau = 0.05
Set to 0.05 as temperature parameters in Eqs. (2) and (3); a hyperparameter choice.

assumptions (4)

domain assumption Caption text similarity is a reliable proxy for audio content similarity
Bridges the text-only computation of relevance to the audio modality; stated in Section III-A without independent validation.
domain assumption The crowdsourced relevance ratings in [14] provide a valid calibration for the logistic mapping
Used to derive Eq. (5) and assumed to transfer across audio-caption datasets, Section V-A.
standard math ListNet listwise ranking loss is an appropriate training objective for soft relevance labels
ListNet is a standard learning-to-rank method [17]; the paper uses it as its training objective.
domain assumption Sentence-BERT embeddings capture the semantic similarity of audio captions
The relevance scores are computed from Sentence-BERT cosine similarities; no audio-specific adaptation is applied, Section III-A.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Text-based Audio Retrieval by Learning from Similarities between Audio Captions." pith.science (2026). https://pith.science/paper/5JC7MTSO

@misc{pith2026241201356,
  author       = {Pith},
  title        = {Pith review of: Text-based Audio Retrieval by Learning from Similarities between Audio Captions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5JC7MTSO}},
  note         = {Machine review of arXiv:2412.01356}
}

read the original abstract

This paper proposes to use similarities of audio captions for estimating audio-caption relevances to be used for training text-based audio retrieval systems. Current audio-caption datasets (e.g., Clotho) contain audio samples paired with annotated captions, but lack relevance information about audio samples and captions beyond the annotated ones. Besides, mainstream approaches (e.g., CLAP) usually treat the annotated pairs as positives and consider all other audio-caption combinations as negatives, assuming a binary relevance between audio samples and captions. To infer the relevance between audio samples and arbitrary captions, we propose a method that computes non-binary audio-caption relevance scores based on the textual similarities of audio captions. We measure textual similarities of audio captions by calculating the cosine similarity of their Sentence-BERT embeddings and then transform these similarities into audio-caption relevance scores using a logistic function, thereby linking audio samples through their annotated captions to all other captions in the dataset. To integrate the computed relevances into training, we employ a listwise ranking objective, where relevance scores are converted into probabilities of ranking audio samples for a given textual query. We show the effectiveness of the proposed method by demonstrating improvements in text-based audio retrieval compared to methods that use binary audio-caption relevances for training.

Figures

Figures reproduced from arXiv: 2412.01356 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 26 canonical work pages

[16]

Estimated Audio-Caption Correspondences Improve Language-Based Audio Retrieval

P. Primus, F. Schmid, and G. Widmer, “Estimated Audio-Caption Cor- respondences Improve Language-Based Audio Retrieval,” 2024, arXiv preprint arXiv:2408.11641

work page Pith review arXiv 2024
[14]

Crowdsourcing and Evaluating Text-Based Audio Retrieval Relevances,

H. Xie, K. Khorrami, O. R ¨as¨anen, and T. Virtanen, “Crowdsourcing and Evaluating Text-Based Audio Retrieval Relevances,” in Proceedings of the 8th Detection and Classification of Acoustic Scenes and Events 2023 Workshop, 2023, pp. 226–230

work page 2023
[1]

Language-Based Audio Retrieval Task in DCASE 2022 Challenge,

H. Xie, S. Lipping, and T. Virtanen, “Language-Based Audio Retrieval Task in DCASE 2022 Challenge,” in Proceedings of the 7th Detection and Classification of Acoustic Scenes and Events 2022 Workshop , 2022, pp. 216–220

work page 2022
[2]

Improving Natural-Language-Based Audio Retrieval with Transfer Learning and Audio & Text Augmentations,

P. Primus and G. Widmer, “Improving Natural-Language-Based Audio Retrieval with Transfer Learning and Audio & Text Augmentations,” in Proceedings of the 7th Detection and Classification of Acoustic Scenes and Events 2022 Workshop , 2022, pp. 166–170

work page 2022
[3]

Matching Text and Audio Embeddings: Exploring Transfer-Learning Strategies for Language-Based Audio Retrieval,

B. Weck, M. P. Fern ´andez, H. Kirchhoff, and X. Serra, “Matching Text and Audio Embeddings: Exploring Transfer-Learning Strategies for Language-Based Audio Retrieval,” in Proceedings of the 7th Detection and Classification of Acoustic Scenes and Events 2022 Workshop , 2022, pp. 206–210

work page 2022
[4]

Large-Scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation,

Y . Wu, K. Chen, T. Zhang, Y . Hui, T. Berg-Kirkpatrick, and S. Dub- nov, “Large-Scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation,” in ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing, 2023, pp. 1–5

work page 2023
[5]

WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research,

X. Mei, C. Meng, H. Liu, Q. Kong, T. Ko, C. Zhao, M. D. Plumbley, Y . Zou, and W. Wang, “WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 32, pp. 3339–3354, 2024

work page 2024
[6]

Auto-ACD: A Large-scale Dataset for Audio-Language Representation Learning,

L. Sun, X. Xu, M. Wu, and W. Xie, “Auto-ACD: A Large-scale Dataset for Audio-Language Representation Learning,” in ACM Multimedia 2024, 2024

work page 2024

Show all 28 references

[7]

Advancing Natural-Language Based Audio Retrieval with Passt and Large Audio-Caption Data Sets,

P. Primus, K. Koutini, and G. Widmer, “Advancing Natural-Language Based Audio Retrieval with Passt and Large Audio-Caption Data Sets,” in Proceedings of the 8th Detection and Classification of Acoustic Scenes and Events 2023 Workshop , 2023, pp. 151–155

2023
[8]

Language-based Audio Retrieval in DCASE 2023 Challenge,

“Language-based Audio Retrieval in DCASE 2023 Challenge,” https: //dcase.community/challenge2023/task-language-based-audio-retrieval, accessed: 2024-04-25

2023
[9]

AudioCaps: Generating Cap- tions for Audios in The Wild,

C. D. Kim, B. Kim, H. Lee, and G. Kim, “AudioCaps: Generating Cap- tions for Audios in The Wild,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)...

2019
[10]

Clotho: an Audio Captioning Dataset,

K. Drossos, S. Lipping, and T. Virtanen, “Clotho: an Audio Captioning Dataset,” in ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing , 2020, pp. 736–740

2020
[11]

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,

N. Reimers and I. Gurevych, “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,” inProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing , 2019, pp. 3982–3992

2019
[12]

Graded Relevance,

T. Sakai, “Graded Relevance,” in Evaluating Information Retrieval and Access Tasks: NTCIR’s Legacy of Research Impact. Singapore: Springer Singapore, 2021, pp. 1–20

2021
[13]

On the effect of relevance scales in crowdsourcing relevance assessments for Information Retrieval evaluation,

K. Roitero, E. Maddalena, S. Mizzaro, and F. Scholer, “On the effect of relevance scales in crowdsourcing relevance assessments for Information Retrieval evaluation,” Information Processing & Management , vol. 58, no. 6, p. 102688, 2021

2021
[15]

Integrating Contin- uous and Binary Relevances in Audio-Text Relevance Learning,

H. Xie, K. Khorrami, O. R ¨as¨anen, and T. Virtanen, “Integrating Contin- uous and Binary Relevances in Audio-Text Relevance Learning,” 2024, arXiv preprint arXiv:2408.14939

2024 arXiv
[17]

Learning to Rank: From Pairwise Approach to Listwise Approach,

Z. Cao, T. Qin, T.-Y . Liu, M.-F. Tsai, and H. Li, “Learning to Rank: From Pairwise Approach to Listwise Approach,” in Proceedings of the 24th International Conference on Machine Learning, 2007, pp. 129–136

2007
[18]

Image-Text Retrieval with Binary and Continuous Label Supervision,

Z. Li, C. Guo, Z. Feng, J.-N. Hwang, Y . Jin, and Y . Zhang, “Image-Text Retrieval with Binary and Continuous Label Supervision,” 2022, arXiv preprint arXiv:2210.11319

2022 arXiv
[19]

Integrating Listwise Ranking into Pairwise-based Image-Text Retrieval,

Z. Li, C. Guo, X. Wang, H. Zhang, and Y . Wang, “Integrating Listwise Ranking into Pairwise-based Image-Text Retrieval,” Knowledge-Based Systems, vol. 287, p. 111431, 2024

2024
[20]

Freesound Technical Demo,

F. Font, G. Roma, and X. Serra, “Freesound Technical Demo,” in Proceedings of the 21st ACM International Conference on Multimedia , 2013, pp. 411–412

2013
[21]

BBC Sound Effects,

“BBC Sound Effects,” https://sound-effects.bbcrewind.co.uk, accessed: 2024-09-25

2024
[22]

SoundBible,

“SoundBible,” https://soundbible.com, accessed: 2024-09-25

2024
[23]

The Benefit of Temporally-Strong Labels in Audio Event Classification,

S. Hershey, D. P. W. Ellis, E. Fonseca, A. Jansen, C. Liu, R. Chan- ning Moore, and M. Plakal, “The Benefit of Temporally-Strong Labels in Audio Event Classification,” in ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing, 2021, pp. 366– 370

2021
[24]

Language-based Audio Retrieval in DCASE 2024 Challenge,

“Language-based Audio Retrieval in DCASE 2024 Challenge,” https: //dcase.community/challenge2024/task-language-based-audio-retrieval, accessed: 2024-09-25

2024
[25]

Efficient Training of Audio Transformers with Patchout,

K. Koutini, J. Schl ¨uter, H. Eghbal-zadeh, and G. Widmer, “Efficient Training of Audio Transformers with Patchout,” in Proceedings of the 23rd Annual Conference of the International Speech Communication Association, 2022, pp. 2753–2757

2022
[26]

RoBERTa: A Robustly Optimized BERT Pretraining Approach,

Y . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V . Stoyanov, “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” 2019, arXiv preprint arXiv:1907.11692

2019 arXiv
[27]

Representation Learning with Contrastive Predictive Coding,

A. Van Den Oord, Y . Li, and O. Vinyals, “Representation Learning with Contrastive Predictive Coding,” 2018, arXiv preprint arXiv:1807.03748

2018 arXiv
[28]

SGDR: Stochastic Gradient Descent with Warm Restarts,

I. Loshchilov and F. Hutter, “SGDR: Stochastic Gradient Descent with Warm Restarts,” in Proceedings of the 2017 International Conference on Learning Representations , 2017

2017

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [16]

Estimated Audio-Caption Correspondences Improve Language-Based Audio Retrieval

P. Primus, F. Schmid, and G. Widmer, “Estimated Audio-Caption Cor- respondences Improve Language-Based Audio Retrieval,” 2024, arXiv preprint arXiv:2408.11641

work page Pith review arXiv 2024

[2] [14]

Crowdsourcing and Evaluating Text-Based Audio Retrieval Relevances,

H. Xie, K. Khorrami, O. R ¨as¨anen, and T. Virtanen, “Crowdsourcing and Evaluating Text-Based Audio Retrieval Relevances,” in Proceedings of the 8th Detection and Classification of Acoustic Scenes and Events 2023 Workshop, 2023, pp. 226–230

work page 2023

[3] [1]

Language-Based Audio Retrieval Task in DCASE 2022 Challenge,

H. Xie, S. Lipping, and T. Virtanen, “Language-Based Audio Retrieval Task in DCASE 2022 Challenge,” in Proceedings of the 7th Detection and Classification of Acoustic Scenes and Events 2022 Workshop , 2022, pp. 216–220

work page 2022

[4] [2]

Improving Natural-Language-Based Audio Retrieval with Transfer Learning and Audio & Text Augmentations,

P. Primus and G. Widmer, “Improving Natural-Language-Based Audio Retrieval with Transfer Learning and Audio & Text Augmentations,” in Proceedings of the 7th Detection and Classification of Acoustic Scenes and Events 2022 Workshop , 2022, pp. 166–170

work page 2022

[5] [3]

Matching Text and Audio Embeddings: Exploring Transfer-Learning Strategies for Language-Based Audio Retrieval,

B. Weck, M. P. Fern ´andez, H. Kirchhoff, and X. Serra, “Matching Text and Audio Embeddings: Exploring Transfer-Learning Strategies for Language-Based Audio Retrieval,” in Proceedings of the 7th Detection and Classification of Acoustic Scenes and Events 2022 Workshop , 2022, pp. 206–210

work page 2022

[6] [4]

Large-Scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation,

Y . Wu, K. Chen, T. Zhang, Y . Hui, T. Berg-Kirkpatrick, and S. Dub- nov, “Large-Scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation,” in ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing, 2023, pp. 1–5

work page 2023

[7] [5]

WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research,

X. Mei, C. Meng, H. Liu, Q. Kong, T. Ko, C. Zhao, M. D. Plumbley, Y . Zou, and W. Wang, “WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 32, pp. 3339–3354, 2024

work page 2024

[8] [6]

Auto-ACD: A Large-scale Dataset for Audio-Language Representation Learning,

L. Sun, X. Xu, M. Wu, and W. Xie, “Auto-ACD: A Large-scale Dataset for Audio-Language Representation Learning,” in ACM Multimedia 2024, 2024

work page 2024