Pith. sign in

REVIEW 4 major objections 5 minor 29 references

Hybrid-Sep: Language-queried audio source separation via pre-trained Model Fusion and Adversarial Diffusion Training

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Hybrid-Sep claims a two-stage fusion of self-supervised audio features and CLAP semantics sets new benchmarks in language-queried audio separation.

desk verdict Strong two-stage LASS system with real SDR gains, but the ACT algorithm as written is degenerate and the benchmark claims need error bars and better baselines. read the letter →

arxiv 2506.16833 v1 pith:LZY243VW submitted 2025-06-20 cs.SD eess.AS

Jianyuan Feng , Guangzheng Li , Yangfei Xu This is my paper · ORCID
classification cs.SDeess.AS
keywords language-queriedaudiosourceseparationCLAPself-supervisedrepresentationBEATsadversarialconsistencytrainingembeddingtransformermodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Hybrid-Sep aims to improve language-queried audio source separation, the task of isolating a sound described in words from a mixture. Its central proposal is a two-stage design: first predict the target audio's embedding from the text query and the mixture, then separate using that audio embedding together with frame-level acoustic features from a self-supervised model. The paper argues that this bottleneck channels language through an audio-conditional representation, easing the ambiguity of direct text-to-separation conditioning, and that its adversarial-consistency training gives diffusion-style quality without multi-step sampling. If the reported results hold, the method outperforms existing systems such as AudioSep and FlowSep on signal quality, semantic similarity, and audio fidelity metrics.

What carries the argument

The load-bearing object is the two-stage conditioning chain and its training schedule. Stage one uses an Audio Embedding Transformer, a stack of 32 transformer layers, to map a frozen T-CLAP text embedding plus frame-wise features from a small STFT-based feature extractor into the target audio's T-CLAP embedding, trained with an L1 loss. Stage two trains the Audio Separation Model on that predicted embedding: the model splits the mixture into four PQMF subbands, converts them to spectrograms, concatenates them with frame features from a 90M-parameter frozen BEATs SSL encoder, and processes the hybrid representation with convolutional-recurrent modules, frequency-axis attention, and target-embedding-aware cross-attention. The Adversarial Consistent Training algorithm simultaneously optimizes an L1 reconstruction loss, a GAN loss from a multiscale spectrogram discriminator, and a consistency loss that asks a shared conditional denoiser to map two differently noise-corrupted versions of the target to the same clean estimate, with EMA schedules on noise level and loss weight drawn from consistency-model training. This conditional denoiser is what lets the paper claim diffusion-like regularization at single-step inference.

What would settle it

Take a held-out set of mixtures whose sound categories are absent from the AudioSet ontology, or replace BEATs with an SSL encoder pre-trained on a matched-size non-AudioSet corpus, then measure SDR, CLAPscore, and FAD against the reported numbers; if the gap over the one-stage model and over AudioSep narrows sharply outside the AudioSet domain, the domain overlap is the cause.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a strong language-queried separation system does not have to condition on text embeddings alone. By adding a first stage that maps a text query plus the mixed audio to the embedding of the target audio, and then feeding that predicted audio embedding into the separator along with BEATs frame features, Hybrid-Sep reports consistent gains across every metric and test set: for example, 8.82 dB SDR on the DE-S set versus 5.71 dB for AudioSep and -16.62 dB for FlowSep at 16 kHz. The paper further claims that the Adversarial Consistent Training schedule, which pairs a multiscale spectrogram discriminator with a conditional denoiser enforcing consistency under added noise, fixes the low-SDR tendency of diffusion-based separators while preserving their semantic accuracy, and that removing the SSL encoder, the denoiser, or the two-stage design each degrades performance.

Load-bearing premise

The system assumes that BEATs, pre-trained on AudioSet at 16 kHz, supplies acoustic features general enough for arbitrary separation inputs, and the keyword evaluation uses mixtures built from the same AudioSet domain; if that overlap inflates the results, the reported advantage may not transfer to open-world sounds.

Editorial extensions

If this is right

  • If the reported scores hold, language-queried separation no longer needs text-audio alignment to happen inside the separator: a predicted audio-embedding bottleneck can carry the semantic load.
  • Diffusion-style language-queried separation can regain amplitude fidelity, since the consistency-denoiser regularizer addresses the low-SDR failure mode of earlier generative sampling without multi-step inference.
  • Pre-trained self-supervised frame features, not only contrastive audio-text embeddings, contribute to separation quality, especially for keyword queries.
  • The system runs at 44.1 kHz, extending beyond the 32 kHz and 16 kHz ceilings of the prior baselines and widening applicability to full-band audio.
  • The two-stage design handles rarely seen sound sources, such as the few-shot 'Kora' example, better than direct text conditioning, suggesting robustness to data imbalance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the bottleneck is the reason for the gain, the same two-stage recipe of predicting the target embedding and then conditioning on it could transfer to other semantically conditioned reconstruction tasks such as text-to-speech or query-based sound enhancement, an extension the paper does not test.
  • The largest reported advantage appears on keyword-audio data drawn from the same AudioSet domain on which BEATs was pre-trained, so an open-world test on out-of-domain sounds would be the decisive check; the paper acknowledges the overlap in its discussion but does not run that experiment.
  • The EMA-consistency denoiser schedule could be read as a general single-step adversarial-consistency training recipe for conditional generation beyond audio, such as text-conditioned image or video generation, though the paper only demonstrates it for audio separation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Hybrid-Sep, a two-stage language-queried audio source separation (LASS) framework. Stage 1 trains an audio embedding transformer (AET) to predict the T-CLAP audio embedding of the target source from the text query and the mixture. Stage 2 trains an audio separation model (ASM) conditioned on this predicted embedding, using SSL-based BEATs features, an L1 reconstruction loss, a multiscale spectrogram discriminator, and a proposed Adversarial Consistent Training (ACT) schedule that includes a conditional denoiser with a consistency loss. The authors report experiments on caption-based and keyword-based test sets, claiming state-of-the-art SDR, CLAPscore, and FAD results over AudioSep and FlowSep, and provide ablations of the SSL encoder, the conditional denoiser, and the two-stage design.

Significance. If the reported results hold, the two-stage design of predicting a target audio embedding before separation is a useful contribution, and combining SSL acoustic features with CLAP-derived embeddings is a plausible direction for LASS. The paper also demonstrates full-band, 44.1 kHz separation, which is a practical advancement over the 16 kHz and 32 kHz limits of the compared baselines. The large-scale training setup and the inclusion of ablations are strengths. However, the central methodological novelty, ACT, is described in Algorithm 1 in a form that appears degenerate, and the evaluation evidence is based on a single table without error bars or significance tests, with one semantic metric closely aligned to the stage-1 training objective. The significance is therefore conditional on resolving these issues.

major comments (4)
  1. The consistency loss in Algorithm 1 does not train the conditional denoiser CD to denoise toward any target signal. L_consist = d[CD(X_noisy1,c), CD(X_noisy2,c).detach()] only enforces that the outputs for two noise-corrupted versions of X_target agree with each other; there is no term such as ||CD(X_noisy,c) - X_target|| or ||CD(X_noisy,c) - X_gen||. Any constant function satisfies this loss exactly, and the gradient through c does not carry information about X_target because X_target appears only in the inputs X_noisy1/X_noisy2 rather than in a target-anchored loss. Unlike the consistency-model objective of Song et al., which anchors outputs to the data at the ODE endpoint, no boundary condition is specified. As written, ACT cannot explain the reported improvements attributed to the CD, and the manuscript must either add the missing reconstruction term to the algorithm or clarify the actual training objective implemented in the code.
  2. The performance claims rest on a single table with no error bars, confidence intervals, or significance tests. This is particularly problematic for the CD ablation: the full Hybrid-Sep model achieves 8.77 dB SDR and 80.4 CLAPscoreA at 44.1 kHz, while Hybrid-Sep w/o CD achieves 8.74 dB SDR and 80.3 CLAPscoreA. These differences are far smaller than the text's characterization of "significant gains in semantic metrics" and could easily be within run-to-run variation. The authors should report multiple seeds or at least provide a statistical comparison before claiming that ACT provides the reported improvement.
  3. The semantic metric CLAPscoreA is closely aligned with the stage-1 training objective. Stage 1 trains the AET to predict the T-CLAP audio embedding of the target audio from the text query and mixture, and the ASM is conditioned on that predicted embedding; CLAPscoreA measures cosine similarity between the T-CLAP embedding of the separated audio and the T-CLAP embedding of the target audio. This creates a circularity for the semantic-quality claim: the model is explicitly trained to produce outputs whose T-CLAP embeddings match the target embedding. While SDR and FAD provide independent evidence, the paper should either use CLAPscoreA only as a secondary metric or justify why it is not a self-fulfilling objective.
  4. The baseline comparison is limited to AudioSep and FlowSep, while the DCASE 2024 first-prize system is mentioned only in the discussion without being included in Table 2. In addition, the keyword-audio test set is constructed from AudioSet evaluation data, and the BEATs encoder used in Hybrid-Sep is pre-trained on AudioSet, so the reported keyword-domain advantage may be inflated by domain overlap. Section 4 itself acknowledges this limitation. A comparison against the DCASE winner and a more thorough discussion of the AudioSet-domain overlap would be needed to substantiate the claimed state-of-the-art status.
minor comments (5)
  1. The abstract contains 'Demo can be find in web'; this should read 'Demo can be found on the web' or similar.
  2. The phrase 'CLAP shows an advantage in sematic-level audio tasks' contains a typo: 'sematic' should be 'semantic'.
  3. The name 'Hybried-Sep' appears in the first sentence; this should be 'Hybrid-Sep'.
  4. The text says 'The overlook of the training and inference process' but likely means 'overview'; please correct.
  5. The table caption would benefit from stating the numbers of test samples and clarifying that the 44.1 kHz rows are ablations, not comparisons to the 16 kHz baselines.

Circularity Check

1 steps flagged · score 6.0 of 10

ACT consistency loss is self-referential: CD is trained to match CD's own output, so the claimed CD-driven semantic gains reduce by construction; SDR/FAD comparisons remain independent.

  1. self definitional [Section 2.2, Algorithm 1 (consistency loss); ablation 'Hybrid-Sep w/o CD' in Table 2]
    "Denoise: X_pred1<-CD(X_noisy1,c), X_pred2<-CD(X_noisy2,c) ... Consistency loss: L_consist<-d[X_pred1, X_pred2.detach()] ... L1 loss: L_L1<-||X_gen-X_target||1"

    In Algorithm 1, CD receives gradients only from L_consist, and the detached target X_pred2 is itself produced by CD on a second noise draw; there is no term such as ||CD(X_noisy,c)-X_target|| and L_L1 applies only to X_gen=ASM(X_mixed). The consistency objective is therefore satisfied by any constant function, and 'denoising consistency' is defined circularly as the model agreeing with itself. Consequently the paper's attribution that 'The introduction of consistency loss yields significant gains in semantic evaluation metrics' (Table 2: CLAPscoreA 79.9 w/o CD vs 80.4 with CD; FAD 1.04 vs 0.85) cannot follow from the stated loss; the reported CD benefit would require an unstated reconstruction or anchor term.

full rationale

The one exhibited reduction-by-construction is the ACT consistency loss: its regression target is CD's own output, so the CD contribution to the reported gains is self-defined rather than derived from the data or text condition. I do not count the stage-1 prediction of T-CLAP target embeddings as circular, because CLAPscoreA is evaluated on the separated waveform, not on the fitted embedding, and SDR and FAD provide independent evidence. The paper contains no load-bearing self-citation chain or imported uniqueness theorem, and the acknowledged AudioSet/BEATs domain overlap is a stated limitation rather than a circular step. Since the central SDR/architecture comparisons stand independently of the degenerate CD loss, the circularity is partial, not total.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central method relies on a large set of hand-chosen hyperparameters and borrowed components such as T-CLAP, BEATs, the Xcodec discriminator, and consistency-model schedules. The most consequential free parameters are the loss weights and schedule functions, which are not fully specified, impeding reproduction.

free parameters (5)
  • lambda_L1 (L1 loss weight) = 1
    Weight for the waveform L1 loss in ACT, set to 1 without a sensitivity study.
  • lambda_consist(k) and sigma(k) schedules = not specified (copied from consistency model training)
    EMA schedules for consistency loss weight and noise level; exact functions are not reported, making the method non-reproducible.
  • PQMF subband count = 4
    Number of subbands in the filter bank; a manually chosen architectural parameter.
  • TRCNN channel sizes = [48, 96, 192, 384, 192, 96, 48, 8]
    Hand-designed channel dimensions for the temporal recurrent CNN blocks.
  • training steps per stage = 60000
    Number of optimization steps for each stage; chosen without reported convergence criteria.
assumptions (5)
  • domain assumption CLAP/T-CLAP audio embeddings provide a semantically meaningful and complete target representation for guiding source separation.
    Used throughout stage 1 and stage 2 (Sections 2.1, 2.2); if false, the two-stage conditioning mechanism has no sound basis.
  • domain assumption Consistency loss on the conditional denoiser improves separation fidelity and semantic alignment.
    The ACT algorithm (Section 2.2) relies on this transfer of consistency training from generative models to separation; no proof is provided.
  • domain assumption BEATs SSL features, pre-trained on AudioSet, generalize to unseen audio and separation contexts.
    Invoked in Section 2.2 for the ASM input; the authors acknowledge in Section 4 that AudioSet-domain overlap may boost keyword test results.
  • domain assumption The multiscale spectrogram discriminator design from Xcodec transfers to the LASS task.
    Stated in Section 2.2: the structure of the discriminator and loss design remains the same as it is in Xcodec.
  • standard math Standard definitions and mathematics of STFT, PQMF, transformers, and GAN losses.
    Background assumed for all computations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hybrid-Sep: Language-queried audio source separation via pre-trained Model Fusion and Adversarial Diffusion Training." pith.science (2026). https://pith.science/paper/LZY243VW

@misc{pith2026250616833,
  author       = {Pith},
  title        = {Pith review of: Hybrid-Sep: Language-queried audio source separation via pre-trained Model Fusion and Adversarial Diffusion Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LZY243VW}},
  note         = {Machine review of arXiv:2506.16833}
}
read the original abstract

Language-queried Audio Separation (LASS) employs linguistic queries to isolate target sounds based on semantic descriptions. However, existing methods face challenges in aligning complex auditory features with linguistic context while preserving separation precision. Current research efforts focus primarily on text description augmentation and architectural innovations, yet the potential of integrating pre-trained self-supervised learning (SSL) audio models and Contrastive Language-Audio Pretraining (CLAP) frameworks, capable of extracting cross-modal audio-text relationships, remains underexplored. To address this, we present HybridSep, a two-stage LASS framework that synergizes SSL-based acoustic representations with CLAP-derived semantic embeddings. Our framework introduces Adversarial Consistent Training (ACT), a novel optimization strategy that treats diffusion as an auxiliary regularization loss while integrating adversarial training to enhance separation fidelity. Experiments demonstrate that HybridSep achieves significant performance improvements over state-of-the-art baselines (e.g., AudioSep, FlowSep) across multiple metrics, establishing new benchmarks for LASS tasks.

Figures

Figures reproduced from arXiv: 2506.16833 by the authors.

Figure 1
Figure 1. 2 stage training of the Hybrid-Sep [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Model structure of audio separation model and conditional [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Case study of different LASS models’ performance in DS-S [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Case study of an audio source trained with small samples [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 13 canonical work pages

  1. [1]

    Separate anything you describe,

    X. Liu, Q. Kong, Y . Zhao, H. Liu, Y . Yuan, Y . Liu, R. Xia, Y . Wang, M. D. Plumbley, and W. Wang, “Separate anything you describe,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024

    work page 2024

  2. [2]

    Exploring text-queried sound event detection with audio source separation,

    H. Yin, J. Bai, Y . Xiao, H. Wang, S. Zheng, Y . Chen, R. K. Das, C. Deng, and J. Chen, “Exploring text-queried sound event detection with audio source separation,” inICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2025, pp. 1–5

    work page 2025

  3. [3]

    Performance improvement of language-queried audio source separation based on caption augmentation from large language models for dcase challenge 2024 task 9,

    D. H. Lee, Y . Song, and H. K. Kim, “Performance improvement of language-queried audio source separation based on caption augmentation from large language models for dcase challenge 2024 task 9,”arXiv preprint arXiv:2406.11248, 2024

    arXiv 2024

  4. [4]

    Language-queried audio source separation enhanced by expanded language-audio contrastive loss,

    H. C. Chung and J. H. Jung, “Language-queried audio source separation enhanced by expanded language-audio contrastive loss,” DCASE2024 Challenge, Tech. Rep, Tech. Rep., 2024

    work page 2024

  5. [5]

    Flowsep: Language-queried sound separation with rectified flow matching,

    Y . Yuan, X. Liu, H. Liu, M. D. Plumbley, and W. Wang, “Flowsep: Language-queried sound separation with rectified flow matching,” in ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2025, pp. 1–5

    work page 2025

  6. [6]

    A reference-free metric for language-queried audio source separation using contrastive language-audio pretraining,

    F. Xiao, J. Guan, Q. Zhu, X. Liu, W. Wang, S. Qi, K. Zhang, J. Sun, and W. Wang, “A reference-free metric for language-queried audio source separation using contrastive language-audio pretraining,”arXiv preprint arXiv:2407.04936, 2024

    arXiv 2024

  7. [7]

    Fr \’echet audio distance: A metric for evaluating music enhancement algorithms,

    K. Kilgour, M. Zuluaga, D. Roblek, and M. Sharifi, “Fr \’echet audio distance: A metric for evaluating music enhancement algorithms,”arXiv preprint arXiv:1812.08466, 2018

    arXiv 2018

  8. [8]

    Consistency models,

    Y . Song, P. Dhariwal, M. Chen, and I. Sutskever, “Consistency models,”

Show all 29 references
  1. [9]

    Autoregressive image generation without vector quantization,

    T. Li, Y . Tian, H. Li, M. Deng, and K. He, “Autoregressive image generation without vector quantization,” 2024. [Online]. Available: https://arxiv.org/abs/2406.11838

    2024 arXiv

  2. [10]

    Drcap: Decoding clap latents with retrieval-augmented generation for zero-shot audio captioning,

    X. Li, W. Chen, Z. Ma, X. Xu, Y . Liang, Z. Zheng, Q. Kong, and X. Chen, “Drcap: Decoding clap latents with retrieval-augmented generation for zero-shot audio captioning,” inICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEE...

    2025

  3. [11]

    Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,

    Y . Wu, K. Chen, T. Zhang, Y . Hui, T. Berg-Kirkpatrick, and S. Dubnov, “Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,” inICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (IC...

    2023

  4. [12]

    T-clap: Temporal-enhanced contrastive language-audio pretraining,

    Y . Yuan, Z. Chen, X. Liu, H. Liu, X. Xu, D. Jia, Y . Chen, M. D. Plumbley, and W. Wang, “T-clap: Temporal-enhanced contrastive language-audio pretraining,” in2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP). IEEE, 2024, pp. 1–6

    2024

  5. [13]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,”IEEE/ACM transactions on audio, speech, and language processing, vol. 29, pp. 3451–3460, 2021

    2021

  6. [14]

    Beats: Audio pre-training with acoustic tokenizers,

    S. Chen, Y . Wu, C. Wang, S. Liu, D. Tompkins, Z. Chen, and F. Wei, “Beats: Audio pre-training with acoustic tokenizers,”arXiv preprint arXiv:2212.09058, 2022

    2022 arXiv

  7. [15]

    Mert: Acoustic music under- standing model with large-scale self-supervised training,

    Y . Li, R. Yuan, G. Zhang, Y . Ma, X. Chen, H. Yin, C. Xiao, C. Lin, A. Ragni, E. Benetoset al., “Mert: Acoustic music under- standing model with large-scale self-supervised training,”arXiv preprint arXiv:2306.00107, 2023

    2023 arXiv

  8. [16]

    High fidelity neural audio compression,

    A. D ´efossez, J. Copet, G. Synnaeve, and Y . Adi, “High fidelity neural audio compression,” 2022. [Online]. Available: https://arxiv.org/abs/2210. 13438

    2022

  9. [17]

    Improved techniques for training consistency models,

    Y . Song and P. Dhariwal, “Improved techniques for training consistency models,” 2023. [Online]. Available: https://arxiv.org/abs/2310.14189

    2023 arXiv

  10. [18]

    Design and evaluation of parallel quadrature mirror filters (pqmf),

    C. Galand and D. Esteban, “Design and evaluation of parallel quadrature mirror filters (pqmf),” inICASSP’83. IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 8. IEEE, 1983, pp. 224–227

    1983

  11. [19]

    Audio set: An ontology and human- labeled dataset for audio events,

    J. F. Gemmeke, D. P. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter, “Audio set: An ontology and human- labeled dataset for audio events,” in2017 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2017, p...

    2017

  12. [20]

    The musdb18 corpus for music separation,

    Z. Rafii, A. Liutkus, F.-R. St ¨oter, S. I. Mimilakis, and R. Bittner, “The musdb18 corpus for music separation,” 2017. [Online]. Available: https://doi.org/10.5281/zenodo.1117372

    2017 doi

  13. [21]

    Cutting music source separation some slakh: A dataset to study the impact of training data quality and quantity,

    E. Manilow, G. Wichern, P. Seetharaman, and J. Le Roux, “Cutting music source separation some slakh: A dataset to study the impact of training data quality and quantity,” in2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA). IEEE, 2019, pp. 45–49

    2019

  14. [22]

    Icassp 2023 deep noise suppression challenge,

    H. Dubey, A. Aazami, V . Gopal, B. Naderi, S. Braun, R. Cutler, H. Gamper, M. Golestaneh, and R. Aichner, “Icassp 2023 deep noise suppression challenge,” inICASSP, 2023

    2023

  15. [23]

    Audiocaps: Generating captions for audios in the wild,

    C. D. Kim, B. Kim, H. Lee, and G. Kim, “Audiocaps: Generating captions for audios in the wild,” inNAACL-HLT, 2019

    2019

  16. [24]

    Wavcaps: A chatgpt-assisted weakly-labelled audio captioning dataset for audio-language multimodal research,

    X. Mei, C. Meng, H. Liu, Q. Kong, T. Ko, C. Zhao, M. D. Plumbley, Y . Zou, and W. Wang, “Wavcaps: A chatgpt-assisted weakly-labelled audio captioning dataset for audio-language multimodal research,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024

    2024

  17. [25]

    Clotho: An audio captioning dataset,

    K. Drossos, S. Lipping, and T. Virtanen, “Clotho: An audio captioning dataset,” 2019. [Online]. Available: https://arxiv.org/abs/1910.09387

    2019 arXiv

  18. [26]

    Fsd50k: an open dataset of human-labeled sound events,

    E. Fonseca, X. Favory, J. Pons, F. Font, and X. Serra, “Fsd50k: an open dataset of human-labeled sound events,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 30, pp. 829–852, 2021

    2021

  19. [27]

    Auto-acd: A large-scale dataset for audio-language representation learning,

    L. Sun, X. Xu, M. Wu, and W. Xie, “Auto-acd: A large-scale dataset for audio-language representation learning,” inProceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 5025–5034

    2024

  20. [28]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  21. [2023]

    Available: https://arxiv.org/abs/2303.01469

    [Online]. Available: https://arxiv.org/abs/2303.01469

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.