REVIEW 12 cited by
Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Contrastive learning has shown remarkable success in the field of multimodal representation learning. In this paper, we propose a pipeline of contrastive language-audio pretraining to develop an audio representation by combining audio data with natural language descriptions. To accomplish this target, we first release LAION-Audio-630K, a large collection of 633,526 audio-text pairs from different data sources. Second, we construct a contrastive language-audio pretraining model by considering different audio encoders and text encoders. We incorporate the feature fusion mechanism and keyword-to-caption augmentation into the model design to further enable the model to process audio inputs of variable lengths and enhance the performance. Third, we perform comprehensive experiments to evaluate our model across three tasks: text-to-audio retrieval, zero-shot audio classification, and supervised audio classification. The results demonstrate that our model achieves superior performance in text-to-audio retrieval task. In audio classification tasks, the model achieves state-of-the-art performance in the zero-shot setting and is able to obtain performance comparable to models' results in the non-zero-shot setting. LAION-Audio-630K and the proposed model are both available to the public.
Forward citations
Cited by 12 Pith papers
-
InvFlowFD: Reference-Free and Background-Set-Free Perceptual Music Quality Metric with Flow Matching Inversion
InvFlowFD measures music quality by inverting audio through a flow matching model and computing the distance of the inverted latents to the model's Gaussian prior.
-
Fusion Embedding: A Unified Embedding Space for Text, Image, Video, and Audio
Trained connectors and audio-only gated adapters integrate audio into a frozen vision-language embedding space, preserving base outputs bit-exactly and yielding emergent audio-image retrieval.
-
Discriminative Axis, Not Data Volume: What a Contrastive Corpus Teaches an Audio Embedding
A contrastive audio embedding learns an attribute only when in-batch negatives cannot be separated without it, so corpus structure, not size or caption vocabulary, controls what is encoded.
-
MADB: A Large-Scale Music Aesthetics Dataset with Professional and Multi-Dimensional Annotations
MADB is a 9,999-track music aesthetics benchmark with multi-dimensional professional annotations revealing that current pretrained audio models capture only partial aesthetic information.
-
Video2Reaction: Mapping Video to Audience Reaction Distribution in the Wild
A new dataset and benchmark maps movie clips to distributions of audience emotional reactions derived from YouTube comments, showing that finetuned vision-language models can predict these distributions from video alone.
-
Auto-AEG: Scalable Data Construction for Open-Vocabulary Audio Event Grounding
Automatically constructed synthetic exact-GT data plus multi-model pseudo-labels with interval-aware GRPO rewards improve LALM open-vocabulary audio event grounding on AEGBench and DESED.
-
SemanticAudio: Audio Generation and Editing in Semantic Space
SemanticAudio improves text-to-audio alignment by generating a compact semantic plan first with a Flow Matching planner and then rendering acoustic latents from that plan, and it performs training-free audio editing b...
-
LLaSO: A Foundational Framework for Reproducible Research in Large Language and Speech Model
LLaSO releases a 3.8B speech-language model, 25.5M training instances, and an evaluation benchmark, claiming a normalized score of 0.72.
-
Summary of DCASE 2026 Task 5: Audio-Dependent Question Answering
On a new 3000-item audio-only benchmark stripped of text-solvable questions, the best AI system scores 58.33%, random guessing scores 25.46%, and all 36 competing systems fail the same 233 items.
-
The Name-Free Gap: Policy-Aware Stylistic Control in Music Generation
Word-based style descriptors generated by an LLM can shift MusicGen outputs toward a target artist's sound almost as much as using the artist's name, defining a name-free gap.
-
Learning Normal Patterns in Musical Loops
A Deep SVDD model using HTS-AT and feature fusion learns normal patterns in variable-length bass and guitar loops, with residual connections improving the learned latent space.
-
Multiple Consistency-guided Test-Time Adaptation for Contrastive Audio-Language Models with Unlabeled Audio
A consistency-guided test-time prompt adaptation method improves CLAP zero-shot audio classification by 4.41% relative on average over DA CLAP across 12 datasets.
Discussion (0). Continue with ORCID to comment.