MAVIN is a dual-tower diffusion framework that produces temporally aligned multi-shot audio-visual content from hierarchical captions with optional multi-identity image and audio references.
SAM Audio: Segment Anything in Audio,
11 Pith papers cite this work. Polarity classification is still indexing.
11
Pith papers citing it
years
2026 11representative citing papers
JAVEdit-100k is the first large-scale dataset for instruction-guided joint audio-visual video editing, accompanied by JAVEditBench and the JAVEdit model that outperforms baselines on five of six metrics.
InstructAV2AV is an end-to-end instruction-guided audio-video joint editing model that adapts a pre-trained backbone with gated attention and two-stage training, outperforming prior methods on 11 metrics after building the InsAVE-80K dataset.
ELSA introduces an event-level semantic alignment metric for reference-free text-to-audio evaluation that reports higher correlation with human ratings than CLAP-based baselines across four benchmarks.
In moderate-sized fine-grained bioacoustics, pretraining scale of masked autoencoders on diverse general audio dominates over domain-specific objectives or data curation for transfer performance.
Audio-Omni unifies audio understanding, generation, and editing in one end-to-end model across domains, backed by a new million-pair AudioEdit dataset, and achieves strong benchmark results.
ALM2Vec learns unified audio embeddings from large audio-language models for text-audio retrieval, instruction-aware retrieval, and other tasks across domains.
Causal probing of attention in audio separation transformers identifies dual pathways and asynchronous convergence, enabling a training-free Layer-Selective Attention Caching method that reduces self-attention computation by ~25% with negligible quality loss.
A shared continuous-latent flow model generates music from text/vision or extracts a target source from a mixture via visual-audio alignment, gated modulation, and dynamic modality masking.
A VAE-plus-diffusion model jointly restores and separates degraded monaural two-speaker dialogue into clean full-duplex speaker tracks with better intelligibility and faster inference than a baseline.
Integrates SAM-Audio dense representations with guided attention and dual distillation for audio-visual class-incremental learning, reporting consistent outperformance on benchmarks.
citing papers explorer
-
MAVIN: Multi-Shot Audio-Visual Generation with Customized Narrative Control
MAVIN is a dual-tower diffusion framework that produces temporally aligned multi-shot audio-visual content from hierarchical captions with optional multi-identity image and audio references.
-
JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curation
JAVEdit-100k is the first large-scale dataset for instruction-guided joint audio-visual video editing, accompanied by JAVEditBench and the JAVEdit model that outperforms baselines on five of six metrics.
-
InstructAV2AV: Instruction-Guided Audio-Video Joint Editing
InstructAV2AV is an end-to-end instruction-guided audio-video joint editing model that adapts a pre-trained backbone with gated attention and two-stage training, outperforming prior methods on 11 metrics after building the InsAVE-80K dataset.
-
ELSA: Acoustic Event-Level Semantic Alignment for Fine-Grained Reference-Free Text-to-Audio Evaluation
ELSA introduces an event-level semantic alignment metric for reference-free text-to-audio evaluation that reports higher correlation with human ratings than CLAP-based baselines across four benchmarks.
-
Masked Autoencoders with Limited Data: Does It Work? A Fine-Grained Bioacoustics Case Study
In moderate-sized fine-grained bioacoustics, pretraining scale of masked autoencoders on diverse general audio dominates over domain-specific objectives or data curation for transfer performance.
-
Audio-Omni: Extending Multi-modal Understanding to Versatile Audio Generation and Editing
Audio-Omni unifies audio understanding, generation, and editing in one end-to-end model across domains, backed by a new million-pair AudioEdit dataset, and achieves strong benchmark results.
-
ALM2Vec: Learning Audio Embeddings for Universal Audio Retrieval with Large Audio-Language Models
ALM2Vec learns unified audio embeddings from large audio-language models for text-audio retrieval, instruction-aware retrieval, and other tasks across domains.
-
Inside the Latent Flow: Causal Deciphering of Attention Dynamics in Audio Separation Foundation Models
Causal probing of attention in audio separation transformers identifies dual pathways and asynchronous convergence, enabling a training-free Layer-Selective Attention Caching method that reduces self-attention computation by ~25% with negligible quality loss.
-
MAGE: Modality-Agnostic Music Generation and Target-Source Extraction
A shared continuous-latent flow model generates music from text/vision or extracts a target source from a mixture via visual-audio alignment, gated modulation, and dynamic modality masking.
-
DialogueSidon: Recovering Full-Duplex Dialogue Tracks from In-the-Wild Dialogue Audio
A VAE-plus-diffusion model jointly restores and separates degraded monaural two-speaker dialogue into clean full-duplex speaker tracks with better intelligibility and faster inference than a baseline.
-
Listen, Look, and Learn: Learning Without Forgetting through SAM-Audio
Integrates SAM-Audio dense representations with guided attention and dual distillation for audio-visual class-incremental learning, reporting consistent outperformance on benchmarks.