REVIEW 18 cited by
Mo\^usai: Text-to-Music Generation with Long-Context Latent Diffusion
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recent years have seen the rapid development of large generative models for text; however, much less research has explored the connection between text and another "language" of communication -- music. Music, much like text, can convey emotions, stories, and ideas, and has its own unique structure and syntax. In our work, we bridge text and music via a text-to-music generation model that is highly efficient, expressive, and can handle long-term structure. Specifically, we develop Mo\^usai, a cascading two-stage latent diffusion model that can generate multiple minutes of high-quality stereo music at 48kHz from textual descriptions. Moreover, our model features high efficiency, which enables real-time inference on a single consumer GPU with a reasonable speed. Through experiments and property analyses, we show our model's competence over a variety of criteria compared with existing music generation models. Lastly, to promote the open-source culture, we provide a collection of open-source libraries with the hope of facilitating future work in the field. We open-source the following: Codes: https://github.com/archinetai/audio-diffusion-pytorch; music samples for this paper: http://bit.ly/44ozWDH; all music samples for all models: https://bit.ly/audio-diffusion.
Forward citations
Cited by 18 Pith papers
-
Diff-Symbo: Text-Controlled Long-Duration Symbolic Music Generation Using Autoregressive Latent Diffusion Model
Diff-Symbo generates long, text-controlled symbolic music by autoregressively extending 8-bar latent diffusion segments conditioned on the previous segment's latent.
-
Dance to Music Generation leveraging Pre-training with Unpaired data and Contrastive Alignment
Beat-guided contrastive alignment of pretrained MotionBERT/MERT features plus ControlNet conditioning of AudioLDM improves dance–music alignment on AIST++ over a MusicGen textual-inversion baseline while remaining com...
-
Next Tokens Denoising for Speech Synthesis
Dragon-FM generates speech autoregressively over two-second chunks while using flow matching inside each chunk, achieving fast synthesis at 12.5 discrete audio tokens per second.
-
MusGO: A Community-Driven Framework For Assessing Openness in Music-Generative AI
MusGO is a community-refined framework with 13 openness categories, applied to 16 music-generative models to produce a public openness leaderboard.
-
Let Your Video Listen to Your Music!
MVAA aligns a video's motion peaks to music beats via keyframe re-timing and diffusion-based inpainting, aiming to preserve the original content while improving rhythmic synchronization.
-
Video-Guided Text-to-Music Generation Using Public Domain Movie Collections
OSSL is the first self-hosted, mood-annotated video-music dataset, and a video adapter on MusicGen-Medium improves film music generation over text-only baselines.
-
A Mixture-Based Framework for Guiding Diffusion Models
MGDM approximates the intractable guided-diffusion posterior with a weighted mixture of likelihood approximations and samples the mixture using a Gibbs sampler with tunable repetitions.
-
XMusic: Towards a Generalized and Controllable Symbolic Music Generation Framework
A multi-modal symbolic music generation framework that parses images, videos, text, tags, and humming into emotion, genre, rhythm, and note controls, then generates and quality-filters MIDI music.
-
FolAI: Synchronized Foley Sound Generation with Semantic and Temporal Alignment
FolAI predicts an editable RMS envelope from silent video and uses it, with semantic embeddings, to condition a Stable Audio diffusion model for 44.1 kHz stereo foley generation.
-
DiffSLT: Enhancing Diversity in Sign Language Translation via Diffusion Model
DiffSLT uses a latent diffusion model conditioned on fused multi-level visual features to produce diverse, accurate sign language translations, and DiffSLT-P conditions on pseudo-glosses to improve accuracy further.
-
CoDiCodec: Unifying Continuous and Discrete Compressed Representations of Audio
CoDiCodec unifies continuous and discrete audio compression in one consistency-trained autoencoder, using FSQ-dropout to serve both continuous ~11 Hz embeddings and 2.38 kbps discrete tokens.
-
Workflow-Based Evaluation of Music Generation Systems
A single-producer workflow evaluation of eight music AI tools finds they work as idea and sound generators but not as complete composers, and proposes a reusable framework.
-
Amuse: Human-AI Collaborative Songwriting with Multimodal Inspirations
A human-AI songwriting tool that generates keyword-relevant chord progressions from images, text, or audio by filtering LLM suggestions with a chord model, shown to increase perceived agency and creativity in a small ...
-
VidMusician: Video-to-Music Generation with Semantic-Rhythmic Alignment via Hierarchical Visual Features
An adaptation of MusicGen that adds semantic conditioning from CLIP global features and rhythmic conditioning from CLIP local inter-frame similarity, trained in two stages, generates video-aligned background music.
-
FlowSonic: Stable Zero-Shot Music Editing via High-Order Trajectory Integration
FlowSonic combines deterministic rectified-flow inversion, cached cross-attention injection, and a 'seeded' third-order Adams-Bashforth solver to report better timbre and genre edits on small datasets.
-
Interpreting Graphic Notation with MusicLDM: An AI Improvisation of Cornelius Cardew's Treatise
A ChatGPT-to-MusicLDM pipeline interprets pages of Cardew's Treatise as text prompts and synthesizes continuous improvised audio using latent-overlap outpainting.
-
Continuous Speech Tokens Makes LLMs Robust Multi-Modality Learners
A speech-to-speech dialogue model that predicts mel-spectrograms with flow matching, jointly trained with discrete text tokens, achieves lower WER than a discrete speech token baseline.
-
ASAudio: A Survey of Advanced Spatial Audio Research
A comprehensive survey that systematically categorizes spatial audio research by representation, task, dataset, and evaluation.
Discussion (0). Continue with ORCID to comment.