REVIEW 6 cited by
Joint Audio and Symbolic Conditioning for Temporally Controlled Text-to-Music Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We present JASCO, a temporally controlled text-to-music generation model utilizing both symbolic and audio-based conditions. JASCO can generate high-quality music samples conditioned on global text descriptions along with fine-grained local controls. JASCO is based on the Flow Matching modeling paradigm together with a novel conditioning method. This allows music generation controlled both locally (e.g., chords) and globally (text description). Specifically, we apply information bottleneck layers in conjunction with temporal blurring to extract relevant information with respect to specific controls. This allows the incorporation of both symbolic and audio-based conditions in the same text-to-music model. We experiment with various symbolic control signals (e.g., chords, melody), as well as with audio representations (e.g., separated drum tracks, full-mix). We evaluate JASCO considering both generation quality and condition adherence, using both objective metrics and human studies. Results suggest that JASCO is comparable to the evaluated baselines considering generation quality while allowing significantly better and more versatile controls over the generated music. Samples are available on our demo page https://pages.cs.huji.ac.il/adiyoss-lab/JASCO.
Forward citations
Cited by 6 Pith papers
-
DiffRhythm+: Controllable and Flexible Full-Length Song Generation with Preference Optimization
DiffRhythm+ improves full-length lyric-to-song generation via balanced data scaling, MuLan-based multimodal style control, and DPO fine-tuning guided by automated aesthetic scorers.
-
MuseControlLite: Multifunctional Music Generation with Lightweight Conditioners
A lightweight adapter that adds rotary position embeddings to decoupled cross-attention enables efficient time-varying style control and audio inpainting/outpainting for text-to-music diffusion Transformers.
-
Video-Guided Text-to-Music Generation Using Public Domain Movie Collections
OSSL is the first self-hosted, mood-annotated video-music dataset, and a video adapter on MusicGen-Medium improves film music generation over text-only baselines.
-
Auto-Regressive vs Flow-Matching: a Comparative Study of Modeling Paradigms for Text-to-Music Generation
Under matched training conditions, auto-regressive models slightly outperform flow-matching on music quality and temporal control, while flow-matching offers faster inference and better inpainting flexibility.
-
Exploring Adapter Design Tradeoffs for Low Resource Music Generation
Adapter placement, architecture, and size strongly change generation quality and cost for MusicGen and Mustango on two non-Western genres, with late-layer, mid-sized (40M) adapters reported as the best tradeoff.
-
Genre Controlled Music Generation via Activation Steering
Activation steering with linear probe weights on MusicGen's residual stream shifts generated music between genres at inference time, outperforming text prompting in CLAP and listener preference but with incomplete reporting.
Discussion (0). Sign in to comment.