REVIEW 6 cited by
InstructME: An Instruction Guided Music Edit And Remix Framework with Latent Diffusion Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Music editing primarily entails the modification of instrument tracks or remixing in the whole, which offers a novel reinterpretation of the original piece through a series of operations. These music processing methods hold immense potential across various applications but demand substantial expertise. Prior methodologies, although effective for image and audio modifications, falter when directly applied to music. This is attributed to music's distinctive data nature, where such methods can inadvertently compromise the intrinsic harmony and coherence of music. In this paper, we develop InstructME, an Instruction guided Music Editing and remixing framework based on latent diffusion models. Our framework fortifies the U-Net with multi-scale aggregation in order to maintain consistency before and after editing. In addition, we introduce chord progression matrix as condition information and incorporate it in the semantic space to improve melodic harmony while editing. For accommodating extended musical pieces, InstructME employs a chunk transformer, enabling it to discern long-term temporal dependencies within music sequences. We tested InstructME in instrument-editing, remixing, and multi-round editing. Both subjective and objective evaluations indicate that our proposed method significantly surpasses preceding systems in music quality, text relevance and harmony. Demo samples are available at https://musicedit.github.io/
Forward citations
Cited by 6 Pith papers
-
EditGen: Harnessing Cross-Attention Control for Instruction-Based Auto-Regressive Audio Editing
Prompt-to-Prompt cross-attention control is adapted to autoregressive audio generation, enabling training-free music editing that outperforms a diffusion baseline.
-
Workflow-Based Evaluation of Music Generation Systems
A single-producer workflow evaluation of eight music AI tools finds they work as idea and sound generators but not as complete composers, and proposes a reusable framework.
-
FlowSonic: Stable Zero-Shot Music Editing via High-Order Trajectory Integration
FlowSonic combines deterministic rectified-flow inversion, cached cross-attention injection, and a 'seeded' third-order Adams-Bashforth solver to report better timbre and genre edits on small datasets.
-
MuMu-LLaMA: Multi-modal Music Understanding and Generation via Large Language Models
MuMu-LLaMA uses LLaMA with pretrained encoders and music decoders to understand and generate music from text, images, and videos, trained on a 167.69 hour machine-annotated dataset.
-
Instruction-Guided Editing Controls for Images and Multimedia: A Survey in LLM era
A survey that organizes over 100 instruction-guided image and multimedia editing papers into a process-based taxonomy, with an emphasis on LLM and MLLM empowered methods.
-
Improving Controllability and Editability for Pretrained Text-to-Music Generation Models
A thesis compilation presenting three complementary approaches to improving editing and control of pretrained text-to-music models, with Instruct-MusicGen demonstrating the strongest stem-level editing results.
Discussion (0). Continue with ORCID to comment.