Pith. sign in

REVIEW 5 cited by

SingSong: Generating musical accompaniments from singing

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2301.12662 v1 pith:L7R4A3RJ submitted 2023-01-30 cs.SD cs.AIcs.LGcs.MMeess.AS

SingSong: Generating musical accompaniments from singing

Chris Donahue , Antoine Caillon , Adam Roberts , Ethan Manilow , Philippe Esling , Andrea Agostinelli , Mauro Verzetti , Ian Simon
show 3 more authors
Olivier Pietquin Neil Zeghidour Jesse Engel
This is my paper
classification cs.SD cs.AIcs.LGcs.MMeess.AS
keywords singsongaudiogenerationinstrumentalmusicmusicalpairsseparation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

We present SingSong, a system that generates instrumental music to accompany input vocals, potentially offering musicians and non-musicians alike an intuitive new way to create music featuring their own voice. To accomplish this, we build on recent developments in musical source separation and audio generation. Specifically, we apply a state-of-the-art source separation algorithm to a large corpus of music audio to produce aligned pairs of vocals and instrumental sources. Then, we adapt AudioLM (Borsos et al., 2022) -- a state-of-the-art approach for unconditional audio generation -- to be suitable for conditional "audio-to-audio" generation tasks, and train it on the source-separated (vocal, instrumental) pairs. In a pairwise comparison with the same vocal inputs, listeners expressed a significant preference for instrumentals generated by SingSong compared to those from a strong retrieval baseline. Sound examples at https://g.co/magenta/singsong

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. MIDI-Informed Singing Accompaniment Generation in a Compositional Song Pipeline

    cs.SD 2026-02 unverdicted novelty 7.0

    MIDI-SAG generates consistent long-form singing accompaniments by feeding symbolic MIDI timing, chords, and structure labels into a compositional pipeline built from pre-trained modules.

  2. Towards Real-Time Human-AI Musical Co-Performance: Accompaniment Generation with Latent Diffusion Models and MAX/MSP

    cs.SD 2026-04 unverdicted novelty 6.0

    A latent diffusion model with consistency distillation generates real-time instrumental accompaniment from live context audio, integrated with MAX/MSP for feasible human-AI co-performance.

  3. AudioPaLM: A Large Language Model That Can Speak and Listen

    cs.CL 2023-06 unverdicted novelty 6.0

    AudioPaLM unifies PaLM-2 and AudioLM to outperform prior systems on speech translation while enabling zero-shot speech-to-text for many unseen language pairs and voice transfer from short prompts.

  4. HAFM: Hierarchical Autoregressive Foundation Model for Music Accompaniment Generation

    cs.SD 2026-04 unverdicted novelty 5.0

    HAFM uses a hierarchical autoregressive model with dual-rate HuBERT and EnCodec tokens to generate coherent instrumental music from vocals, achieving FAD 2.08 on MUSDB18 while matching prior systems with fewer parameters.

  5. HAFM: Hierarchical Autoregressive Foundation Model for Music Accompaniment Generation

    cs.SD 2026-04 unverdicted novelty 5.0

    A three-stage hierarchical AR model with dual-rate HuBERT/EnCodec tokens improves vocal-conditioned accompaniment generation, reaching FAD 1.71 and 51.5% preference vs ground truth on MUSDB18.