Pith. sign in

REVIEW 4 major objections 4 minor 32 references

Full-song generation reaches commercial-grade quality by separating hierarchical token planning from flow-matching rendering.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 10:20 UTC pith:UYSTCQXK

load-bearing objection A credible, well-engineered industrial system report whose strongest independent evidence (the leaderboard) is unverifiable from the paper, and whose 15-of-18 automatic benchmark lead is undermined by SongBench's dual role as training reward and evaluation metric. the 4 major comments →

arxiv 2607.20253 v3 pith:UYSTCQXK submitted 2026-07-22 cs.SD cs.AIeess.AS

Pushing the Frontier of Full-Song Generation: Hierarchical Autoregressive Planning Meets Flow-Matching Rendering

Junyu Dai , Xinyue Fan , Weiqin Li , Xiangang Li , Yunjia Li , Bin Ma , Yukun Ma , Chongjia Ni
show 9 more authors
Yufei Shi Biao Tian Haoxu Wang Menglin Wu Jianwei Yu Huaicheng Zhang Han Zhao Shengkui Zhao Haina Zhu
This is my paper
classification cs.SD cs.AIeess.AS
keywords full-song generationhierarchical autoregressive modelflow-matching renderingRVQ tokenizationcover song generationmelody modelingpreference optimizationmusic generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that producing a full-length song with vocals is best done in two stages: first, a hierarchical language model autoregressively writes a compact discrete plan of the song as eight-stream audio tokens; second, a large flow-matching transformer renders that plan into a high-fidelity continuous audio latent, conditioned also on lyrics and captions. This division of labor, the paper claims, closes the quality gap to the leading commercial vocal-music systems: its anonymous leaderboard submission lands statistically in the same tier as the second-ranked model, and it leads 15 of 18 dimensions of the authors' multilingual automatic benchmark. The same framework also supports instrumental music and cover-song generation, the latter guided by a two-level melody tokenization (coarse MIDI notes plus fine F0). The paper additionally shows that reward-based post-training (DPO, GRPO, and on-policy distillation) improves musicality and rendering quality. A sympathetic reader would care because the result suggests a practical recipe for building a single controllable, high-quality song generator rather than a niche model.

Core claim

The central claim is that the hybrid architecture — hierarchical autoregressive token planning in hybrid-LM followed by non-causal full-song flow matching in FullDiT — achieves full-song vocal generation quality that is statistically tied with the second-ranked system on an external blind-preference leaderboard, and leads most dimensions of the authors' own 500-example multilingual evaluation. The paper also claims the two-level melody module (MIDI-note tokens for contour, F0 tokens for fine pitch) preserves melody in cover generation, and that its combination of DPO, GRPO, and on-policy distillation, plus flow-based GRPO for the renderer, lifts musicality and audio fidelity. These are the c

What carries the argument

The load-bearing component is the separation of planning from rendering. hybrid-LM factorizes the joint distribution of an 8-codebook RVQ token sequence as a product of a global 8B LLM predicting the first codebook token per 25 Hz frame and a local 0.4B LLM filling the seven residual codebooks within each frame. FullDiT then performs non-causal flow matching in a continuous VAE latent space, conditioned on the entire codec sequence, lyrics, and caption, with Error-Matched Distractor Conditioning to tolerate upstream token errors. The two-level melody module discretizes vocal melody into MIDI-note tokens and F0 tokens that guide cover generation.

Load-bearing premise

The paper's independent external evidence depends on the anonymous leaderboard entry being exactly the system described here; no public mapping or artifact is provided to verify that identity, so if the entry is any other model, the external quality-tier claim has no basis.

What would settle it

Look up the leaderboard's public entry list: if the anonymous submission has no downloadable samples or model card matching this system, the external evidence is unsupported. More directly, run a preregistered blind listening test pitting this system against the second-ranked commercial model on a fresh set of prompts; a clear majority preference for the competitor would contradict the claimed statistical tie.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Full-song quality can be achieved by separating long-range planning from acoustic rendering, so neither model alone needs to solve both tasks.
  • A single framework can cover lyrics-to-song, instrumental, and cover generation by sharing the tokenizer and renderer, with melody added as explicit coarse and fine tokens.
  • Reward-based post-training using automatic musicality scores can shift a pretrained generator toward more musical output without large-scale human preference labels.
  • Full-song flow matching over thousands of frames suggests long-range structure can be rendered in one pass, avoiding the coherence limits of chunk-wise synthesis.
  • Error-Matched Distractor Conditioning makes the renderer robust to imperfect upstream tokens, which is essential when the planner is a separate model.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The external leaderboard evidence hinges on the unverified identity of the anonymous submission; with no public artifacts linking that entry to this system, the independent confirmation is currently a claim, not a demonstrated fact.
  • The automatic benchmark uses the authors' chosen evaluator models and prompts; a fully independent listening study would be needed to confirm the quality tier on fresh material.
  • The plan-then-render recipe is a general template for long-form conditional generation: a coarse discrete planner plus a full-sequence continuous renderer could transfer to other long-form audio or video domains.
  • Dedicated quantitative results for instrumental and cover generation are deferred, so those two supported tasks are presently established only by design and examples, not by evaluation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper describes a unified full-song generation framework that combines an 8-codebook semantic-aware RVQ tokenizer, a hierarchical autoregressive language model ('hybrid-LM') with an 8B global and 0.4B local component, a full-song flow-matching DiT renderer ('FullDiT'), and a two-level melody module for cover-song generation. The system is trained and post-trained with SFT, DPO, GRPO, and OPD, including a flow-based GRPO stage for FullDiT. The authors claim competitive performance: on their 500-example multilingual automatic benchmark they report the highest point estimate in 15 of 18 dimensions, and on the Artificial Analysis Music with Vocals leaderboard their anonymously submitted system, 'Lucky Dolphin', is reported to have Elo 1129 with an official rank range of 2–3, statistically tied with the second-ranked system.

Significance. If the claims are upheld, this is a substantial systems contribution: it integrates discrete token planning, continuous flow-matching rendering, melody conditioning, and preference-based post-training in one full-song pipeline, and the controlled 1.5B FullDiT ablations in Table 2 are a good template for isolating design choices. The use of an external blind-preference leaderboard, if the submission identity is verifiable, provides independent evidence that the system is in the top tier of current vocal-music systems. The paper is also honest about the scope limitation for instrumental and cover-song evaluation. However, the main automatic-benchmark evidence is weakened by a metric-circularity problem: SongBench is used both as a training reward and as the primary evaluation metric, and the reported wins are point estimates without uncertainty quantification.

major comments (4)
  1. [§2.3.1, §2.6.2, §2.6.3, §3.2 (Table 3)] SongBench is used in the training loop: it is a curation filter in §2.3.1, the label source for DPO pairs in §2.6.2, and the reward model in GRPO in §2.6.3. The same SongBench dimensions then appear as the first seven rows of Table 3. Reporting that the model 'leads' on these dimensions is therefore partly a statement about optimization alignment, not independent song quality. The paper should either report the non-circular metrics (SongEval, AudioBox-Aesthetic, CMI-Reward) as the primary evidence, provide a decontamination analysis, or add a human listening study on held-out prompts. At minimum, the overlap must be disclosed in the evaluation section.
  2. [§3.2, Table 3] All comparisons in Table 3 are point estimates with no confidence intervals or significance tests. Several claimed wins are tiny, e.g., Vocal 7.6234 vs Mureka V8 7.6248 and Production Quality 8.2986 vs 8.2923; the 'highest mean in 15/18 dimensions' claim substantially overstates what point estimates alone establish. The authors should provide bootstrap confidence intervals, paired significance tests, or per-prompt score distributions, especially for the non-SongBench evaluators.
  3. [Figure 1, §1] The external validation rests on the anonymous leaderboard entry 'Lucky Dolphin' being exactly the proposed system. The paper provides no public artifact, commit, or release mapping the leaderboard submission to the architecture described here, so a reader cannot verify the identity of the submission. If the mapping cannot be made public, the claim should be softened to 'an anonymous system that we believe corresponds to ours' and the uncertainty about identity acknowledged as a limitation of the external evidence.
  4. [§1, §3, §4] The framework is presented as supporting three tasks — lyrics-to-song, instrumental, and cover-song generation — but the quantitative evaluation covers only lyrics-to-song vocal generation. The paper itself states that 'dedicated quantitative evaluation of instrumental and cover song generation remains future work.' For a paper whose central contribution includes these tasks, this is a scope mismatch. Either include basic automatic or listening results for instrumental and cover generation, or explicitly reframe the title and claims around lyrics-to-song generation.
minor comments (4)
  1. [Abstract] Typo: 'hybird-LM' should be 'hybrid-LM'.
  2. [Figure 1 / §1] The figure caption gives Elo 1129, while the text description of the leaderboard gap uses confidence intervals. It would help to state the sample size and the date of the snapshot in the caption, since leaderboards change.
  3. [Table 3] The note that Production Complexity is 'descriptive and not inherently better when larger' is useful, but the text in §3.2 still interprets higher Production Complexity as 'richer component layering.' Please clarify whether this is a quality advantage or merely a descriptive property.
  4. [§3.1, Table 2] The wording 'M1 is preferred to M2 under LM-generated codec' is supported by the song-level preference row, but the sample size is only N=100 with five raters; please report inter-rater agreement or per-rater variance.

Circularity Check

1 steps flagged

The 15-of-18 auto-eval advantage is partially circular: SongBench is used as a training reward and then as the leading evaluation metric.

specific steps
  1. fitted input called prediction [Sections 2.3.1, 2.6.2, 2.6.3, 3.2; Table 3]
    "The candidates are evaluated using SongBench, which assigns an automatic score s(yi) to each generated sample according to its overall musical quality. Candidate pairs are then constructed according to the difference between the scores. … We therefore use the SongBench Musicality score to select a higher-quality subset … Each candidate is rendered into audio and automatically evaluated by SongBench for overall musical quality. … we report complementary automatic evaluations using SongBench [26], SongEval [31], AudioBox-Aesthetic [22], and CMI-Reward [20]."

    SongBench is the reward model used to build DPO preference pairs (Eq. 9), to compute GRPO rewards and advantages (Eqs. 13/15), and to select high-quality training data in Stage 2. The same SongBench metric is then reported as the first seven evaluation dimensions in Table 3 and contributes to the headline '15 of 18' lead. Since hybrid-LM was explicitly optimized to increase SongBench scores, the SongBench portion of the evaluation is not an independent confirmation of song quality; it reflects alignment to the training objective. The non-SongBench evaluators in Table 3 were also used in corpus curation, though not as direct RL rewards, weakening but not eliminating their independence.

full rationale

The paper's strongest independent support is the external Artificial Analysis Music with Vocals leaderboard, which is blind and outside the authors' control; however, its relevance depends on the unverified identity of the anonymous 'Lucky Dolphin' submission. The automatic-benchmark claim in Table 3 is partially circular because SongBench is both a training-time reward and the leading evaluation metric, and the SongBench reference includes one of the present paper's authors (Huaicheng Zhang). This makes some of the 15-of-18 wins at least partly a measure of how well the model optimized that specific reward rather than a neutral test of song quality. The reported gaps are also point estimates without confidence intervals. At the same time, not all evidence is circular: SongEval, AudioBox-Aesthetic, and CMI-Reward are separate evaluators, and the external leaderboard, if properly authenticated, would ground the tier 2-3 claim independently. The circularity is therefore partial, not total.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

No new physical or conceptual entities are introduced; the system combines existing methods. The ledger instead records the hand-set weights, data-derived replacement probabilities, evaluation assumptions, and the unverifiable leaderboard identity that the central claim depends on.

free parameters (6)
  • first-codebook loss weight ratio = 5:1
    Eq. (3) sets L_audio = 5 L0 + (1/7) Σ L_k; the 5:1 ratio is hand-chosen to balance temporal planning vs residual refinement.
  • EMDC replacement probabilities p_k = 1 - Acc@1_k, per codebook
    §2.4; replacement rates for distractor conditioning are derived from measured teacher-forced token accuracies, making them empirical model-specific values.
  • four-way CFG guidance scales = (s_codec, s_lyrics, s_caption) = (1, 2, 1)
    Eq. (7); the scales used in the ablations are selected by hand for inference and are not derived from first principles.
  • DPO preference threshold δ = unspecified
    Eq. (9); a pair is retained only when the score difference exceeds δ, but the threshold value is not reported in the paper.
  • post-training combination weights λ_SFT, λ_RSFT, λ_OPD, λ1, λ2 = unspecified
    Eqs. (12), (16), (18), and (19); these coefficients control the auxiliary losses and reward normalization, but their values are not given.
  • GRPO group size G and variance-exclusion threshold = unspecified
    §2.6.3; group-relative rewards and the decision to exclude low-variance groups depend on G and a minimum variance threshold, neither of which is reported.
axioms (5)
  • domain assumption The 8-codebook RVQ tokenizer at 25 Hz provides a semantic-aware discrete representation sufficient for both lyrics/vocals and accompaniment, with c0 capturing enough global structure for temporal planning.
    Invoked throughout §2.2–2.3; if the tokens lose acoustic or semantic detail, hybrid-LM cannot plan or condition correctly.
  • domain assumption The hierarchical factorization p(A|X,M) = ∏ p(c0_t) ∏ p(c^k_t) from HeartMuLa [30] is a valid and sufficient generative model for full songs.
    Eq. (1), §2.3.2; the factorization is inherited from prior work and is not independently verified in this paper.
  • standard math Flow matching with linear interpolation z_σ = (1−σ)z0 + σϵ and velocity v* = ϵ−z0 is a valid objective for full-song latent acoustic synthesis, and the fixed VAE decoder is adequate for 48-kHz stereo output.
    Eqs. (5)–(6); standard flow-matching theory is invoked, but its suitability for 327-second latent sequences is assumed.
  • domain assumption The automatic metrics (SongBench, SongEval, Audiobox Aesthetics, CMI-Reward) are valid proxies for human musical quality, and the Artificial Analysis leaderboard is unbiased.
    §3 and Fig. 1; evaluation validity is assumed, and SongBench has an author overlap with the present paper.
  • domain assumption The anonymous leaderboard entry 'Lucky Dolphin' was produced by this exact system.
    Fig. 1 caption; no public mapping, code, or artifact verifies the identity of the submission.

pith-pipeline@v1.3.0-alltime-deepseek · 14290 in / 13817 out tokens · 113009 ms · 2026-08-01T10:20:09.638867+00:00 · methodology

0 comments
read the original abstract

In this report, we present a unified song generation framework capable of producing high-quality full-length music from lyrics, text descriptions, and musical attributes. The proposed framework supports three tasks: Lyrics-to-Song Generation, which generates complete songs from text descriptions, lyrics, and musical attributes; Instrumental Music Generation, which creates music without vocals; and Cover Song Generation, which reinterprets existing songs with different styles while preserving their melodic content. Architecturally, our system consists of four main components: a semantic-aware tokenizer, hybird-LM, FullDiT, and a two-level melody module. The tokenizer encodes audio into 8-codebook RVQ tokens for efficient discrete music representation. Based on these tokens, hybird-LM performs hierarchical autoregressive audio-token modeling for full-song generation. To improve audio fidelity, FullDiT performs full-song flow matching in a continuous VAE latent space conditioned on codec tokens, lyrics, and text captions. For cover song generation, the melody module extracts and discretizes melody cues from reference audio to guide generation while preserving the original melodic content. Finally, we investigate DPO, GRPO, and OPD as reward-based post-training strategies for hybird-LM and apply flow-based GRPO to FullDiT to improve musicality and rendering quality. Experimental results on a multilingual automatic benchmark, complemented by the Artificial Analysis Music with Vocals leaderboard, show that the proposed framework achieves competitive performance in the evaluated settings.

Figures

Figures reproduced from arXiv: 2607.20253 by Biao Tian, Bin Ma, Chongjia Ni, Haina Zhu, Han Zhao, Haoxu Wang, Huaicheng Zhang, Jianwei Yu, Junyu Dai, Menglin Wu, Shengkui Zhao, Weiqin Li, Xiangang Li, Xinyue Fan, Yufei Shi, Yukun Ma, Yunjia Li.

Figure 1
Figure 1. Figure 1: Results on the Artificial Analysis Music with Vocals leaderboard. The leaderboard entry [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the proposed framework. Given a text caption, optional lyrics, and optional reference audio, hybrid-LM autoregressively generates audio tokens. For cover song generation, the melody module extracts and discretizes melody cues from the reference audio as additional conditions for hybrid-LM. FullDiT then generates a continuous full-song VAE latent conditioned on the codec sequence, text caption, … view at source ↗
Figure 3
Figure 3. Figure 3: Three-stage training of the proposed tokenizer, which consists of BEST-RQ-style pretraining, [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Overview of the music data curation and staged training pipeline. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Overview of the hierarchical autoregressive architecture of hybrid-LM. Structured music [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Architecture of FullDiT. Given full-song RVQ tokens, lyrics, and a text caption, FullDiT injects frame-aligned codec embeddings into the audio hidden states and performs non-causal full￾song flow matching to generate a continuous VAE latent. Full-context text and audio modeling. Lyrics and captions are independently encoded and pro￾vided directly to FullDiT rather than being used only by hybrid-LM. The cap… view at source ↗
Figure 7
Figure 7. Figure 7: Given a raw input audio, we first apply a BS-RoFormer separator [19] to extract the vocal [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Overview of the RL-based post-training pipeline, including DPO and GRPO. [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

32 extracted references · 13 linked inside Pith

  1. [1]

    Musiclm: Generating music from text.arXiv preprint arXiv:2301.11325, 2023

    Andrea Agostinelli, Timo I Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, et al. Musiclm: Generating music from text.arXiv preprint arXiv:2301.11325, 2023

    Pith/arXiv arXiv 2023

  2. [2]

    Music generation benchmarking methodology

    Artificial Analysis. Music generation benchmarking methodology. https:// artificialanalysis.ai/music/methodology, 2026. Accessed: 2026-07-19

    2026

  3. [3]

    V ocals music leaderboard

    Artificial Analysis. V ocals music leaderboard. https://artificialanalysis.ai/music/ leaderboard/vocals, 2026. Leaderboard snapshot accessed: 2026-07-18

    2026

  4. [4]

    Yourmt3+: Multi- instrument music transcription with enhanced transformer architectures and cross-dataset stem augmentation

    Sungkyun Chang, Emmanouil Benetos, Holger Kirchhoff, and Simon Dixon. Yourmt3+: Multi- instrument music transcription with enhanced transformer architectures and cross-dataset stem augmentation. In2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP), pages 1–6. IEEE, 2024

    2024

  5. [5]

    Musicldm: Enhancing novelty in text-to-music generation using beat-synchronous mixup strategies

    Ke Chen, Yusong Wu, Haohe Liu, Marianna Nezhurina, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. Musicldm: Enhancing novelty in text-to-music generation using beat-synchronous mixup strategies. InICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1206–1210. IEEE, 2024

    2024

  6. [6]

    Visqol v3: An open source production ready objective speech and audio metric

    Michael Chinen, Felicia SC Lim, Jan Skoglund, Nikita Gureev, Feargus O’Gorman, and Andrew Hines. Visqol v3: An open source production ready objective speech and audio metric. In 2020 twelfth international conference on quality of multimedia experience (QoMEX), pages 1–6. IEEE, 2020

    2020

  7. [7]

    Self-supervised learning with random-projection quantizer for speech recognition

    Chung-Cheng Chiu, James Qin, Yu Zhang, Jiahui Yu, and Yonghui Wu. Self-supervised learning with random-projection quantizer for speech recognition. InInternational Conference on Machine Learning, pages 3915–3924. PMLR, 2022

    2022

  8. [8]

    Simple and controllable music generation.Advances in neural information processing systems, 36:47704–47720, 2023

    Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, and Alexandre Défossez. Simple and controllable music generation.Advances in neural information processing systems, 36:47704–47720, 2023

    2023

  9. [9]

    Ace-step 1.5: Pushing the boundaries of open-source music generation.arXiv preprint arXiv:2602.00744, 2026

    Junmin Gong, Yulin Song, Wenxiao Zhao, Sen Wang, Shengyuan Xu, Jing Guo, and Xuerui Yang. Ace-step 1.5: Pushing the boundaries of open-source music generation.arXiv preprint arXiv:2602.00744, 2026

    arXiv 2026

  10. [10]

    Ace-step: A step towards music generation foundation model.arXiv preprint arXiv:2506.00045, 2025

    Junmin Gong, Sean Zhao, Sen Wang, Shengyuan Xu, and Joe Guo. Ace-step: A step towards music generation foundation model.arXiv preprint arXiv:2506.00045, 2025

    Pith/arXiv arXiv 2025

  11. [11]

    Visqol: an objective speech quality model.EURASIP Journal on Audio, Speech, and Music Processing, 2015(1):13, 2015

    Andrew Hines, Jan Skoglund, Anil C Kokaram, and Naomi Harte. Visqol: an objective speech quality model.EURASIP Journal on Audio, Speech, and Music Processing, 2015(1):13, 2015. 16

    2015

  12. [12]

    Classifier-free diffusion guidance.CoRR, abs/2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.CoRR, abs/2207.12598, 2022

    Pith/arXiv arXiv 2022

  13. [13]

    Levo: High-quality song generation with multi- preference alignment.Advances in Neural Information Processing Systems, 38:102448–102479, 2026

    Shun Lei, Yaoxun Xu, Huaicheng Zhang, Hangting Chen, Yixuan Zhang, Chenyu Yang, Haina Zhu, Shuai Wang, Zhiyong Wu, Dong Yu, et al. Levo: High-quality song generation with multi- preference alignment.Advances in Neural Information Processing Systems, 38:102448–102479, 2026

    2026

  14. [14]

    Levo 2: Stable and melodious song gener- ation via hierarchical representation modeling and progressive post-training.arXiv preprint arXiv:2606.30642, 2026

    Shun Lei, Huaicheng Zhang, Dapeng Wu, Yaoxun Xu, Lishi Zuo, Wei Tan, Hangting Chen, Guangzheng Li, Jianwei Yu, Zhiyong Wu, et al. Levo 2: Stable and melodious song gener- ation via hierarchical representation modeling and progressive post-training.arXiv preprint arXiv:2606.30642, 2026

    Pith/arXiv arXiv 2026

  15. [15]

    Songecho: Towards cover song generation via instance-adaptive element-wise linear modulation.arXiv preprint arXiv:2602.19976, 2026

    Sifei Li, Yang Li, Zizhou Wang, Yuxin Zhang, Fuzhang Wu, Oliver Deussen, Tong-Yee Lee, and Weiming Dong. Songecho: Towards cover song generation via instance-adaptive element-wise linear modulation.arXiv preprint arXiv:2602.19976, 2026

    arXiv 2026

  16. [16]

    Duo-tok: Dual-track semantic music tokenizer for vocal-accompaniment generation.arXiv preprint arXiv:2511.20224, 2025

    Rui Lin, Zhiyue Wu, Jiahe Le, Kangdi Wang, Weixiong Chen, Junyu Dai, and Tao Jiang. Duo-tok: Dual-track semantic music tokenizer for vocal-accompaniment generation.arXiv preprint arXiv:2511.20224, 2025

    arXiv 2025

  17. [17]

    Khala: Scaling acoustic token language models toward high-fidelity music generation.arXiv preprint arXiv:2605.01790, 2026

    Jiafeng Liu, Yuanliang Dong, Hongjia Liu, Yuqing Cheng, Zhancheng Guo, Huijing Liang, Wenbo Zhan, Yuming Sun, Xiaobing Li, Feng Yu, et al. Khala: Scaling acoustic token language models toward high-fidelity music generation.arXiv preprint arXiv:2605.01790, 2026

    Pith/arXiv arXiv 2026

  18. [18]

    Flow-grpo: Training flow matching models via online rl.arXiv preprint arXiv:2505.05470, 2025

    Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-grpo: Training flow matching models via online rl.arXiv preprint arXiv:2505.05470, 2025

    Pith/arXiv arXiv 2025

  19. [19]

    Music source separation with band-split rope transformer

    Wei-Tsung Lu, Ju-Chiang Wang, Qiuqiang Kong, and Yun-Ning Hung. Music source separation with band-split rope transformer. InICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 481–485. IEEE, 2024

    2024

  20. [20]

    CMI- RewardBench: Evaluating music reward models with compositional multimodal instruction

    Yinghao Ma, Haiwen Xia, Hewei Gao, Weixiong Chen, Yuxin Ye, Yuchen Yang, Sungkyun Chang, Mingshuo Ding, Yizhi Li, Ruibin Yuan, Simon Dixon, and Emmanouil Benetos. CMI- RewardBench: Evaluating music reward models with compositional multimodal instruction. arXiv preprint arXiv:2603.00610, 2026

    Pith/arXiv arXiv 2026

  21. [21]

    Diffrhythm: Blazingly fast and embarrassingly simple end-to-end full-length song generation with latent diffusion.arXiv preprint arXiv:2503.01183, 2025

    Ziqian Ning, Huakang Chen, Yuepeng Jiang, Chunbo Hao, Guobin Ma, Shuai Wang, Jixun Yao, and Lei Xie. Diffrhythm: Blazingly fast and embarrassingly simple end-to-end full-length song generation with latent diffusion.arXiv preprint arXiv:2503.01183, 2025

    Pith/arXiv arXiv 2025

  22. [22]

    Meta audiobox aesthetics: Unified automatic quality assessment for speech, music, and sound

    Andros Tjandra, Yi-Chiao Wu, Baishan Guo, John Hoffman, Brian Ellis, Apoorv Vyas, Bowen Shi, Sanyuan Chen, Matt Le, Nick Zacharov, Carleigh Wood, Ann Lee, and Wei-Ning Hsu. Meta audiobox aesthetics: Unified automatic quality assessment for speech, music, and sound. arXiv preprint arXiv:2502.05139, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    Flowse-grpo: Training flow matching speech enhancement via online reinforcement learning

    Haoxu Wang, Biao Tian, Yiheng Jiang, Zexu Pan, Shengkui Zhao, Bin Ma, Daren Chen, and Xiangang Li. Flowse-grpo: Training flow matching speech enhancement via online reinforcement learning. InICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 16182–16186, 2026

    2026

  24. [24]

    Flowtts-grpo: Online reinforcement learning with multi-objective reward optimization for flow-matching based text-to-speech, 2026

    Haoxu Wang, Biao Tian, Weiqin Li, Xiang Lv, Han Zhao, and Xiangang Li. Flowtts-grpo: Online reinforcement learning with multi-objective reward optimization for flow-matching based text-to-speech, 2026

    2026

  25. [25]

    Rmvpe: A robust model for vocal pitch estimation in polyphonic music.arXiv preprint arXiv:2306.15412, 2023

    Haojie Wei, Xueke Cao, Tangpeng Dan, and Yueguo Chen. Rmvpe: A robust model for vocal pitch estimation in polyphonic music.arXiv preprint arXiv:2306.15412, 2023

    Pith/arXiv arXiv 2023

  26. [26]

    SongBench: A fine-grained multi-aspect benchmark for song quality assessment.arXiv preprint arXiv:2604.25937, 2026

    Dapeng Wu, Shun Lei, Wei Tan, Guangzheng Li, Yunzhe Wang, Huaicheng Zhang, Lishi Zuo, and Zhiyong Wu. SongBench: A fine-grained multi-aspect benchmark for song quality assessment.arXiv preprint arXiv:2604.25937, 2026. 17

    Pith/arXiv arXiv 2026

  27. [27]

    Mucodec: Ultra low-bitrate music codec for music generation

    Yaoxun Xu, Hangting Chen, Jianwei Yu, Wei Tan, Shun Lei, Zhiwei Lin, Rongzhi Gu, and Zhiyong Wu. Mucodec: Ultra low-bitrate music codec for music generation. InProceedings of the 33rd ACM International Conference on Multimedia, pages 689–698, 2025

    2025

  28. [28]

    Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram

    Ryuichi Yamamoto, Eunwoo Song, and Jae-Min Kim. Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram. In2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6199–6203. IEEE, 2020

    2020

  29. [29]

    Songbloom: Coherent song generation via interleaved autoregressive sketching and diffusion refinement

    Chenyu Yang, Shuai Wang, Hangting Chen, Wei Tan, Jianwei Yu, and Haizhou Li. Songbloom: Coherent song generation via interleaved autoregressive sketching and diffusion refinement. Advances in Neural Information Processing Systems, 38:34256–34277, 2026

    2026

  30. [30]

    Heartmula: A family of open sourced music foundation models.arXiv preprint arXiv:2601.10547, 2026

    Dongchao Yang, Yuxin Xie, Yuguo Yin, Zheyu Wang, Xiaoyu Yi, Gongxi Zhu, Xiaolong Weng, Zihan Xiong, Yingzhe Ma, Dading Cong, et al. Heartmula: A family of open sourced music foundation models.arXiv preprint arXiv:2601.10547, 2026

    Pith/arXiv arXiv 2026

  31. [31]

    SongEval: A benchmark dataset for song aesthetics evaluation.arXiv preprint arXiv:2505.10793, 2025

    Jixun Yao, Guobin Ma, Huixin Xue, Huakang Chen, Chunbo Hao, Yuepeng Jiang, Haohe Liu, Ruibin Yuan, Jin Xu, Wei Xue, Hao Liu, and Lei Xie. SongEval: A benchmark dataset for song aesthetics evaluation.arXiv preprint arXiv:2505.10793, 2025

    Pith/arXiv arXiv 2025

  32. [32]

    Yue: Scaling open foundation models for long-form music generation.arXiv preprint arXiv:2503.08638, 2025

    Ruibin Yuan, Hanfeng Lin, Shuyue Guo, Ge Zhang, Jiahao Pan, Yongyi Zang, Haohe Liu, Yiming Liang, Wenye Ma, Xingjian Du, et al. Yue: Scaling open foundation models for long-form music generation.arXiv preprint arXiv:2503.08638, 2025. 18

    arXiv 2025