Pith. sign in

REVIEW 3 major objections 5 minor 44 references

Adding explicit phoneme timing to melody-conditioned cover generation cuts lyric errors by more than half.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 23:38 UTC pith:YI37BSGL

load-bearing objection Solid engineering paper that actually fixes lyric mangling in melody-conditioned cover generation; the PER drop is real and the ablations are the right ones, with one clear domain-shift caveat on Phonsa. the 3 major comments →

arxiv 2607.26698 v1 pith:YI37BSGL submitted 2026-07-29 cs.SD cs.AIeess.AS

MPEcho: A Melody and Phoneme-Aware Generative Framework for Controllable Cover Song Generation

Wei-Jaw Lee , Hsuan-Yu Yeh , Ting-Yi Hu , Chih-Pin Tan , Fang-Duo Tsai , Yi-Hsuan Yang This is my paper
classification cs.SD cs.AIeess.AS
keywords cover song generationphoneme conditioninglength regulatorsinging voice synthesismelody controlphoneme alignmentlyrics-to-song
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Cover song generation must keep a reference song’s melody and lyrics while freely reinventing the rest of the arrangement. Prior melody-only models still produce garbled lyrics because they lack precise phoneme timing. MPEcho borrows the length-regulator idea from singing-voice synthesis, feeding the generator both a melody contour and a duration-expanded phoneme sequence extracted by a new Whisper-based aligner called Phonsa. The result is substantially lower phoneme error rate, competitive pitch accuracy, and higher listener preference, showing that fine-grained linguistic control can be added to full-song generative models without sacrificing musical quality.

Core claim

Explicit SVS-style phoneme conditioning plus a length regulator, when fused with melody conditioning inside a lyrics-to-song backbone, reduces phoneme error rate from 45.62 % to 18.65 % while preserving raw pitch and chroma accuracy and improving subjective scores for melody consistency, vocal naturalness, prompt adherence and overall quality.

What carries the argument

The SVS-style phoneme arrangement: a phoneme encoder followed by a length regulator that expands each phoneme embedding according to Phonsa-derived onset/offset durations, producing a time-aligned conditioning stream that is injected via instance-adaptive adapters.

Load-bearing premise

The phoneme timestamps produced by Phonsa, trained only on clean solo singing corpora, remain accurate enough on commercial pop tracks that contain accompaniment leakage and production effects to serve as reliable duration targets.

What would settle it

Re-run the identical MPEcho training and evaluation pipeline using ground-truth human phoneme alignments on the same internal Chinese pop set; if the PER gap versus SongEcho largely disappears, the reported gains are artifacts of Phonsa’s label noise rather than true linguistic control.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Full-song generative models can inherit the fine temporal control previously limited to isolated singing-voice synthesizers.
  • Automatic phoneme aligners trained on SVS data become a practical prerequisite for controllable cover-song systems.
  • Multi-condition classifier-free guidance that separately scales text, lyrics and time-varying streams measurably reduces condition conflict.
  • Word-level timing is insufficient; contiguous phoneme-level boundaries are required for adapter-based lyric control.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same length-regulator pattern could be attached to other open lyrics-to-song backbones beyond the one used here, suggesting a general recipe for lyric-faithful style transfer.
  • Because Phonsa already supports unsupervised segmentation, the pipeline could be extended to lyric-free or multi-lingual cover generation with only modest additional alignment data.
  • Listener preference for joint melody-plus-phoneme control implies that objective pitch and PER metrics still under-weight the joint percept of intelligibility and tunefulness.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes MPEcho, an extension of SongEcho for controllable cover song generation that adds explicit SVS-style phoneme conditioning via a phoneme encoder and length regulator on top of melody (F0 + V/UV) conditioning. To supply phoneme sequences and durations, the authors introduce Phonsa, a Whisper-based forced aligner/segmenter trained on public Mandarin SVS corpora. On an internal Chinese pop CSG set, ablations show that joint melody+phoneme conditioning lowers PER from SongEcho’s 45.62% to 18.65% while keeping competitive RPA/RCA and Audiobox scores; multi-condition CFG further helps, and a 25-listener MOS prefers the full model. Code, weights, and demos are released.

Significance. If the reported gains hold under reliable target-domain alignments, the work is a clear, practical advance for audio-domain CSG: it shows that SVS-style phoneme timing can be grafted onto a pretrained LTS backbone with lightweight adapters and yields measurable lyric intelligibility gains without collapsing melody control. The controlled ablations (melody-only / phoneme-only / M+P, SVS- vs Jam-style, three guidance schemes), external metrics (mir_eval, SongPrep PER, Audiobox, CLAP), human MOS, and public code/weights are genuine strengths that make the directional claim falsifiable and reusable. The main significance is therefore methodological—bringing fine-grained linguistic control into full-song generation—rather than a new generative backbone.

major comments (3)
  1. [§4–5, Table 1, Table 2] §4–5 and Table 1: Phonsa is trained and validated exclusively on clean SVS sets (M4Singer, Opencpop; test GTsinger), yet it supplies the duration sequence D that the length regulator expands into cp for every track of the internal ~1,427 h commercial Chinese-pop CSG corpus. No MAE/PCO/PCAS, error distribution, or human spot-check of Phonsa alignments on that target domain (accompaniment leakage, production effects, different singers) is reported. Because the central PER claim (Table 2, ID 1→3: 0.4562→0.1865) rests on these timestamps both at train and inference time, the paper must quantify alignment quality on a held-out slice of the CSG data (or a matched commercial-style set). Without that, it remains possible that part of the PER drop reflects fitting to systematically biased labels rather than true linguistic control.
  2. [§6.2, Table 2] §6.2 / Table 2: PER is computed by running SongPrep on the generated audio. SongPrep is also used for structure parsing of the training data (§5). The paper should clarify whether SongPrep’s lyric/phoneme front-end was held completely out of the evaluation loop (or report an independent ASR/PER pipeline) so that the large PER reduction cannot be partly attributed to matching SongPrep’s own error modes. A short cross-check with an external Mandarin ASR would strengthen the lyric-accuracy claim.
  3. [§6.3, Table 4] §6.3 / Table 4: The subjective test uses only 12 items and 25 listeners. While the ranking is consistent with the objective ablations, the confidence intervals are wide and the prompt set is LLM-generated. For a claim of clear preference of M+P(MC) over SongEcho, either expand the listening set or report pairwise significance tests; otherwise the MOS should be framed more cautiously as supportive rather than decisive.
minor comments (5)
  1. [Title] Title and running header contain spacing artifacts (“A W ARE”, “GENERA TIVE”, “IMPLEMENTA TION”). Clean for camera-ready.
  2. [§3.2] Eq. (4) is dense; a short expanded derivation or pseudocode for the multi-condition APG would help reproducibility.
  3. [Figure 3, §3.1] Figure 3 caption and body should explicitly define the [V_F] token and how word-level timestamps are converted to phoneme slots in the Jam-style baseline.
  4. [§6.2] CLAP scores are uniformly low; the domain-mismatch footnote is helpful but could be moved into the main text of §6.2 so readers do not over-interpret absolute CLAP values.
  5. [§4] State the phoneme inventory size |C| and whether breath/boundary tokens are language-specific; this matters for the claimed multilingual future work.

Circularity Check

1 steps flagged

No significant circularity: PER/pitch/MOS gains are measured by external tools on held-out tracks; only a minor non-load-bearing self-citation for multi-condition CFG.

specific steps
  1. self citation load bearing [§3.2 Eq. (4); refs [16], [23]]
    "Following previous works [16, 23], we instead apply multiple CFG on APG: ... where ωt, ωl, ωa are the guidance scales for text, lyrics, and time-varying controls (melody and phoneme), respectively"

    Multi-condition guidance form is imported from MuseControlLite (overlapping authors) plus InstructPix2Pix. This is methodological scaffolding only: Table 3 still ablates N/SE/MC empirically, and the main PER claim does not reduce to the citation. Minor and non-load-bearing.

full rationale

MPEcho’s central claim (explicit SVS-style phoneme conditioning + LR lowers PER from 0.4562 to 0.1865 while preserving RPA/RCA and raising MOS) is an empirical ablation result, not a derivation that collapses to its inputs. Phonsa is trained and scored on public SVS corpora (M4Singer/Opencpop train, GTsinger test) with independent alignment metrics (MAE/PCO/PCAS vs MFA); those timestamps then condition the CSG model, but the reported CSG metrics (SongPrep PER, mir_eval pitch, Audiobox, human MOS) are external and not algebraically forced by the duration sequence D. SongEcho and ACE-Step baselines are reimplemented on the same internal split and backbone. Guidance scales are chosen hyperparameters, not fitted constants renamed as predictions. The sole minor self-touch is adopting multi-condition APG/CFG style from the authors’ prior MuseControlLite; that choice is validated by Table 3 ablations and is not a uniqueness theorem that forces the PER result. Domain-shift risk of Phonsa on commercial pop is a correctness/assumption issue, not circularity. Score 1 for the non-load-bearing self-cite only.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 2 invented entities

The claim rests on standard generative-modeling practice (flow matching, CFG/APG), the transferability of SVS duration control to full-mix CSG, and the reliability of automatic phoneme timestamps and automatic lyric ASR as evaluation oracles. No new physical entities; free parameters are ordinary ML hyperparameters and guidance scales selected on validation metrics.

free parameters (4)
  • multi-condition CFG scales (ω_t, ω_l, ω_a) = (15.0, 7.5, 5.0)
    Chosen by ablation (Table 3); final reported system uses (15.0, 7.5, 5.0). Directly affects RPA/RCA/PER trade-off that supports the central claim.
  • SongEcho-style APG mix α = 0.5
    Fixed at 0.5 for fair comparison with SongEcho; still a free inference hyperparameter in the SE baseline path.
  • Phonsa multi-task weight λ and training schedule
    λ balances CTC vs frame CE; 24k steps, dual learning rates (5e-3 head, 1e-5 encoder). Affects alignment quality that feeds MPEcho.
  • phoneme FFT depth/width and adapter LR (1e-4, 50k steps) = 4-layer 2-head FFT, Dp=256; AdamW 1e-4, 50k steps
    Architecture and optimization choices for the 12.1M phoneme encoder and IA-EiLM adapters; not derived from theory.
axioms (5)
  • domain assumption Explicit phoneme identity plus duration boundaries are sufficient linguistic conditioning to improve lyric accuracy in a pretrained LTS DiT when injected via adapters and length regulation.
    Imported from SVS (FastSpeech-style) and treated as transferable to full-song CSG (§1, §3.1).
  • domain assumption Flow-matching denoising on ACE-Step latents with IA-EiLM adapters can absorb new time-varying conditions (melody + phoneme) without destroying pretrained musicality.
    Inherited from ACE-Step and SongEcho training recipe (§3, Eq. 1).
  • domain assumption SongPrep ASR transcripts are an adequate oracle for computing PER on generated Mandarin singing.
    Used as the sole lyric-accuracy metric in Tables 2–3; error rate of SongPrep itself on AI singing is not reported (§6.2).
  • domain assumption RMVPE F0 and V/UV extraction plus Phonsa timestamps from the reference vocal are faithful enough proxies for the intended melody and lyric timing of a cover.
    Conditioning pipeline shared with SongEcho and extended in §3–4.
  • standard math Standard flow-matching / CFG / APG mathematics as used in recent diffusion music models.
    Eqs. (1)–(4); not re-derived.
invented entities (2)
  • MPEcho independent evidence
    purpose: Name for the full melody+phoneme CSG system (ACE-Step + SongEcho melody path + phoneme encoder/LR + multi-condition guidance).
    System-level composition; no new physical object. Independent evidence is the reported metrics and promised weights.
  • Phonsa independent evidence
    purpose: Whisper-based phoneme aligner/segmenter supplying P and D for the length regulator.
    New trained model with architectural tweaks (chunked self-attention, breath/boundary tokens). Evaluated on public GTsinger against MFA; code/weights promised.

pith-pipeline@v1.2.0-daily-grok45 · 17308 in / 3806 out tokens · 62289 ms · 2026-07-30T23:38:44.115176+00:00 · methodology

0 comments
read the original abstract

Cover song generation (CSG) should preserve the melodic and linguistic content of a reference song while recreating the remaining musical components. The state-of-the-art model SongEcho utilizes $F_0$ sequences and voiced/unvoiced (V/UV) tags for conditioning; however, implicit linguistic information from V/UV tags cannot guarantee lyric accuracy, leading to a high phoneme error rate (PER). Inspired by singing voice synthesis (SVS), we propose MPEcho, which integrates a phoneme encoder and a length regulator (LR) into the SongEcho framework. By providing explicit phoneme-level conditioning and precise temporal boundaries, MPEcho significantly reduces PER. To enable this, we developed Phonsa, a Whisper-based automatic transcription model that provides high-precision phoneme-level annotations for singing voices, overcoming the scarcity of high-quality audio-phoneme pairs. Experimental results validate the effectiveness of Phonsa for alignment and MPEcho for end-to-end CSG. The audio samples, code and weights can be accessed from https://lonian6.github.io/MPEcho.github.io/.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

44 extracted references · 10 linked inside Pith

  1. [1]

    MPEcho: A Melody and Phoneme-Aware Generative Framework for Controllable Cover Song Generation

    INTRODUCTION Cover song generation (CSG) is an emerging generative task in music information retrieval and generative AI [1], requiring the preservation of a song’s core melody and lyri- cal structure while enabling creative reinterpretation of vo- cal performance and arrangement. This balance between structural preservation and creative freedom makes CSG...

    Pith/arXiv arXiv 2026

  2. [2]

    Recent systems [2, 3, 5, 12] produce coherent long-form music with high fidelity

    BACKGROUND LTS aims to synthesize full songs from lyrics and text prompts. Recent systems [2, 3, 5, 12] produce coherent long-form music with high fidelity. CSG, a subtask of mu- sic style transfer, derives new adaptations while preserving a source song’s identity. While symbolic CSG (e.g., piano covers) often decouples attribute extraction from genera- t...

  3. [3]

    MPECHO Figure 2(a) provides an overview of MPEcho, extending SongEcho [1] by introducing a dedicated phoneme condi- tioning branch. It inherits SongEcho’s architecture, includ- ing the ACE-Step DiT backbone [2], the instance-adaptive element-wise linear modulation (IA-EiLM) adapters, and the RMVPE [21] extractor for vocal melody conditioning. Formally, le...

  4. [4]

    Given a singing voice audio signal repre- sented by acoustic featuresx= [x 1, x2,

    PHONSA Figure 2(b) offers an overview of Phonsa, a unified frame- work for phoneme-level forced alignment and segmenta- tion, building upon and extending prior word-level ap- proach [18]. Given a singing voice audio signal repre- sented by acoustic featuresx= [x 1, x2, . . . , xT ], where xt denotes the Mel-spectrogram at framet, our goal is to estimate b...

  5. [5]

    A subset of samples is randomly selected from each of these training datasets to serve as validation data during training

    IMPLEMENTA TION DETAILS For Phonsa, we use M4Singer [24], Opencpop [25] for training. A subset of samples is randomly selected from each of these training datasets to serve as validation data during training. We use the GTsinger dataset [26] exclu- sively as our test set. The training, validation, and test sets comprised audio data totaling 30.92 hours, 1...

  6. [6]

    EXPERIMENTS 6.1 Objective Evaluation of Phonsa We first evaluate the performance of phonetic transcrip- tion tasks, comparing Phonsa with MFA [11] and two ab- lated versions of Phonsa, one without the boundary tokens (‘Phonsa w/o BT’), the other using the RNN backbone of

  7. [7]

    CONCLUSION In this paper, we have presented MPEcho, the first frame- work integrating explicit phoneme-level condition into LTS to achieve better CSG. By incorporating an SVS- inspired phonetic control branch and a length regulator into melody-guided cover generation, MPEcho provides finer-grained linguistic control and improves lyric intel- ligibility ov...

  8. [8]

    We consider two tasks: alignment and segmentation

    (‘Phonsa (RNN)’) instead of self-attention. We consider two tasks: alignment and segmentation. Forforced alignment, we adopt three objective metrics. Mean absolute error (MAE) measures the average devia- tion, in milliseconds, between predicted and ground-truth phoneme boundaries. Percentage of correct onsets (PCO) computes the proportion of predicted ons...

    arXiv 1965

  9. [9]

    ACKNOWLEDGMENT The work is supported by grants from Google Asia Pacific, the National Science and Technology Council of Taiwan (NSTC 114-2628-E-002-013-MY3), and the Ministry of Education (MOE) of Taiwan (for Taiwan Centers of Ex- cellence in Artificial Intelligence)

  10. [10]

    After using this service, we re- viewed and edited the content carefully and take full re- sponsibility for the content of the published article

    AI USAGE STA TEMENT During the preparation of this work, we used Gemini, ChatGPT to refine the linguistic quality and improve the clarity of the manuscript. After using this service, we re- viewed and edited the content carefully and take full re- sponsibility for the content of the published article. The use of these technologies was strictly limited to ...

  11. [11]

    Montreal forced aligner: Trainable text-speech alignment using kaldi

    M. McAuliffe, M. Socolof, S. Mihuc, M. Wagner, and M. Sonderegger, “Montreal forced aligner: Trainable text-speech alignment using kaldi.” inInterspeech, vol. 2017, 2017, pp. 498–502

    2017

  12. [12]

    SongEcho: Cover song gener- ation via instance-adaptive element-wise linear modu- lation,

    S. Li, Y . Li, Z. Wang, Y . Zhang, F. Wu, O. Deussen, T.- Y . Lee, and W. Dong, “SongEcho: Cover song gener- ation via instance-adaptive element-wise linear modu- lation,” inProceedings of the International Conference on Learning Representations (ICLR), 2026

    2026

  13. [13]

    Ace- step: A step towards music generation foundation model,

    J. Gong, S. Zhao, S. Wang, S. Xu, and J. Guo, “Ace- step: A step towards music generation foundation model,”arXiv preprint arXiv:2506.00045, 2025

    Pith/arXiv arXiv 2025

  14. [14]

    DiffRhythm: Blazingly fast and embarrassingly sim- ple end-to-end full-length song generation with latent diffusion,

    N. Ziqian, C. Huakang, J. Yuepeng, H. Chunbo, M. Guobin, W. Shuai, Y . Jixun, and X. Lei, “DiffRhythm: Blazingly fast and embarrassingly sim- ple end-to-end full-length song generation with latent diffusion,”arXiv preprint arXiv:2503.01183, 2025

    Pith/arXiv arXiv 2025

  15. [15]

    Jam: A tiny flow-based song generator with fine- grained controllability and aesthetic alignment,

    R. Liu, C.-Y . Hung, N. Majumder, T. Gautreaux, A. A. Bagherzadeh, C. Li, D. Herremans, and S. Poria, “Jam: A tiny flow-based song generator with fine- grained controllability and aesthetic alignment,” 2025. [Online]. Available: https://arxiv.org/abs/2507.20880

    Pith/arXiv arXiv 2025

  16. [16]

    Songbloom: Coherent song generation via interleaved autoregressive sketching and diffusion refinement,

    C. Yang, S. Wang, H. Chen, W. Tan, J. Yu, and H. Li, “Songbloom: Coherent song generation via interleaved autoregressive sketching and diffusion refinement,” in The Thirty-ninth Annual Conference on Neural Infor- mation Processing Systems, 2025

    2025

  17. [17]

    Diffsinger: Singing voice synthesis via shallow diffusion mecha- nism,

    J. Liu, C. Li, Y . Ren, F. Chen, and Z. Zhao, “Diffsinger: Singing voice synthesis via shallow diffusion mecha- nism,” inProceedings of the AAAI conference on ar- tificial intelligence, vol. 36, no. 10, 2022, pp. 11 020– 11 028

    2022

  18. [18]

    First, we move from word-level to phoneme-level modeling, enabling finer-grained temporal control for singing voice, where timing variations are more pronounced

    are three-fold. First, we move from word-level to phoneme-level modeling, enabling finer-grained temporal control for singing voice, where timing variations are more pronounced. Second, we replace the RNN-based classi- fier [18] with a chunked self-attention architecture, which improves computational efficiency through parallelization while better capturi...

  19. [19]

    Latent flow matching for expressive singing voice synthesis,

    M. Yun and Y .-H. Choi, “Latent flow matching for expressive singing voice synthesis,”arXiv preprint arXiv:2601.00217, 2026

    arXiv 2026

  20. [20]

    Techsinger: Tech- nique controllable multilingual singing voice synthesis via flow matching,

    W. Guo, Y . Zhang, C. Pan, R. Huang, L. Tang, R. Li, Z. Hong, Y . Wang, and Z. Zhao, “Techsinger: Tech- nique controllable multilingual singing voice synthesis via flow matching,” inProceedings of the AAAI Con- ference on Artificial Intelligence, vol. 39, no. 22, 2025, pp. 23 978–23 986

    2025

  21. [21]

    Fastspeech: Fast, robust and control- lable text to speech,

    Y . Ren, Y . Ruan, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y . Liu, “Fastspeech: Fast, robust and control- lable text to speech,”Advances in neural information processing systems, vol. 32, 2019

    2019

  22. [22]

    Robust speech recog- nition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recog- nition via large-scale weak supervision,” inInterna- tional conference on machine learning. PMLR, 2023, pp. 28 492–28 518

    2023

  23. [23]

    Jukebox: A generative model for music,

    P. Dhariwal, H. Jun, C. Payne, J. W. Kim, A. Radford, and I. Sutskever, “Jukebox: A generative model for music,”arXiv preprint arXiv:2005.00341, 2020

    Pith/arXiv arXiv 2005

  24. [24]

    Pop2piano: Pop audio-based pi- ano cover generation,

    J. Choi and K. Lee, “Pop2piano: Pop audio-based pi- ano cover generation,” inICASSP 2023-2023 IEEE In- ternational Conference on Acoustics, Speech and Sig- nal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  25. [25]

    Picogen: Gen- erate piano covers with a two-stage approach,

    C.-P. Tan, S.-H. Guan, and Y .-H. Yang, “Picogen: Gen- erate piano covers with a two-stage approach,” inPro- ceedings of the 2024 International Conference on Mul- timedia Retrieval, 2024, pp. 1180–1184

    2024

  26. [26]

    Etude: Piano cover gener- ation with a three-stage approach-extract, structuralize, and decode,

    T.-Y . Chen and Y .-J. Joung, “Etude: Piano cover gener- ation with a three-stage approach-extract, structuralize, and decode,”arXiv preprint arXiv:2509.16522, 2025

    arXiv 2025

  27. [27]

    Musecontrollite: Mul- tifunctional music generation with lightweight condi- tioners,

    F.-D. Tsai, S.-L. Wu, W. Lee, S.-P. Yang, B.-R. Chen, H.-C. Cheng, and Y .-H. Yang, “Musecontrollite: Mul- tifunctional music generation with lightweight condi- tioners,”arXiv preprint arXiv:2506.18729, 2025

    Pith/arXiv arXiv 2025

  28. [28]

    Phoneme level lyrics alignment and text- informed singing voice separation,

    K. Schulze-Forster, C. S. J. Doire, G. Richard, and R. Badeau, “Phoneme level lyrics alignment and text- informed singing voice separation,”IEEE/ACM Trans- actions on Audio, Speech, and Language Processing, vol. 29, pp. 2382–2395, 2021

    2021

  29. [29]

    Adapting pretrained speech model for mandarin lyrics transcription and alignment,

    J.-Y . Wang, C.-I. Leong, Y .-C. Lin, L. Su, and J.-S. R. Jang, “Adapting pretrained speech model for mandarin lyrics transcription and alignment,” in2023 IEEE Au- tomatic Speech Recognition and Understanding Work- shop (ASRU), 2023, pp. 1–8

    2023

  30. [30]

    Contrastive learning-based audio to lyrics alignment for multiple languages,

    S. Durand, D. Stoller, and S. Ewert, “Contrastive learning-based audio to lyrics alignment for multiple languages,” inICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Process- ing (ICASSP), 2023, pp. 1–5

    2023

  31. [31]

    Stars: A unified frame- work for singing transcription, alignment, and re- fined style annotation,

    W. Guo, Y . Zhang, C. Pan, Z. Zhu, R. Li, Z. Chen, W. Xu, F. Wu, and Z. Zhao, “Stars: A unified frame- work for singing transcription, alignment, and re- fined style annotation,” inFindings of the Association for Computational Linguistics: ACL 2025, 2025, pp. 15 081–15 093

    2025

  32. [32]

    Rmvpe: A robust model for vocal pitch estimation in polyphonic music,

    H. Wei, X. Cao, T. Dan, and Y . Chen, “Rmvpe: A robust model for vocal pitch estimation in polyphonic music,”arXiv preprint arXiv:2306.15412, 2023

    Pith/arXiv arXiv 2023

  33. [33]

    Eliminat- ing oversaturation and artifacts of high guidance scales in diffusion models,

    S. Sadat, O. Hilliges, and R. M. Weber, “Eliminat- ing oversaturation and artifacts of high guidance scales in diffusion models,” inThe Thirteenth International Conference on Learning Representations, 2024

    2024

  34. [34]

    Instruct- pix2pix: Learning to follow image editing instruc- tions,

    T. Brooks, A. Holynski, and A. A. Efros, “Instruct- pix2pix: Learning to follow image editing instruc- tions,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 18 392–18 402

    2023

  35. [35]

    M4singer: A multi-style, multi-singer and musical score provided mandarin singing corpus,

    L. Zhang, R. Li, S. Wang, L. Deng, J. Liu, Y . Ren, J. He, R. Huang, J. Zhu, X. Chen, and Z. Zhao, “M4singer: A multi-style, multi-singer and musical score provided mandarin singing corpus,” inThirty- sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022

    2022

  36. [36]

    Opencpop: A high-quality open source chinese popular song cor- pus for singing voice synthesis,

    Y . Wang, X. Wang, P. Zhu, J. Wu, H. Li, H. Xue, Y . Zhang, L. Xie, and M. Bi, “Opencpop: A high-quality open source chinese popular song cor- pus for singing voice synthesis,”arXiv preprint arXiv:2201.07429, 2022

    Pith/arXiv arXiv 2022

  37. [37]

    Gtsinger: A global multi-technique singing corpus with realistic music scores for all singing tasks,

    Y . Zhang, C. Pan, W. Guo, R. Li, Z. Zhu, J. Wang, W. Xu, J. Lu, Z. Hong, C. Wanget al., “Gtsinger: A global multi-technique singing corpus with realistic music scores for all singing tasks,”Advances in Neu- ral Information Processing Systems, vol. 37, pp. 1117– 1140, 2024

    2024

  38. [38]

    Songprep: A preprocessing framework and end-to-end model for full-song struc- ture parsing and lyrics transcription,

    W. Tan, S. Lei, H. Zhang, G. Li, Y . Zhang, H. Chen, J. Yu, R. Gu, and D. Yu, “Songprep: A preprocessing framework and end-to-end model for full-song struc- ture parsing and lyrics transcription,”arXiv preprint arXiv:2509.17404, 2025

    arXiv 2025

  39. [39]

    Qwen2-audio tech- nical report,

    Y . Chu, J. Xu, Q. Yang, H. Wei, X. Wei, Z. Guo, Y . Leng, Y . Lv, J. He, J. Linet al., “Qwen2-audio tech- nical report,”arXiv preprint arXiv:2407.10759, 2024

    Pith/arXiv arXiv 2024

  40. [40]

    Es- sentia: An audio analysis library for music informa- tion retrieval,

    D. Bogdanov, N. Wack, E. Gómez Gutiérrez, S. Gu- lati, P. Herrera Boyer, O. Mayor, G. Roma Trepat, J. Salamon, J. R. Zapata González, and X. Serra, “Es- sentia: An audio analysis library for music informa- tion retrieval,” inBritto A, Gouyon F , Dixon S, editors. 14th Conference of the International Society for Mu- sic Information Retrieval (ISMIR); 2013 ...

    2013

  41. [41]

    Phoneme segmentation us- ing self-supervised speech models,

    L. Strgar and D. Harwath, “Phoneme segmentation us- ing self-supervised speech models,” in2022 IEEE Spo- ken Language Technology Workshop (SLT), 2023, pp. 1067–1073

    2023

  42. [42]

    Meta audiobox aesthetics: Unified automatic qual- ity assessment for speech, music, and sound,

    A. Tjandra, Y .-C. Wu, B. Guo, J. Hoffman, B. Ellis, A. Vyas, B. Shi, S. Chen, M. Le, N. Zacharovet al., “Meta audiobox aesthetics: Unified automatic qual- ity assessment for speech, music, and sound,”arXiv preprint arXiv:2502.05139, 2025

    Pith/arXiv arXiv 2025

  43. [43]

    Clap learning audio concepts from natural language supervision,

    B. Elizalde, S. Deshmukh, M. Al Ismail, and H. Wang, “Clap learning audio concepts from natural language supervision,” inICASSP 2023-2023 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  44. [44]

    Mir_eval: A transparent implementation of common mir metrics

    C. Raffel, B. McFee, E. J. Humphrey, J. Salamon, O. Nieto, D. Liang, D. P. Ellis, and C. C. Raffel, “Mir_eval: A transparent implementation of common mir metrics.” inISMIR, vol. 10, 2014, p. 2014

    2014