Pith. sign in

REVIEW 3 major objections 5 minor 54 references

Neurodyne: Neural Pitch Manipulation with Representation Learning and Cycle-Consistency GAN

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Neurodyne claims that neural pitch manipulation can reach high fidelity without paired in- and out-of-tune data, by learning a pitch-independent latent code and enforcing cycle-consistency during training.

desk verdict Solid neural pitch-manipulation system with a genuinely new cycle-consistency objective; the F0RMSE numbers need an independent pitch-estimator cross-check before they carry the weight claimed. read the letter →

arxiv 2505.15368 v4 pith:HR5PFGTS submitted 2025-05-21 cs.SD eess.AS

Yicheng Gu , Chaoren Wang , Zhizheng Wu , Lauri Juvela This is my paper · ORCID
classification cs.SDeess.AS
keywords pitchmanipulationsingingvoicesynthesisadversarialrepresentationlearningcycle-consistencyGANpitch-independentlatentcorrectionneuralvocodergenerativenetworks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Neurodyne proposes a neural pitch-manipulation system that drops the usual source-filter decomposition and its imperfect independence assumption. Instead, an adversarial objective pushes pitch information out of a latent code while a decoder re-injects the target pitch through neural source-filter oscillators. Because no paired in-tune/out-of-tune recordings exist, the system manufactures such pairs implicitly through two cycle-consistency losses, one that inverts a manipulation and one that equates one-step and two-step manipulations. The paper reports that this design outperforms DSP and neural baselines in audio quality, singer similarity, and pitch accuracy on both global-key and template-based tasks. If correct, the result means high-fidelity pitch correction can be trained from ordinary in-tune singing data alone.

What carries the argument

The load-bearing object is the pitch-independent latent representation produced by adversarial representation learning. A pitch predictor tries to estimate the singer's pitch vector from the latent code, the encoder tries to maximize the flatness of that prediction, and the resulting leak penalty makes the latent carry timbre and content but not pitch. A multi-scale oscillator, made of four neural source-filter excitation generators at different sampling rates, injects the target pitch contour into the decoder at several resolutions. The cycle-consistency losses, inversion and composition, supply the implicit paired data and keep the decoder from ignoring the given pitch and falling back on leaked pitch from the encoder.

What would settle it

Measure how much pitch information remains recoverable from the frozen latent code by training a separate probe to predict the sung fundamental frequency from latent vectors; if the probe's accuracy stays high on manipulated audio, the pitch-independent claim fails. A second check is to evaluate F0RMSE using a different pitch tracker than the one used for training; if the reported gap over baselines shrinks or reverses, the objective gains are partly an artifact of the chosen estimator.

Watch

Extended reading notes

Core claim

The central claim is that pitch manipulation can be made more faithful by separating pitch from the rest of the singing voice in the learned representation rather than in hand-designed acoustic parameters. Neurodyne's encoder is trained so that a pitch predictor cannot recover the fundamental frequency from the latent code, and this adversarially cleaned code is combined with a separately given target pitch contour to drive the decoder. Two cycle-consistency terms then teach the decoder to follow the target pitch: in one, manipulating and then restoring the original pitch contour must recover the input; in the other, reaching a target contour in one step must match reaching it in two steps through an intermediate contour. Experiments in global-key shifts from $-12$ to $+12$ semitones and in template-based correction of out-of-tune recordings show the system beating the classical DSP baselines and the neural-vocoder baselines on F0 error and on listener-rated quality and similarity.

Load-bearing premise

The pitch estimator that defines the adversarial leak penalty (the loss that tries to keep pitch information out of the latent code) and extracts the training targets must be accurate and unbiased across all manipulation amounts; if it is fooled by the shifted audio, the system can score well on that penalty while still keeping pitch information in the latent code.

Editorial extensions

If this is right

  • Pitch-correction systems could be trained from ordinary in-tune singing corpora, without needing matched out-of-tune recordings.
  • Extreme semitone shifts become trainable because the cycle-consistency construction exposes the model to them during training.
  • The same adversarial-hiding plus cycle-consistency recipe could apply to features other than pitch, such as speaker identity or emotional style.
  • Replacing source-filter assumptions with a learned pitch-independent code may reduce the metallic artifacts that classical resynthesis introduces.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the accuracy of the pitch estimator used in the leak loss is itself an assumption; if that estimator confuses timbre with pitch, the latent may still leak pitch and the reported F0 improvements could partly reflect estimator behavior. A direct test is to swap in a different pitch tracker at evaluation and see whether the objective F0 gains persist.
  • Editorial inference: the composition cycle-consistency loss effectively asserts that a one-step manipulation to a target contour equals a two-step manipulation through an intermediate contour, so the learned operator should approximately satisfy that composition law; one could measure how far it deviates on random intermediate contours.
  • Editorial inference: because the ablation shows each training term is necessary, the design is likely sensitive to the relative loss weights; a lighter-weight variant could replace the four discriminators with a single time-frequency discriminator to test how much of the quality gain comes from the representation learning versus the adversarial audio training.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Neurodyne, a GAN-based singing-voice pitch-manipulation system. Instead of relying on source-filter disentanglement, it learns a pitch-independent latent representation through adversarial representation learning with a pitch predictor, and it trains with copy-synthesis, inversion cycle-consistency, and composition cycle-consistency to create paired in- and out-of-tune data implicitly. The model is evaluated on global-key and template-based pitch manipulation against DSP baselines and neural baselines, with objective F0 metrics, predicted MOS and similarity scores, MUSHRA listening tests, ablations, and a demo-page comparison with commercial plugins. The paper claims improved synthesis quality, singer similarity, and pitch accuracy over the baselines.

Significance. If the claims hold, the main contribution is a demonstration that high-fidelity neural pitch manipulation can be trained without explicit paired out-of-tune data and without source-filter independence assumptions. The 580-hour multilingual training setup, the cycle-consistency formulations, and the ablation study separating the contributions of copy-synthesis, inversion cycle-consistency, composition cycle-consistency, and adversarial representation learning are all valuable. The paper also provides a detailed architecture description, reproducible training details, and a public demo page, which strengthen the contribution. However, the significance is moderated by two evaluation issues: the F0 estimator used for training and evaluation is not independently validated on pitch-shifted audio, and several key subjective comparisons are not shown to be statistically significant.

major comments (3)
  1. [Section 3.1, Section 3.2, Eq. (1)] The F0 estimator TorchFCPE is load-bearing in three places: extracting F0 targets in preprocessing (Section 3.1), defining the predicted pitch vector used in the leak loss and pitch-predictor loss (Section 2.2, Eq. 1), and supplying the F0 measurements behind the reported F0RMSE/FPC results (Section 3.2, via Amphion). The paper does not validate TorchFCPE on pitch-shifted or vocoder-processed singing, and it reports no cross-check with an independent F0 tracker. If TorchFCPE is biased on shifted audio, the leak loss could be minimized by making the predictor's output low-variance even when the latent still carries pitch information, and the F0RMSE improvements in Tables 1 and 3 could partly reflect matching the biased estimator rather than the true F0. Please add an independent pitch-tracker evaluation (e.g., CREPE, YIN, or pYIN) on the same manipulated audio, report F0RMSE broken down by shift magnitude, and state explicitly which F0 estimator Amphion uses for the reported metrics.
  2. [Table 3 and Section 3.4] The abstract and Section 3.4 claim that Neurodyne outperforms the baselines in subjective quality and singer similarity, but the template-based results in Table 3 do not support a statistically significant advantage over the closest baseline: Q-MUSHRA is 78.3±2 for Neurodyne versus 78.2±2 for PC-NSF, and S-MUSHRA is 78.8±2 versus 78.7±2. These differences are well within the reported 95% confidence intervals, and no pairwise significance test is provided. Several global-key cells in Table 1 also overlap with the best baseline (e.g., S-MUSHRA at 0 semitones is 90.5±3 for Neurodyne versus 90.2±3 for SiFi-GAN and 91.8±2 for TD-PSOLA). Please report paired significance tests (e.g., Wilcoxon signed-rank) for the MUSHRA data, or temper the 'outperforms' claim to reflect the cells where the differences are actually significant.
  3. [Section 3.5, Fig. 3] The ablation study in Fig. 3 presents point estimates for F0RMSE, MOS-Pred, and SIM-O without confidence intervals, error bars, or statistical tests. Since the paper uses the large F0RMSE values for the w/o ICC, w/o CCC, and w/o ARL variants to justify the necessity of each component, the ablation would be more convincing with repeated-run variance or a statistical comparison. In addition, the x-axis label 'auto' is not defined in the caption or text; please clarify what it denotes and how that setting differs from the fixed semitone shifts.
minor comments (5)
  1. [Section 2.1 vs. Eq. (1)] Please clarify the relationship between TorchFCPE and CREPE in the pitch predictor: Section 2.1 says the predictor follows TorchFCPE, while Eq. (1) says the predicted pitch vector follows CREPE. If FCPE uses a CREPE-style pitch-vector output, state this explicitly.
  2. [Section 3.2] The sentence 'A total of 35 and 10 utterances will be assessed in each setting individually' is ambiguous; please state explicitly which setting used 35 utterances and which used 10.
  3. [Section 3.1 and Table 1] Please define the acronym PC-NSF when it is introduced, and clarify that the Ground Truth and BigVGAN rows in Table 1 serve as copy-synthesis references rather than pitch-manipulation systems.
  4. [Section 2.2, Eq. (1)] The leak loss notation Lleak(ŷ) = n2/(nbins−1) bins Var(ŷ) is not self-contained; please define n2 and specify the dimension over which the variance is computed.
  5. [Section 3.4] The comparison with commercial plugins is described only via the demo page; to be part of the scientific evaluation, please describe the sample selection, listening protocol, and scoring method, or move that material to a clearly labeled informal supplement.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Neurodyne's claims rest on external benchmarks and self-supervised losses, not on fitted constants, self-cited uniqueness theorems, or definitions that reduce to their own inputs.

full rationale

Neurodyne's central claims are supported by controlled comparisons against external baselines (WORLD, TD-PSOLA, SiFi-GAN, PC-NSF, DiffPitcher) using objective and MUSHRA metrics. The training objectives in Eqs. (1)-(4) are self-supervised: the adversarial representation learning loss is an explicit minimax game between the encoder and the pitch predictor (Eq. 1), and the cycle-consistency terms (Eqs. 3-4) enforce inversion and composition behavior without using paired ground-truth data. None of these equations is defined in terms of the reported evaluation metrics; the leak loss optimizes the variance of a learned predictor's output, not the F0RMSE of the final audio, and the cycle losses are consistency constraints, not fitted parameters renamed as predictions. The paper's self-citations (refs. 19, 21, 26, 27) provide discriminators and datasets used as components; none carries the load of the main claim or forbids alternatives via a uniqueness theorem. The remaining concern—that TorchFCPE supplies training F0 labels and may also align with the F0RMSE evaluation—is an evaluation-alignment risk, not a circular construction, because the paper does not state that the Amphion F0RMSE uses TorchFCPE and the training loss does not directly minimize that metric. The derivation chain is therefore self-contained against external evidence.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central result rests on the empirical validity of the F0 estimator, the convergence of the HFC adversarial game, the non-triviality of the cycle-consistency solutions, and the perceptual validity of the evaluation metrics. None of these are derived in the paper; they are standard domain assumptions for this kind of neural audio system.

free parameters (2)
  • Loss weight for multi-scale mel-spectrogram loss = 15
    Used in copy-synthesis, inversion, and composition losses (Eqs. 2, 3, 4); chosen by hand with no sensitivity study reported.
  • Loss weight for feature matching loss = 2
    Used in copy-synthesis and inversion losses (Eq. 2, 3); chosen by hand following HiFiGAN, with no sensitivity analysis.
assumptions (4)
  • domain assumption TorchFCPE provides sufficiently accurate F0 labels for all training targets and evaluation metrics.
    Section 3.1 uses TorchFCPE for F0 extraction; Eq. 1 uses the same F0 prediction in the leak loss; if this estimator is biased, both training and F0RMSE are affected.
  • domain assumption The HFC adversarial game can learn to strip pitch information from the latent representation without removing singer identity or content.
    Section 2.2 follows HFC [13], assuming the game between encoder and pitch predictor converges to a pitch-independent code.
  • domain assumption Cycle-consistency objectives have no trivial satisfying solution in which the decoder ignores the target pitch contour.
    Sections 2.3.2 and 2.3.3 assume that inversion and composition losses force the decoder to use the conditioning F0; the ablation in Fig. 3 supports this empirically but it is not proven.
  • domain assumption The MUSHRA listeners and SSL-based similarity metrics reflect perceptual quality and singer identity.
    Section 3.2 relies on 15 volunteer listeners and pretrained models as proxies for human judgment.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Neurodyne: Neural Pitch Manipulation with Representation Learning and Cycle-Consistency GAN." pith.science (2026). https://pith.science/paper/HR5PFGTS

@misc{pith2026250515368,
  author       = {Pith},
  title        = {Pith review of: Neurodyne: Neural Pitch Manipulation with Representation Learning and Cycle-Consistency GAN},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HR5PFGTS}},
  note         = {Machine review of arXiv:2505.15368}
}
read the original abstract

Pitch manipulation is the process of producers adjusting the pitch of an audio segment to a specific key and intonation, which is essential in music production. Neural-network-based pitch-manipulation systems have been popular in recent years due to their superior synthesis quality compared to classical DSP methods. However, their performance is still limited due to their inaccurate feature disentanglement using source-filter models and the lack of paired in- and out-of-tune training data. This work proposes Neurodyne to address these issues. Specifically, Neurodyne uses adversarial representation learning to learn a pitch-independent latent representation to avoid inaccurate disentanglement and cycle-consistency training to create paired training data implicitly. Experimental results on global-key and template-based pitch manipulation demonstrate the effectiveness of the proposed system, marking improved synthesis quality while maintaining the original singer identity.

Figures

Figures reproduced from arXiv: 2505.15368 by the authors.

Figure 1
Figure 1. Architecture and training schemes for Neurodyne. The model consists of an encoder, a pitch predictor, a decoder, a multi￾scale oscillator, and four different discriminators. The training scheme adopts inversion and composition cycle-consistency training to construct paired in- and out-of-tune data implicitly. 2.2. Adversarial Representation Learning We apply adversarial representation learning to obtain a pitch￾inde… view at source ↗
Figure 2
Figure 2. The pitch manipulation strategy. 2.4. Pitch Manipulation Strategy As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The evaluation results of the ablation study. “CS” means copy-synthesis, “ICC” means inversion cycle￾consistency, “CCC” means composition cycle-consistency, and “ARL” means adversarial representation learning. To further illustrate the effectiveness of our proposed sys￾tem, we also compared Neurodyne with two SOTA commercial softwares. Specifically, we manually selected 4 Chinese and 4 English representative paired … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 49 canonical work pages

  1. [1]

    However, the performance of existing pitch-manipulation systems is still limited, which will generate unnatural modified audio with au- dible artifacts

    Introduction Pitch manipulation is an essential process in music production where the producer can adjust the pitch of an audio segment to correct out-of-tune notes and improve the intonation. However, the performance of existing pitch-manipulation systems is still limited, which will generate unnatural modified audio with au- dible artifacts. Thus, build...

    work page

  2. [2]

    Methodology This section details the model architecture, training scheme, and pitch-manipulation strategies of Neurodyne. 2.1. Model Architecture As shown in Fig. 1a, Neurodyne consists of an encoder, a pitch predictor, a decoder, a multi-scale oscillator, and four different discriminators. The encoder includes an initial Conv1D layer and five CNN-based R...

    work page Pith review arXiv 2025

  3. [3]

    CS” means copy-synthesis, “ICC

    Experiments We evaluate the effectiveness of our proposed system in two settings. We apply global-key [7] and template-based [4] pitch manipulation to compare the robustness and evaluate the ap- plication in real-world scenarios. In global-key pitch manip- ulation, utterances will be globally manipulated by a specific amount of semitones. In template-base...

    work page

  4. [4]

    It obtains optimized pitch- independent features that avoid the artifacts brought by source- filter-model-based disentanglement

    Conclusion This paper introduces Neurodyne, a novel pitch manipula- tion system that utilizes adversarial representation learning and cycle-consistency training. It obtains optimized pitch- independent features that avoid the artifacts brought by source- filter-model-based disentanglement. It implicitly creates paired in- and out-of-tune training data to ...

    work page

  5. [5]

    Acknowledgement We acknowledge the computational resources provided by the Aalto Science-IT project. We acknowledge the EuroHPC Joint Undertaking for awarding this project access to the Eu- roHPC supercomputer LUMI, hosted by CSC (Finland) and the LUMI consortium through a EuroHPC Regular Access call. This work is also supported by the 2023 Shenzhen stabi...

    work page 2023

  6. [6]

    Diphone synthesis using an overlap-add technique for speech waveforms concatenation,

    Francis Charpentier and M. G. Stella, “Diphone synthesis using an overlap-add technique for speech waveforms concatenation,” in ICASSP, 1986

    work page 1986

  7. [7]

    WORLD: a vocoder-based high-quality speech synthesis system for real-time applications,

    Masanori Morise, et al., “WORLD: a vocoder-based high-quality speech synthesis system for real-time applications,” IEICE Trans Inf Syst, vol. 99, no. 7, pp. 1877–1884, 2016

    work page 2016

  8. [8]

    Learning the Beauty in Songs: Neural Singing V oice Beautifier,

    Jinglin Liu, et al., “Learning the Beauty in Songs: Neural Singing V oice Beautifier,” inACL, 2022

    work page 2022

Show all 54 references
  1. [9]

    Diff-Pitcher: Diffusion-Based Singing V oice Pitch Correction,

    Jiarui Hai and Mounya Elhilali, “Diff-Pitcher: Diffusion-Based Singing V oice Pitch Correction,” inWASPAA, 2023

    2023

  2. [10]

    Deep Autotuner: A Pitch Correcting Net- work for Singing Performances,

    Sanna Wager, et al., “Deep Autotuner: A Pitch Correcting Net- work for Singing Performances,” in ICASSP, 2020

    2020

  3. [11]

    KaraTuner: Towards End-to-End Natural Pitch Correction for Singing V oice in Karaoke,

    Xiaobin Zhuang, et al., “KaraTuner: Towards End-to-End Natural Pitch Correction for Singing V oice in Karaoke,” in Interspeech, 2022

    2022

  4. [12]

    Source-Filter HiFi-GAN: Fast and Pitch Controllable High-Fidelity Neural V ocoder,

    Reo Yoneyama, et al., “Source-Filter HiFi-GAN: Fast and Pitch Controllable High-Fidelity Neural V ocoder,” inICASSP, 2023

    2023

  5. [13]

    Harmonic-Net: Fundamental Fre- quency and Speech Rate Controllable Fast Neural V ocoder,

    Keisuke Matsubara, et al., “Harmonic-Net: Fundamental Fre- quency and Speech Rate Controllable Fast Neural V ocoder,” TASLP, 2023

    2023

  6. [14]

    HiFi-GAN: High-Fidelity Denoising and Dere- verberation Based on Speech Deep Features in Adversarial Net- works,

    Jiaqi Su, et al., “HiFi-GAN: High-Fidelity Denoising and Dere- verberation Based on Speech Deep Features in Adversarial Net- works,” in Interspeech, 2020

    2020

  7. [15]

    BigVGAN: A Universal Neural V ocoder with Large-Scale Training,

    Sang-gil Lee, et al., “BigVGAN: A Universal Neural V ocoder with Large-Scale Training,” in ICLR, 2023

    2023

  8. [16]

    PeriodGrad: Towards Pitch-Controllable Neural V ocoder Based on a Diffusion Probabilistic Model,

    Yukiya Hono, et al., “PeriodGrad: Towards Pitch-Controllable Neural V ocoder Based on a Diffusion Probabilistic Model,” in ICASSP, 2024

    2024

  9. [17]

    FIRNet: Fundamental Frequency Con- trollable Fast Neural V ocoder With Trainable Finite Impulse Re- sponse Filter,

    Yamato Ohtani, et al., “FIRNet: Fundamental Frequency Con- trollable Fast Neural V ocoder With Trainable Finite Impulse Re- sponse Filter,” in ICASSP, 2024

    2024

  10. [18]

    Hider-Finder-Combiner: An Adversarial Architecture for General Speech Signal Modification,

    Jacob J. Webber, et al., “Hider-Finder-Combiner: An Adversarial Architecture for General Speech Signal Modification,” in Inter- speech, 2020

    2020

  11. [19]

    NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models,

    Zeqian Ju, et al., “NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models,” in ICML, 2024

    2024

  12. [20]

    Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks,

    Jun-Yan Zhu, et al., “Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks,” in ICCV, 2017

    2017

  13. [21]

    High-Fidelity Audio Compression with Improved RVQGAN,

    Rithesh Kumar, et al., “High-Fidelity Audio Compression with Improved RVQGAN,” in NeurIPS, 2023

    2023

  14. [22]

    Conformer: Convolution-augmented Trans- former for Speech Recognition,

    Anmol Gulati, et al., “Conformer: Convolution-augmented Trans- former for Speech Recognition,” in Interspeech, 2020

    2020

  15. [23]

    Neural Source-filter-based Waveform Model for Statistical Parametric Speech Synthesis,

    Xin Wang, et al., “Neural Source-filter-based Waveform Model for Statistical Parametric Speech Synthesis,” in ICASSP, 2019

    2019

  16. [24]

    An Investigation of Time-Frequency Rep- resentation Discriminators for High-Fidelity V ocoders,

    Yicheng Gu, et al., “An Investigation of Time-Frequency Rep- resentation Discriminators for High-Fidelity V ocoders,” TASLP, 2024

    2024

  17. [25]

    High Fidelity Neural Audio Com- pression,

    Alexandre D ´efossez, et al., “High Fidelity Neural Audio Com- pression,” Trans. Mach. Learn. Res., 2023

    2023

  18. [26]

    Multi-scale sub-band constant-q transform discriminator for high-fidelity vocoder,

    Yicheng Gu, et al., “Multi-scale sub-band constant-q transform discriminator for high-fidelity vocoder,” in ICASSP, 2024

    2024

  19. [27]

    Crepe:A Convolutional Representation for Pitch Estimation,

    Jong Wook Kim, et al., “Crepe:A Convolutional Representation for Pitch Estimation,” in ICASSP, 2018

    2018

  20. [28]

    CREPE Notes: A new method for segmenting pitch contours into discrete notes,

    Xavier Riley and Simon Dixon, “CREPE Notes: A new method for segmenting pitch contours into discrete notes,” arXiv:2311.08884, 2023

    2023 arXiv

  21. [29]

    Emilia: An Extensive, Multilingual, and Di- verse Speech Dataset for Large-Scale Speech Generation,

    Haorui He, et al., “Emilia: An Extensive, Multilingual, and Di- verse Speech Dataset for Large-Scale Speech Generation,” inSLT, 2024

    2024

  22. [30]

    Emilia: A Large-Scale, Extensive, Multilin- gual, and Diverse Dataset for Speech Generation,

    Haorui He, et al., “Emilia: A Large-Scale, Extensive, Multilin- gual, and Diverse Dataset for Speech Generation,” 2025

    2025

  23. [31]

    SingNet: Towards a Large-Scale, Diverse, and In-the-Wild Singing V oice Dataset,

    Yicheng Gu, et al., “SingNet: Towards a Large-Scale, Diverse, and In-the-Wild Singing V oice Dataset,”OpenReview, 2024

    2024

  24. [32]

    Solid State Bus-Comp: A Large-Scale and Diverse Dataset for Dynamic Range Compressor Virtual Analog Modeling,

    Yicheng Gu, et al., “Solid State Bus-Comp: A Large-Scale and Diverse Dataset for Dynamic Range Compressor Virtual Analog Modeling,” 2025

    2025

  25. [33]

    The NUS sung and spoken lyrics corpus: A quantitative comparison of singing and speech,

    Zhiyan Duan, et al., “The NUS sung and spoken lyrics corpus: A quantitative comparison of singing and speech,” inAPSIPA, 2013

    2013

  26. [34]

    Automatic identification of emotional cues in Chinese opera singing,

    Dawn AA Black, et al., “Automatic identification of emotional cues in Chinese opera singing,” ICMPC, 2014

    2014

  27. [35]

    V ocalSet: A Singing V oice Dataset,

    Julia Wilkins, et al., “V ocalSet: A Singing V oice Dataset,” in ISMIR, 2018

    2018

  28. [36]

    Children’s song dataset for singing voice research,

    Soonbeom Choi, et al., “Children’s song dataset for singing voice research,” in ISMIR, 2020

    2020

  29. [37]

    PJS: phoneme-balanced Japanese singing- voice corpus,

    Junya Koguchi, et al., “PJS: phoneme-balanced Japanese singing- voice corpus,” in APSIPA, 2020

    2020

  30. [38]

    NHSS: A speech and singing parallel database,

    Bidisha Sharma, et al., “NHSS: A speech and singing parallel database,” Speech Commun., 2021

    2021

  31. [39]

    Multi-Singer: Fast Multi-Singer Singing V oice V ocoder With A Large-Scale Corpus,

    Rongjie Huang, et al., “Multi-Singer: Fast Multi-Singer Singing V oice V ocoder With A Large-Scale Corpus,” inACM MM, 2021

    2021

  32. [40]

    Tohoku kiritan singing database: A singing database for statistical parametric singing synthesis using japanese pop songs,

    Itsuki Ogawa and Masanori Morise, “Tohoku kiritan singing database: A singing database for statistical parametric singing synthesis using japanese pop songs,” AST, 2021

    2021

  33. [41]

    Muskits: an End-to-end Music Processing Toolkit for Singing V oice Synthesis,

    Jiatong Shi, et al., “Muskits: an End-to-end Music Processing Toolkit for Singing V oice Synthesis,” inInterspeech, 2022

    2022

  34. [42]

    DiffSinger: Singing V oice Synthesis via Shal- low Diffusion Mechanism,

    Jinglin Liu, et al., “DiffSinger: Singing V oice Synthesis via Shal- low Diffusion Mechanism,” in AAAI, 2022

    2022

  35. [43]

    M4Singer: A Multi-Style, Multi-Singer and Musical Score Provided Mandarin Singing Corpus,

    Lichao Zhang, et al., “M4Singer: A Multi-Style, Multi-Singer and Musical Score Provided Mandarin Singing Corpus,” in NeurIPS, 2022

    2022

  36. [44]

    Opencpop: A High-Quality Open Source Chi- nese Popular Song Corpus for Singing V oice Synthesis,

    Yu Wang, et al., “Opencpop: A High-Quality Open Source Chi- nese Popular Song Corpus for Singing V oice Synthesis,” inInter- speech, 2022

    2022

  37. [45]

    SingStyle111: A Multilingual Singing Dataset With Style Transfer,

    Shuqi Dai, et al., “SingStyle111: A Multilingual Singing Dataset With Style Transfer,” in ISMIR, 2023

    2023

  38. [46]

    FT-GAN: Fine-Grained Tune Modeling for Chinese Opera Synthesis,

    Meizhen Zheng, et al., “FT-GAN: Fine-Grained Tune Modeling for Chinese Opera Synthesis,” in AAAI, 2024

    2024

  39. [47]

    Singing voice data scaling-up: An introduc- tion to ace-opencpop and kising-v2,

    Jiatong Shi, et al., “Singing voice data scaling-up: An introduc- tion to ace-opencpop and kising-v2,” arXiv:2401.17619, 2024

    2024 arXiv

  40. [48]

    Gtsinger: A global multi-technique singing corpus with realistic music scores for all singing tasks,

    Yu Zhang, et al., “Gtsinger: A global multi-technique singing corpus with realistic music scores for all singing tasks,” arXiv:2409.13832, 2024

    2024

  41. [49]

    Decoupled Weight Decay Regularization,

    Ilya Loshchilov and Frank Hutter, “Decoupled Weight Decay Regularization,” in ICLR, 2019

    2019

  42. [50]

    The Singing V oice Conversion Chal- lenge 2023,

    Wen-Chin Huang, et al., “The Singing V oice Conversion Chal- lenge 2023,” vol. arXiv:2306.14422, 2023

    2023 arXiv

  43. [51]

    Amphion: An Open-Source Audio, Music and Speech Generation Toolkit,

    Xueyao Zhang, et al., “Amphion: An Open-Source Audio, Music and Speech Generation Toolkit,” in SLT, 2024

    2024

  44. [52]

    Leveraging Content-based Features from Multiple Acoustic Models for Singing V oice Conversion,

    Xueyao Zhang, et al., “Leveraging Content-based Features from Multiple Acoustic Models for Singing V oice Conversion,”CoRR, vol. abs/2310.11160, 2023

    2023 arXiv

  45. [53]

    MOS-Bench: Benchmarking General- ization Abilities of Subjective Speech Quality Assessment Mod- els,

    Wen-Chin Huang, et al., “MOS-Bench: Benchmarking General- ization Abilities of Subjective Speech Quality Assessment Mod- els,” arXiv:2411.03715, 2024

    2024 arXiv

  46. [54]

    Singer Identity Representation Learning Using Self-Supervised Techniques,

    Bernardo Torres, et al., “Singer Identity Representation Learning Using Self-Supervised Techniques,” in ISMIR, 2023

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.