Pith. sign in

REVIEW 4 major objections 4 minor 36 references

Singing Voice Conversion with Accompaniment Using Self-Supervised Representation-Based Melody Features

T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that replacing explicit pitch trackers with learned, weighted sums of self-supervised speech representations (WavLM/HuBERT) as melody features makes any-to-one singing voice conversion accurate under background music…

desk verdict Useful incremental extension of the authors' own RobustSVC with WavLM and weighted-sum, but the melody metric is partly self-referential and the baselines are not quite fair; worth a serious referee. read the letter →

arxiv 2502.04722 v1 pith:H2524PEZ submitted 2025-02-07 cs.SD cs.LGeess.AS

Wei Chen , Binzhu Sha , Jing Yang , Zhuo Wang , Fan Fan , Zhiyong Wu This is my paper · ORCID
classification cs.SDcs.LGeess.AS
keywords singingvoiceconversionself-supervisedlearningmelodyextractionWavLMHuBERTbackgroundmusicrobustnesspitchtrackingsourceseparation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Singing voice conversion usually assumes the input is a clean vocal track, so background music makes pitch extractors fail and converted vocals go off-key. This paper tries to fix that by taking melody information not from a dedicated pitch tracker but from the internal layers of self-supervised speech models (HuBERT and WavLM), combining all layers with learned weights and running them through transformer blocks. The claimed result is that this melody feature, especially from WavLM, keeps pitch accurate even at 0 dB signal-to-noise ratio: pitch root-mean-square error falls from 0.297 with Crepe to 0.199, and pitch correlation rises from 0.815 to 0.935. If the claim holds, a singing voice converter can work directly on accompanied audio without a source-separation stage and its artifacts, preserving both melody and target-speaker similarity.

What carries the argument

The central object is the SSL-based melody extractor: a pretrained HuBERT or WavLM model whose per-layer hidden states are combined into a weighted sum, $O_t=\sum_{l=0}^{L} w_l h_l^t$, with learnable weights $w_l$; the summed representation is processed by Feed Forward Transformer blocks to predict pitch, energy, and voiced/unvoiced flags, and the final 256-dimensional block output becomes the melody feature fed to the SVC encoder. The mechanism that carries the argument is the combination of weighted-sum layer fusion, a brief end-to-end fine-tuning of the SSL weights (5,000 steps, then frozen to avoid catastrophic forgetting), and training-time mixing of background music into the vocal data. Weight visualizations show that before fine-tuning only shallow layers encode melody, whereas fine-tuning shifts substantial melody information into higher transformer layers, which the authors connect to the large F0RMSE gains from 0.360 (single-layer HuBERT) to 0.176 (proposed WavLM).

What would settle it

Take a set of songs with a known reference melody (for example, synthesized or performed against a precise MIDI score), mix in accompaniment at 0 dB SNR, convert with the proposed WavLM system, and compare the converted pitch to the known notes rather than to a tracker-derived contour; if the proposed method does not keep its advantage over Crepe and Separated+Crepe against this reference, the central claim of superior melody accuracy fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a 256-dimensional melody representation extracted from a fine-tuned self-supervised model can replace an explicit pitch contour as the melodic conditioning in any-to-one singing voice conversion, and that this change is what makes conversion handle accompaniment. The melody extractor computes a learnable weighted sum of all hidden-layer outputs of WavLM (or HuBERT), $O=\sum_{l=0}^{L} w_l h_l^t$, feeds this through Feed Forward Transformer blocks that predict pitch, energy, and voiced/unvoiced flags, and uses the resulting hidden state as the melody feature. The authors train the extractor on vocal tracks mixed with background music, fine-tune the SSL weights for only 5,000 steps, then freeze them. In BGM conditions the proposed system achieves the best F0RMSE/F0CORR at every SNR tested; under clean audio its objective metrics are comparable to the baselines, while its listener-rated naturalness and similarity are higher. At 0 dB the WavLM system reaches F0RMSE 0.199 and F0CORR 0.935. The paper also claims that fine-tuning is what activates melody information in the upper transformer layers: weight visualisations show that before fine-tuning only the CNN and lower encoder layers contribute, while after fine-tuning the higher layers contribute substantially.

Load-bearing premise

The claim depends on the ground-truth melody being the median pitch reported by three software trackers (PYIN, REAPER, Parselmouth) on the clean vocal; if those trackers are systematically wrong for expressive singing or dense accompaniment, the reported pitch-accuracy improvements could partly reflect better matching of the trackers rather than better melody.

Editorial extensions

If this is right

  • An SVC pipeline with this melody extractor can accept a full song with accompaniment as input and output a clean vocal in the target singer's timbre, so users do not need to run source separation first.
  • At 0 dB SNR the claimed melody accuracy (F0RMSE 0.199, F0CORR 0.935) is better than both the neural tracker Crepe and the separated+Crepe pipeline, meaning the gain comes from the representation, not from cleaning the audio.
  • Because the feature carries energy and voiced/unvoiced information, converted audio retains more of the source's expressive timing and articulation than a scalar F0 contour would provide.
  • The extractor generalizes across two different SSL architectures (HuBERT and WavLM), and WavLM's stronger results suggest that the choice of pretrained model matters more than the downstream head design.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not test is whether the same melody feature transfers to any-to-many or multi-singer conversion, since the proposed model is any-to-one; the melody extractor itself is singer-independent and should be portable.
  • Because the objective pitch metric is computed against F0 trackers of the same family used to create training labels, a stronger test would use a synthetic mixture with a known musical ground truth (for example, MIDI-rendered accompaniment and a click-track performance) to see whether the apparent gain reflects true pitch or merely alignment to tracker behavior.
  • The 0 dB condition is the paper's most dramatic improvement; pushing SNR lower or adding nonstationary noise such as applause, speech, or competing vocals would reveal whether the SSL representations eventually hit the same failure floor as conventional trackers.
  • The weight visualizations suggest a testable prediction: a fixed pretrained SSL model with no fine-tuning should have most melody information in shallow layers, so a lightweight probe on those layers may approximate the proposed extractor at lower computational cost.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes an any-to-one singing voice conversion (SVC) system that uses self-supervised learning (SSL) representations, specifically WavLM and HuBERT, to extract melody features that are robust to background music (BGM). The melody extractor combines weighted-summed SSL hidden states with FFT blocks, and is fine-tuned on BGM-mixed data. The proposed system is compared against three baselines (Original Pitch&Energy, Crepe, Separated+Crepe) under clean and noisy (0, 5, 10, 15 dB SNR) conditions, using F0RMSE, F0CORR, and MOS. The central claim is that the proposed method significantly outperforms baselines in melody accuracy and in subjective similarity/naturalness, especially at low SNR.

Significance. If the claimed improvements are substantiated, the paper would offer a practical solution for SVC with accompaniment, which is a real and underexplored problem. The layer-weight analysis (Fig. 3) and the comparison of SSL variants are also useful empirical contributions. However, the current evaluation has several confounds: the objective pitch metrics may favor the proposed extractor due to being trained on the same pitch-estimator family, no significance tests support the 'significantly outperforms' wording, the clean-condition MOS advantage is within overlapping confidence intervals, and the Crepe baseline is not adapted to BGM data. These issues currently prevent the evidence from carrying the strength of the central claim.

major comments (4)
  1. [Section III-B and IV-A] The melody extractor is trained with ground-truth pitch labels defined as the median of PYIN, REAPER, and Parselmouth (Section III-B). The objective evaluation in Section IV-A computes F0RMSE and F0CORR between converted and source audio, but the manuscript does not state which F0 extraction method is used for these metrics. If the same PYIN/REAPER/Parselmouth family is used, the proposed model is being evaluated on how well it reproduces the behavior of the very estimators it was trained to imitate, while the Original Pitch&Energy baseline consumes those same median labels without a learned module trained to reproduce them. Please specify the F0 extraction method used in the evaluation, and if it coincides with the training labels, add an evaluation against an independent pitch reference (e.g., a different pitch tracker or manual annotations) and include a control where Crepe is also fine-tuned on the BGM-mixed training set.
  2. [Table II and Abstract] The abstract and Section V claim that the proposed method 'significantly outperforms' baselines, but no significance tests are reported for F0RMSE or F0CORR, and the MOS 95% confidence intervals on clean audio overlap with the Original Pitch&Energy baseline (NMOS 3.54±0.14 vs 3.38±0.14; SMOS 3.71±0.14 vs 3.58±0.13). At 0 dB SNR, the objective gain over Separated+Crepe is small (F0RMSE 0.199 vs 0.202; F0CORR 0.935 vs 0.928). Please report paired significance tests across test items and listeners, or temper the 'significantly' language to match the actual strength of the evidence.
  3. [Section III-B and IV-B] The proposed melody extractor is trained on BGM-accompanied singing data and its SSL model is fine-tuned, whereas the Crepe baseline is used as a pretrained model without any adaptation to BGM. The observed advantage may therefore be due to the additional BGM training data rather than to the SSL representation per se. To isolate the contribution of SSL-based features, please add a baseline where Crepe is fine-tuned on the same BGM data, or an ablation that uses the same architecture and training data but a non-SSL front-end.
  4. [Section IV-B and Ref [22]] The manuscript describes RobustSVC (Ref. [22]) as the backbone of the proposed framework, but RobustSVC is not included in the comparison in Table II. Since RobustSVC is a directly comparable SVC method for the same noisy-condition setting, its omission makes the 'comparison with state-of-the-art methods' incomplete. Please add RobustSVC as a baseline or provide a clear justification for its exclusion.
minor comments (4)
  1. [Section IV-A] The definition of F0RMSE is confusing: 'The RMSE between the converted and source waveforms is computed in the absence of BGM' should refer to F0 contours, not waveforms, and 'source' needs to be defined (original clean vocal? original mixture?). Also, F0RMSE is an accuracy metric and should not be described as measuring naturalness.
  2. [Table I] The table header 'Method Condition' is odd because the Condition column is empty, and the table does not indicate which SNR condition the results correspond to. Please clarify whether the numbers are averaged over SNR levels and what the checkmarks in the FT/WS/FFT columns denote.
  3. [Section II-B (last paragraph)] The sentence 'we fine-tune the pretrained HiFi-GAN vocoder using ground-truth-aligned [26]' is incomplete and Reference [26] does not describe a method called 'ground-truth-aligned.' Please rewrite to describe the fine-tuning procedure and the reference appropriately.
  4. [Section IV-C and Fig. 3] The y-axis of Fig. 3 is unlabeled, so it is unclear what the plotted weight values represent. Also, the claim that only shallow layers capture melody information in the pretrained model is based on learned weights, which may not fully reflect the information content; consider supporting it with a layer-wise probe or stating the limitation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the melody extractor is trained on held-out pitch labels and the SVC output is evaluated by re-estimating F0 from generated audio, with no equation making the predicted quantity equal to the training labels by construction.

full rationale

The paper's derivation chain is not circular. The melody extractor (Section III-B) is trained to predict pitch/energy/VUV labels, where 'ground-truth pitches are extracted from clean audio by taking the median of PYIN, REAPER, and Parselmouth.' The F0RMSE/F0CORR metrics (Section IV-A) are computed on converted audio by 'aligning F0 contours of the original and synthesized audio using dynamic time warping and then computing the Pearson correlation coefficient.' The proposed model does not output these labels directly: it produces 256-dimensional melody features that are fed into an encoder-decoder SVC framework (Section II), and the metrics are re-estimated from the synthesized waveform. The held-out test set, containing unseen BGM at SNR 0-15 dB, means the reported improvements are not a fitted-value replay. The only self-citation is [22], which supplies the backbone Recognition-synthesis architecture; that earlier work is not invoked as a uniqueness theorem or as a substitute for the melody-extraction experiments, and the central comparison (Tables I-II) is against external baselines under identical conditions. A potential risk is that the training labels and the metric reference both derive from the same automatic pitch-estimator family, so the metric may partly reward tracker affinity; however, the paper does not state that F0RMSE/F0CORR use the median PYIN/REAPER/Parselmouth estimator, and no equation reduces the predicted quantity to the training label by construction. Under the standing rules, that is a correctness robustness concern rather than demonstrated circularity, so no circular step is flagged.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method introduces a learned weighted-sum over SSL layers as a free parameter set, plus domain assumptions about the ground-truth pitch labels and mixture realism. No new physical or conceptual entities are postulated.

free parameters (3)
  • SSL layer weights w_l = learned per layer, not reported
    The weighted-sum weights in Eq. (1) are learned and central to the melody feature output; their learned values are not listed.
  • Fine-tuning steps for SSL model = 5000
    Chosen by hand to mitigate catastrophic forgetting; not justified by a sweep.
  • BGM mixing probability = 0.5
    Chosen for training data augmentation; not swept in experiments.
assumptions (4)
  • domain assumption SSL models pretrained on Librispeech speech contain transferable melody-relevant information
    Relied on in Section II-B; the extractor fine-tunes these models for only 5k steps.
  • domain assumption Median of PYIN, REAPER, and Parselmouth provides ground-truth pitch
    Section III-B defines training labels and evaluation references from these estimators.
  • domain assumption F0RMSE and F0CORR reflect melody preservation
    Section IV-A uses these metrics as primary objective evidence; no human-validated pitch ground truth is used.
  • domain assumption Adding MUSDB background music to clean vocals at random SNRs approximates real accompaniment
    Training and test use artificial mixes, not naturally recorded songs with BGM.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Singing Voice Conversion with Accompaniment Using Self-Supervised Representation-Based Melody Features." pith.science (2026). https://pith.science/paper/H2524PEZ

@misc{pith2026250204722,
  author       = {Pith},
  title        = {Pith review of: Singing Voice Conversion with Accompaniment Using Self-Supervised Representation-Based Melody Features},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H2524PEZ}},
  note         = {Machine review of arXiv:2502.04722}
}
read the original abstract

Melody preservation is crucial in singing voice conversion (SVC). However, in many scenarios, audio is often accompanied with background music (BGM), which can cause audio distortion and interfere with the extraction of melody and other key features, significantly degrading SVC performance. Previous methods have attempted to address this by using more robust neural network-based melody extractors, but their performance drops sharply in the presence of complex accompaniment. Other approaches involve performing source separation before conversion, but this often introduces noticeable artifacts, leading to a significant drop in conversion quality and increasing the user's operational costs. To address these issues, we introduce a novel SVC method that uses self-supervised representation-based melody features to improve melody modeling accuracy in the presence of BGM. In our experiments, we compare the effectiveness of different self-supervised learning (SSL) models for melody extraction and explore for the first time how SSL benefits the task of melody extraction. The experimental results demonstrate that our proposed SVC model significantly outperforms existing baseline methods in terms of melody accuracy and shows higher similarity and naturalness in both subjective and objective evaluations across noisy and clean audio environments.

Figures

Figures reproduced from arXiv: 2502.04722 by the authors.

Figure 2
Figure 2. Melody extractor. both constructed with Feed Forward Transformer (FFT) [23] blocks. Notably, the melody encoder applies conditional in￾stance normalization [24] after the FFT blocks to remove the timbre information of the source singer. The decoder, based on FFT blocks, concatenates and decodes the input melody embeddings and content embeddings to generate Mel￾spectrograms. Additionally, the model uses adversarial l… view at source ↗
Figure 3
Figure 3. Visualized weight that extract representations from SSL models. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 29 canonical work pages

  1. [22]

    RobustSVC: HuBERT-based Melody Extractor and Adversarial Learning for Robust Singing Voice Conversion

    W. Chen, X. Zhao, J. Chen, B. Sha, Z. Lin, and Z. Wu, “Robustsvc: Hubert-based melody extractor and adversarial learning for robust singing voice conversion,” 2024. [Online]. Available: https://arxiv.org/abs/2409.06237

    work page Pith review arXiv 2024

  2. [1]

    Statis- tical singing voice conversion with direct waveform modification based on the spectrum differential,

    K. Kobayashi, T. Toda, G. Neubig, S. Sakti, and S. Nakamura, “Statis- tical singing voice conversion with direct waveform modification based on the spectrum differential,” in Fifteenth Annual Conference of the International Speech Communication Association , 2014

    work page 2014

  3. [2]

    Applying voice conversion to concate- native singing-voice synthesis,

    F. Villavicencio and J. Bonada, “Applying voice conversion to concate- native singing-voice synthesis,” in Eleventh annual conference of the international speech communication association , 2010

    work page 2010

  4. [3]

    Statisti- cal singing voice conversion based on direct waveform modification with global variance,

    K. Kobayashi, T. Toda, G. Neubig, S. Sakti, and S. Nakamura, “Statisti- cal singing voice conversion based on direct waveform modification with global variance,” in Sixteenth Annual Conference of the International Speech Communication Association . Citeseer, 2015

    work page 2015

  5. [4]

    One-to-many and many-to-one voice conversion based on eigenvoices,

    T. Toda, Y . Ohtani, and K. Shikano, “One-to-many and many-to-one voice conversion based on eigenvoices,” in 2007 IEEE International Conference on Acoustics, Speech and Signal Processing-ICASSP’07 , vol. 4. IEEE, 2007, pp. IV–1249

    work page 2007

  6. [5]

    Ppg-based singing voice conversion with adversarial representation learning,

    Z. Li, B. Tang, X. Yin, Y . Wan, L. Xu, C. Shen, and Z. Ma, “Ppg-based singing voice conversion with adversarial representation learning,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2021, pp. 7073–7077

    work page 2021

  7. [6]

    Phonetic pos- teriorgrams for many-to-one voice conversion without parallel data training,

    L. Sun, K. Li, H. Wang, S. Kang, and H. Meng, “Phonetic pos- teriorgrams for many-to-one voice conversion without parallel data training,” in 2016 IEEE International Conference on Multimedia and Expo (ICME) . IEEE, 2016, pp. 1–6

    work page 2016

  8. [7]

    Vits-based singing voice conversion leveraging whisper and multi-scale f0 modeling,

    Z. Ning, Y . Jiang, Z. Wang, B. Zhang, and L. Xie, “Vits-based singing voice conversion leveraging whisper and multi-scale f0 modeling,” in 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2023, pp. 1–8

    work page 2023

Show all 36 references
  1. [8]

    Self- supervised representations for singing voice conversion,

    T. Jayashankar, J. Wu, L. Sari, D. Kant, V . Manohar, and Q. He, “Self- supervised representations for singing voice conversion,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2023, pp. 1–5

    2023

  2. [9]

    Durian-sc: Duration informed attention network based singing voice conversion system,

    L. Zhang, C. Yu, H. Lu, C. Weng, C. Zhang, Y . Wu, X. Xie, Z. Li, and D. Yu, “Durian-sc: Duration informed attention network based singing voice conversion system,” arXiv preprint arXiv:2008.03009 , 2020

    2008 arXiv

  3. [10]

    Hierarchical disentangled representation learning for singing voice conversion,

    N. Takahashi, M. K. Singh, and Y . Mitsufuji, “Hierarchical disentangled representation learning for singing voice conversion,” in 2021 Interna- tional Joint Conference on Neural Networks (IJCNN) . IEEE, 2021, pp. 1–7

    2021

  4. [11]

    Fastsvc: Fast cross-domain singing voice conversion with feature-wise linear modulation,

    S. Liu, Y . Cao, N. Hu, D. Su, and H. Meng, “Fastsvc: Fast cross-domain singing voice conversion with feature-wise linear modulation,” in 2021 ieee international conference on multimedia and expo (icme) . IEEE, 2021, pp. 1–6

    2021

  5. [12]

    Singing voice conversion with disentangled representations of singer and vocal tech- nique using variational autoencoders,

    Y .-J. Luo, C.-C. Hsu, K. Agres, and D. Herremans, “Singing voice conversion with disentangled representations of singer and vocal tech- nique using variational autoencoders,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)...

    2020

  6. [13]

    Diffsvc: A diffusion probabilistic model for singing voice conversion,

    S. Liu, Y . Cao, D. Su, and H. Meng, “Diffsvc: A diffusion probabilistic model for singing voice conversion,” in 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) . IEEE, 2021, pp. 741–748

    2021

  7. [14]

    pyin: A fundamental frequency estimator using probabilistic threshold distributions,

    M. Mauch and S. Dixon, “pyin: A fundamental frequency estimator using probabilistic threshold distributions,” in 2014 ieee international conference on acoustics, speech and signal processing (icassp) . IEEE, 2014, pp. 659–663

    2014

  8. [15]

    Hifi-svc: Fast high fidelity cross-domain singing voice conversion,

    Y . Zhou and X. Lu, “Hifi-svc: Fast high fidelity cross-domain singing voice conversion,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2022, pp. 6667–6671

    2022

  9. [16]

    Crepe: A convolutional representation for pitch estimation,

    J. W. Kim, J. Salamon, P. Li, and J. P. Bello, “Crepe: A convolutional representation for pitch estimation,” in 2018 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2018, pp. 161–165

    2018

  10. [17]

    Towards high-fidelity singing voice conversion with acoustic reference and con- trastive predictive coding,

    C. Wang, Z. Li, B. Tang, X. Yin, Y . Wan, Y . Yu, and Z. Ma, “Towards high-fidelity singing voice conversion with acoustic reference and con- trastive predictive coding,” arXiv preprint arXiv:2110.04754 , 2021

    2021 arXiv

  11. [18]

    Adversarial speaker disentanglement using unannotated external data for self-supervised representation-based voice conversion,

    X. Zhao, S. Wang, Y . Chao, Z. Wu, and H. Meng, “Adversarial speaker disentanglement using unannotated external data for self-supervised representation-based voice conversion,” in 2023 IEEE International Conference on Multimedia and Expo (ICME) . Los Alamitos, CA, USA: IEEE Co...

    2023

  12. [19]

    Boosting self-supervised embeddings for speech enhancement,

    K.-H. Hung, S. wei Fu, H.-H. Tseng, H.-T. Chiang, Y . Tsao, and C.-W. Lin, “Boosting self-supervised embeddings for speech enhancement,” in Interspeech 2022 , 2022, pp. 186–190

    2022

  13. [20]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM transactions on audio, speech, and language processing , vol. 29, pp. 3451–3460, 2021

    2021

  14. [21]

    Wavlm: Large-scale self-supervised pre- training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao et al. , “Wavlm: Large-scale self-supervised pre- training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  15. [23]

    Fastspeech 2: Fast and high-quality end-to-end text to speech,

    Y . Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y . Liu, “Fastspeech 2: Fast and high-quality end-to-end text to speech,” in International Conference on Learning Representations

  16. [24]

    A learned representation for artistic style,

    V . Dumoulin, J. Shlens, and M. Kudlur, “A learned representation for artistic style,” arXiv preprint arXiv:1610.07629 , 2016

    2016 arXiv

  17. [25]

    Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,

    J. Kong, J. Kim, and J. Bae, “Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,” Advances in Neural Information Processing Systems , vol. 33, pp. 17 022–17 033, 2020

    2020

  18. [26]

    Natural tts synthesis by conditioning wavenet on mel spectrogram predictions,

    J. Shen, R. Pang, R. J. Weiss, M. Schuster, N. Jaitly, Z. Yang, Z. Chen, Y . Zhang, Y . Wang, R. Skerrv-Ryan et al. , “Natural tts synthesis by conditioning wavenet on mel spectrogram predictions,” in 2018 IEEE international conference on acoustics, speech and signal processin...

    2018

  19. [27]

    Why does self-supervised learning for speech recognition benefit speaker recognition?

    S. Chen, Y . Wu, C. Wang, S. Liu, Z. Chen, P. Wang, G. Liu, J. Li, J. Wu, X. Yu, and F. Wei, “Why does self-supervised learning for speech recognition benefit speaker recognition?” in Interspeech 2022, 2022, pp. 3699–3703

    2022

  20. [28]

    Musdb18-a corpus for music separation,

    Z. Rafii, A. Liutkus, F.-R. St ¨oter, S. I. Mimilakis, and R. Bittner, “Musdb18-a corpus for music separation,” 2017

    2017

  21. [29]

    Multi-singer: Fast multi-singer singing voice vocoder with a large-scale corpus,

    R. Huang, F. Chen, Y . Ren, J. Liu, C. Cui, and Z. Zhao, “Multi-singer: Fast multi-singer singing voice vocoder with a large-scale corpus,” in Proceedings of the 29th ACM International Conference on Multimedia , 2021, pp. 3945–3954

    2021

  22. [30]

    Opencpop: A high-quality open source chinese popular song corpus for singing voice synthesis,

    Y . Wang, X. Wang, P. Zhu, J. Wu, H. Li, H. Xue, Y . Zhang, L. Xie, and M. Bi, “Opencpop: A high-quality open source chinese popular song corpus for singing voice synthesis,” arXiv preprint arXiv:2201.07429 , 2022

    2022 arXiv

  23. [31]

    Librispeech: an asr corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: an asr corpus based on public domain audio books,” in2015 IEEE international conference on acoustics, speech and signal processing (ICASSP) . IEEE, 2015, pp. 5206–5210

    2015

  24. [32]

    Catastrophic forgetting in connectionist networks,

    R. M. French, “Catastrophic forgetting in connectionist networks,” Trends in cognitive sciences , vol. 3, no. 4, pp. 128–135, 1999

    1999

  25. [33]

    Dis- entangling content and fine-grained prosody information via hybrid asr bottleneck features for voice conversion,

    X. Zhao, F. Liu, C. Song, Z. Wu, S. Kang, D. Tuo, and H. Meng, “Dis- entangling content and fine-grained prosody information via hybrid asr bottleneck features for voice conversion,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (...

    2022

  26. [34]

    Wenet: Production oriented streaming and non- streaming end-to-end speech recognition toolkit,

    Z. Yao, X. W. Di Wu, B. Zhang, F. Yu, C. Yang, Z. Peng, X. Chen, L. Xie, and X. Lei, “Wenet: Production oriented streaming and non- streaming end-to-end speech recognition toolkit,” 2021

    2021

  27. [35]

    Hybrid transformers for music source separation,

    S. Rouard, F. Massa, and A. D ´efossez, “Hybrid transformers for music source separation,” in ICASSP 2023-2023 IEEE International Confer- ence on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2023, pp. 1–5

    2023

  28. [36]

    The singing voice conversion challenge 2023,

    W.-C. Huang, L. P. Violeta, S. Liu, J. Shi, and T. Toda, “The singing voice conversion challenge 2023,” in 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) . IEEE, 2023, pp. 1–8

    2023

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.