Pith. sign in

REVIEW 4 major objections 5 minor 21 references

Music2Fail: Transfer Music to Failed Recorder Style

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Failed recorder playing is a learnable AI music style

desk verdict New failed-recorder dataset and task framing are the real contributions; the claimed VAE-GAN superiority is provisional given single-performer target and weak significance. read the letter →

arxiv 2411.18075 v1 pith:J6TBACRA submitted 2024-11-27 cs.SD eess.AS

Chon In Leong , I-Ling Chung , Kin-Fong Chao , Jun-You Wang , Yi-Hsuan Yang , Jyh-Shing Roger Jang This is my paper · ORCID
classification cs.SDeess.AS
keywords failedmusicstyletransferFR109datasetrecorderoff-pitchperformanceVAE-GANStarGANDDSPWienerentropy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that deliberately failed recorder performance, which most music style transfer work ignores, can be a legitimate target domain for style transfer. The authors introduce FR109, a dataset of 109 pieces of intentionally off-pitch recorder playing by a professional, and use it to convert clean violin, clarinet, and saxophone recordings into audio that listeners still recognize as a flawed recorder. They find that VAE-GAN outperforms StarGAN and DDSP on this task, with the best objective and subjective scores. The paper argues that learning the inharmonic, noise-heavy character of failed playing is a meaningful stress test for style transfer models that normally assume both source and target are well played.

What carries the argument

The central object is the FR109 dataset: 109 pieces, 5.05 hours of soprano recorder performed by a professional with deliberate errors such as cracked voice, weird dynamics, failed tonguing, overblowing, and underblowing. The transfer pipeline converts source audio to Mel spectrograms, passes them through a style-transfer generator (StarGAN or VAE-GAN), and reconstructs the waveform with the BigVSAN vocoder. The paper quantifies the style signal using Wiener entropy, which measures how noise-like a signal is, and uses that metric to show that failed recorder is clearly noisier than well-played instruments and that current models still fall short of reproducing the full noise level.

What would settle it

Record a second failed-recorder dataset from several different performers, retrain VAE-GAN on FR109 as before, and evaluate on this new set; if FAD jumps substantially or listeners no longer identify the output as a recorder, the style is not general, whereas comparable scores would confirm the style is a stable target.

Watch

Extended reading notes

Core claim

The central claim is that failed-music style transfer is feasible: a model trained on the FR109 failed-recorder dataset can turn well-played source audio into audio that sounds like an off-pitch recorder while preserving the melody, and VAE-GAN does this better than StarGAN or DDSP on the Bach10 test set. The paper reports FAD 7.27, MOS style similarity 2.98, melody similarity 3.56, and sound quality 3.00 for VAE-GAN, and it uses Wiener entropy to show that the failed-recorder style is characterized by much more noise (entropy 0.0345) than well-played instruments (0.0005), so the model must learn to generate deliberately noisy output.

Load-bearing premise

The entire failed-recorder style is captured by one professional player's intentionally flawed performances, so the model may be learning that player's idiosyncrasies rather than a general 'failed recorder' style.

Editorial extensions

If this is right

  • VAE-GAN's per-domain decoders are better suited than StarGAN's single unified decoder for capturing the noisy, inharmonic character of failed playing.
  • DDSP's pitch-invariance assumption breaks down when the target is intentionally off-pitch, so differentiable-synthesizer pipelines need modification for this scenario.
  • The FR109 dataset gives style transfer researchers a reusable benchmark for testing models on expressive or intentionally flawed instrumental styles.
  • The measured gap between FR109's Wiener entropy and that of converted audio indicates that even the best tested model only partially reproduces the failed-recorder noise profile.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because FR109 was recorded by only one professional, the current findings may capture that player's individual 'failed' style rather than a general failed-recorder category; a multi-player version of the dataset would test whether the style is transferable.
  • The same approach could apply to other deliberately degraded or expressive instrumental styles, such as vocal fry, jazz inflections, or distorted electric guitar, turning performance 'mistakes' into a controllable style dimension.
  • Wiener entropy could be used as a training-time regularizer to push generated audio closer to the failed-recorder noise profile, since the paper only uses it for analysis.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces a new task, “failed-music style transfer,” where normally played audio is converted to the style of a deliberately poor soprano recorder performance. The authors contribute the FR109 dataset: 109 recordings (5.05 hours) by one professional recorder player who intentionally injects errors such as cracked voice, overblowing, and failed tonguing. They compare three existing methods (StarGAN, VAE-GAN, and DDSP) for converting violin, clarinet, and saxophone clips from the Bach10 dataset into the failed-recorder target, using FAD, MOS listening tests, Mel-spectrogram inspection, and Wiener entropy. Their empirical claim is that VAE-GAN is the best of the tested models, with FAD 7.27 and MOS scores of SS 2.98, MS 3.56, SQ 3.00, beating StarGAN and DDSP. The paper also analyzes spectrographic differences, such as inharmonic partials, between well-played and failed-recorder audio.

Significance. If the FR109 dataset is accepted as representing a coherent failed-recorder style, the paper opens a new benchmark scenario for style transfer and provides a publicly available dataset to support it. The central premise, however, is that one professional player’s intentionally introduced errors define a generalizable style, and this is not validated by any inter-performer consistency check. The paper also includes no derivation; its value rests on the empirical comparison and dataset release. The comparison is useful preliminary evidence that adversarial spectrogram-based models can capture noisy, inharmonic target styles, while a DSP-based synthesizer (DDSP) fails under its pitch-invariance assumption. Overall, the contribution is more a dataset-and-task proposal than a conclusive model ranking, and the load-bearing assumption about the target style’s representativeness must be addressed before the broader claims can be accepted.

major comments (4)
  1. [Section II-C (FR109 dataset)] The entire target domain is a single professional player’s intentionally failed recordings. The paper repeatedly refers to “failed recorder style” in general, but no evidence is given that FR109 is representative of failed recorder playing beyond this one individual. There is no inter-performer consistency check, no per-piece annotation of error types, and no analysis of how much the learned spectrogram changes are player-specific. Consequently, the FAD and MOS results may measure fidelity to one person’s idiosyncratic artifacts rather than to a generalizable failed-recorder style. The paper should either add a second performer (even a small validation set) or explicitly restrict all claims to “the failed-recorder style of the FR109 player” and discuss the limitation.
  2. [Section V-A vs. Table III] The analysis in Section V-A states that, based on Mel spectrograms and informal listening, StarGAN achieves better style similarity to a failed recorder than VAE-GAN (e.g., inharmonic partials are “not as clearly” present in VAE-GAN). This directly contradicts the formal MOS SS scores in Table III, which show VAE-GAN (2.98) ahead of StarGAN (2.54). The paper must reconcile this inconsistency: either the spectrogram inspection and informal listening are unreliable, or the MOS result is unexpected and needs explanation. As written, the contradiction undermines the credibility of both the objective analysis and the subjective evaluation.
  3. [Section IV-B1, Table II (FAD)] The FAD scores are reported as single point estimates with no variance, confidence interval, or bootstrap uncertainty. With only 10 Bach10 pieces as the evaluation set, the difference between StarGAN (13.87) and VAE-GAN (7.27) cannot be assessed for significance. The text “StarGAN performs slightly worse than VAE-GAN” is therefore unsupported by the reported numbers. Please provide uncertainty estimates (e.g., bootstrap over clips or multiple FAD computations) or soften the claim accordingly.
  4. [Section IV-B2 (Subjective evaluation)] The listening test uses one randomly chosen clip per source-target pair, i.e., three clips per model in total (violin-to-recorder, clarinet-to-recorder, saxophone-to-recorder), with 16 responses. The presented p-values (SS p=0.09, MS p=0.06, SQ p=0.02) are only significant for SQ. The conclusion that “StarGAN’s overall performance falls behind VAE-GAN’s on all metrics” is thus not supported by conventional significance thresholds for two of the three metrics. Additionally, the paper does not state whether the 16 responses were pooled across the three clips per model or analyzed per clip; this should be clarified, and the statistical power of using a single clip per condition should be discussed.
minor comments (5)
  1. [Section IV-B1] There is a typo: “V AR-GAN” should be “VAE-GAN.”
  2. [Section IV-B1] The text says FAD is a “reference-free metric,” but FAD compares an evaluation set against a reference embedding set, so it is reference-based. Please correct the wording to avoid ambiguity.
  3. [Section IV-B1] The sentence “StarGAN performs slightly worse than VAE-GAN on both datasets” is misleading because only the Bach10 dataset is used for FAD evaluation; no FAD on URMP or FR109 test clips is reported.
  4. [Section IV-A] Training details are sparse: the paper does not report hyperparameters (e.g., learning rate, number of steps, adversarial loss weights) for the three methods or how the 90/10 split of FR109 was performed. Adding these details would improve reproducibility.
  5. [Section V-B] The STFT parameters used to compute Wiener entropy (window size, hop, FFT size) are not specified, making it difficult to reproduce the entropy values in Tables IV and V.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the FR109 target style is evaluated on a held-out split and external Bach10 sources, so no derivation reduces to its inputs.

full rationale

This paper proposes a new dataset and benchmarks existing style-transfer models against it; it contains no derivation from first principles whose conclusion is assumed in the inputs. The target domain is defined by the FR109 dataset, and the models are trained on a 90% split and evaluated on the held-out 10% split plus external Bach10 sources. FAD reference is the FR109 test split, which is a standard held-out target-domain reference rather than a fitted input; it is not used to fit any parameter. The Wiener-entropy and spectrogram analyses are post-hoc descriptive analyses of model outputs, not predictions derived from the data that define them. There are no self-citations, no imported uniqueness theorems, and no ansatz smuggled via citation; the three baselines (StarGAN, VAE-GAN, DDSP) and BigVSAN vocoder are external published methods with pretrained weights. The main limitation, that FR109 contains one professional player so the learned style may be idiosyncratic, is an external-validity threat rather than circularity: the paper's claim is about converting to this dataset's failed-recorder style, and it does not argue that the style generalizes to other players. Thus no load-bearing step reduces to its own inputs.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper's empirical claims rest on the coherence of the FR109 dataset as a target domain, the validity of pretrained components such as BigVSAN, VGGish, and CREPE, and the use of Wiener entropy and inharmonic partials as style proxies. No free parameters are fitted by the paper itself; the transferred models inherit hyperparameters from prior work.

assumptions (5)
  • domain assumption A single professional's deliberately poor recorder recordings define a coherent, generalizable failed recorder target style.
    Section II-C records the FR109 dataset with one professional and gives no inter-performer consistency or style-coherence analysis.
  • domain assumption The pretrained BigVSAN vocoder trained on LibriTTS speech can synthesize faithful audio from Mel spectrograms of musical input.
    Section III states BigVSAN pretrained weights on LibriTTS are used, with no music-domain adaptation or verification reported.
  • domain assumption Wiener entropy and visible inharmonic partials are valid quantitative proxies for failed recorder style similarity.
    Section V-B uses STFT Wiener entropy to compare datasets and outputs, but no human validation links Wiener entropy values to perceived failed-ness.
  • domain assumption FAD with VGGish embeddings measures style closeness for this noisy target.
    Section IV-B1 uses FAD with a pretrained VGGish audio classifier; no calibration or confidence intervals are reported.
  • domain assumption CREPE pitch estimates on off-pitch recorder audio are reliable enough to report FR109 pitch statistics.
    Section II-C uses CREPE to compute pitch mean 905 Hz and maximum 1990 Hz, with no validation on deliberately off-pitch recorder audio.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Music2Fail: Transfer Music to Failed Recorder Style." pith.science (2026). https://pith.science/paper/J6TBACRA

@misc{pith2026241118075,
  author       = {Pith},
  title        = {Pith review of: Music2Fail: Transfer Music to Failed Recorder Style},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J6TBACRA}},
  note         = {Machine review of arXiv:2411.18075}
}
read the original abstract

The goal of music style transfer is to convert a music performance by one instrument into another while keeping the musical contents unchanged. In this paper, we investigate another style transfer scenario called ``failed-music style transfer''. Unlike the usual music style transfer where the content remains the same and only the instrumental characteristics are changed, this scenario seeks to transfer the music from the source instrument to the target instrument which is deliberately performed off-pitch. Our work attempts to transfer normally played music into off-pitch recorder music, which we call ``failed-style recorder'', and study the results of the conversion. To carry out this work, we have also proposed a dataset of failed-style recorders for this task, called ``FR109 Dataset''. Such an experiment explores the music style transfer task in a more expressive setting, as the generated audio should sound like an ``off-pitch recorder'' while maintaining a certain degree of naturalness.

Figures

Figures reproduced from arXiv: 2411.18075 by the authors.

Figure 1
Figure 1. Mel spectrograms of failed recorder music, the red rectangular parts [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The Mel spectrograms of a failed-music style transfer example. (a) The [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 16 canonical work pages

  1. [1]

    MelGAN-VC: Voice Conversion and Audio Style Transfer on arbitrarily long samples using Spectrograms

    M. Pasini, “Melgan-vc: V oice conversion and audio style transfer on arbitrarily long samples using spectrograms,” arXiv preprint arXiv:1910.03713 , Oct. 2019

    work page Pith review arXiv 1910

  2. [2]

    Timbretron: A wavenet(cyclegan(cqt(audio))) pipeline for musical timbre transfer,

    S. Huang, Q. Li, C. Anil, X. Bao, S. Oore, and R. B. Grosse, “Timbretron: A wavenet(cyclegan(cqt(audio))) pipeline for musical timbre transfer,” CoRR, vol. abs/1811.09620, Nov. 2018

    arXiv 2018

  3. [3]

    DDSP: differentiable digital signal processing,

    J. H. Engel, L. Hantrakul, C. Gu, and A. Roberts, “DDSP: differentiable digital signal processing,” CoRR, vol. abs/2001.04643, Jan. 2020

    arXiv 2001

  4. [4]

    Differentiable WORLD synthesizer- based neural vocoder with application to end-to-end audio style transfer,

    S. Nercessian, “Differentiable WORLD synthesizer- based neural vocoder with application to end-to-end audio style transfer,” CoRR, vol. abs/2208.07282, 2022. arXiv: 2208.07282

    arXiv 2022

  5. [5]

    Modulated Variational auto-Encoders for many-to-many musical timbre transfer

    A. Bitton, P. Esling, and A. Chemla-Romeu-Santos, “Modulated variational auto-encoders for many-to-many musical timbre transfer,” CoRR, vol. abs/1810.00222, Oct. 2018

    work page Pith review arXiv 2018

  6. [6]

    Transplayer: Timbre style transfer with flexible timbre control,

    Y . Wu, Y . He, X. Liu, Y . Wang, and R. B. Dannen- berg, “Transplayer: Timbre style transfer with flexible timbre control,” in 2023 IEEE International Conference on Acoustics, Speech and Signal Processing , Rhodes Island, Greece, 2023, pp. 1–5

    work page 2023

  7. [7]

    Timbre trans- fer with variational auto encoding and cycle-consistent adversarial networks,

    R. S. Bonnici, M. Benning, and C. Saitis, “Timbre trans- fer with variational auto encoding and cycle-consistent adversarial networks,” in 2022 International Joint Con- ference on Neural Networks, Padua, Italy, 2022, pp. 1–8

    work page 2022

  8. [8]

    Timbre transfer using image-to-image denoising diffusion mod- els,

    L. Comanducci, F. Antonacci, and A. Sarti, “Timbre transfer using image-to-image denoising diffusion mod- els,” arXiv preprint arXiv:2307.04586 , Jul. 2023

    arXiv 2023

Show all 21 references
  1. [9]

    Cyclegan-vc: Non-parallel voice conversion using cycle-consistent adversarial net- works,

    T. Kaneko and H. Kameoka, “Cyclegan-vc: Non-parallel voice conversion using cycle-consistent adversarial net- works,” in 2018 26th European Signal Processing Con- ference, 2018, pp. 2100–2104

    2018

  2. [10]

    Unsupervised image-to-image translation networks,

    M.-Y . Liu, T. Breuel, and J. Kautz, “Unsupervised image-to-image translation networks,” in Advances in Neural Information Processing Systems , vol. 30, Curran Associates, Inc., 2017

    2017

  3. [11]

    Learning the beauty in songs: Neural singing voice beautifier,

    J. Liu, C. Li, Y . Ren, Z. Zhu, and Z. Zhao, “Learning the beauty in songs: Neural singing voice beautifier,” in Proceedings of the 60th Annual Meeting of the Asso- ciation for Computational Linguistics , Dublin, Ireland, 2022, pp. 7970–7983

    2022

  4. [12]

    Cre- ating a multitrack classical music performance dataset for multimodal music analysis: Challenges, insights, and applications,

    B. Li, X. Liu, K. Dinesh, Z. Duan, and G. Sharma, “Cre- ating a multitrack classical music performance dataset for multimodal music analysis: Challenges, insights, and applications,” IEEE Transactions on Multimedia , vol. 21, no. 2, pp. 522–535, Feb. 2018

    2018

  5. [13]

    Multiple fundamen- tal frequency estimation by modeling spectral peaks and non-peak regions,

    Z. Duan, B. Pardo, and C. Zhang, “Multiple fundamen- tal frequency estimation by modeling spectral peaks and non-peak regions,” IEEE Trans. Speech Audio Process. , vol. 18, no. 8, pp. 2121–2133, Nov. 2010

    2010

  6. [14]

    Crepe: A convolutional representation for pitch estimation,

    J. W. Kim, J. Salamon, P. Li, and J. P. Bello, “Crepe: A convolutional representation for pitch estimation,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing , 2018, pp. 161–165

    2018

  7. [15]

    Stargan: Unified generative adversarial net- works for multi-domain image-to-image translation,

    Y . Choi, M. Choi, M. Kim, J.-W. Ha, S. Kim, and J. Choo, “Stargan: Unified generative adversarial net- works for multi-domain image-to-image translation,” in Proceedings of the IEEE conference on computer vision and pattern recognition , Salt Lake City, Utah, USA, 2018, pp. 8789–8797

    2018

  8. [16]

    Bigvsan: En- hancing gan-based neural vocoders with slicing adver- sarial network,

    T. Shibuya, Y . Takida, and Y . Mitsufuji, “Bigvsan: En- hancing gan-based neural vocoders with slicing adver- sarial network,” in 2024 IEEE International Conference on Acoustics, Speech and Signal Processing , Seoul, Korea, 2024, pp. 10 121–10 125

    2024

  9. [17]

    Libritts: A corpus derived from librispeech for text-to-speech,

    H. Zen, V . Dang, R. Clark, et al. , “Libritts: A corpus derived from librispeech for text-to-speech,” in The 20th Annual Conference of the International Speech Com- munication Association , Graz, Austria: ISCA, 2019, pp. 1526–1530

    2019

  10. [18]

    Fr ´echet audio distance: A metric for evaluat- ing music enhancement algorithms,

    K. Kilgour, M. Zuluaga, D. Roblek, and M. Shar- ifi, “Fr ´echet audio distance: A metric for evaluat- ing music enhancement algorithms,” arXiv preprint arXiv:1812.08466, Dec. 2018

    2018 arXiv

  11. [19]

    Cnn ar- chitectures for large-scale audio classification,

    S. Hershey, S. Chaudhuri, D. P. W. Ellis, et al., “Cnn ar- chitectures for large-scale audio classification,” in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing , New Orleans, Louisiana, USA, 2017, pp. 131–135

    2017

  12. [20]

    Mean opinion score (mos) revisited: Methods and applica- tions, limitations and alternatives,

    R. C. Streijl, S. Winkler, and D. S. Hands, “Mean opinion score (mos) revisited: Methods and applica- tions, limitations and alternatives,” Multimedia Systems, vol. 22, no. 2, pp. 213–227, 2016

    2016

  13. [21]

    Transform coding of audio signals using perceptual noise criteria,

    J. Johnston, “Transform coding of audio signals using perceptual noise criteria,” IEEE Journal on Selected Areas in Communications , vol. 6, no. 2, pp. 314–323, 1988

    1988

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.