Is GAN Necessary for Mel-Spectrogram-based Neural Vocoder?

Hui-Peng Du; Rui-Chen Zheng; Yang Ai; Ye-Xin Lu; Zhen-Hua Ling

REVIEW 3 major objections 4 minor 33 references

Is GAN Necessary for Mel-Spectrogram-based Neural Vocoder?

T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that GAN training is unnecessary for mel-spectrogram neural vocoders that explicitly predict phase, and that FreeGAN reaches listening quality comparable to GAN-based vocoders without it.

desk verdict A solid, useful empirical result—GAN really does look unnecessary for explicit-phase vocoders—but the paper's 'comparable' claim rests on null-hypothesis tests that can't support equivalence, and the missing code and error bars make the ablation harder to trust. read the letter →

arxiv 2508.07711 v1 pith:ECSPTYJF submitted 2025-08-11 eess.AS

Hui-Peng Du , Yang Ai , Rui-Chen Zheng , Ye-Xin Lu , Zhen-Hua Ling This is my paper · ORCID

classification eess.AS

keywords neuralvocodermel-spectrogramGAN-freetrainingexplicitphasepredictionamplitudeprioranti-wrappinglossspeechsynthesisinverseshort-timeFouriertransform

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether GAN training is truly required to make a mel-spectrogram neural vocoder produce natural speech. Its answer is no, at least for vocoders that explicitly predict the phase spectrum: FreeGAN, a fully GAN-free vocoder, first predicts amplitude, then phase, and reconstructs the waveform by inverse STFT, and its speech quality is comparable to GAN-based systems such as BigVGAN and APNet2 in listening tests. To make up for the discarded discriminator, FreeGAN adds an amplitude prior computed from the mel-spectrogram via a Mel pseudo-inverse filter, a SNAKE-ConvNeXt v2 backbone, and a frequency-weighted anti-wrapping phase loss. The paper also shows that other explicit-phase-prediction vocoders can drop GAN when given these additions, suggesting the result is a general recipe rather than a one-off architecture.

What carries the argument

The load-bearing mechanism is the amplitude-phase serial prediction pipeline driven by an amplitude prior. FreeGAN computes the prior as $A^+ = \max(|XM^+|, \epsilon)$, the absolute value of the mel-spectrogram filtered by the Mel pseudo-inverse matrix, and a single SNAKE-ConvNeXt v2 block predicts the amplitude spectrum from it. Four SNAKE-ConvNeXt v2 blocks then predict the phase spectrum conditioned on that amplitude estimate, using a parallel estimation architecture and anti-wrapping losses; the new frequency-weighted anti-wrapping phase loss assigns growing weights to higher frequency bins. This machinery replaces the implicit phase supervision that GAN discriminators normally provide w

What would settle it

Run the full FreeGAN recipe at 48 kHz or on reverberant/noisy speech, where the mel pseudo-inverse amplitude prior is a poorer carrier of phase-relevant detail. If listening tests then show a significant MOS gap against BigVGAN or HiFi-GAN trained under the same conditions, the load-bearing premise fails. A cheaper check: plot per-frequency phase error (e.g., group-delay error) for FreeGAN; the paper reports high-frequency phase was the first thing to degrade without GAN, so showing the frequency-weighted anti-wrapping loss cannot close the high-frequency tail of that error would also falsify

Watch

Extended reading notes

Core claim

The paper's central claim is that adversarial supervision is not a necessary ingredient for high-quality mel-spectrogram vocoding; what matters is explicit phase prediction plus targeted compensations. FreeGAN scores 4.12 in MOS compared with 4.14 for BigVGAN (p = 0.61) and 4.13 for APNet2 (p = 0.76) on the VCTK test set, while outperforming all baselines on SNR, MCD, F0-RMSE, and V/UV error. It uses 2.70 GFLOPS for one second of speech, trains in 191 seconds per epoch versus 1937 for BigVGAN, and stores only a 13.4M-parameter generator because there is no discriminator. The paper further demonstrates that the recipe transfers: APNet2 and FreeV, both explicit-phase vocoders, retain quality w

Load-bearing premise

The argument depends on the premise, carried over from prior phase-prediction work, that a sufficiently accurate amplitude spectrum contains the information needed to predict the phase spectrum without any adversarial supervision; if that premise fails, FreeGAN's quality should fall short of GAN-based vocoders no matter how the losses are weighted.

Editorial extensions

If this is right

A mel-spectrogram vocoder can produce GAN-comparable speech with no discriminators, so GAN training is not a necessary ingredient in the explicit-phase-prediction family.
GAN-free vocoding is much cheaper to train and deploy: FreeGAN trains about 15 times faster than BigVGAN and removes all discriminator storage.
The three proposed additions form a transferable recipe: APNet2 and FreeV match their GAN-equipped versions after the additions, with ABX p-values of 1.00 and 0.177.
Vocoders that do not explicitly predict phase remain GAN-dependent; for them the recipe only narrows but does not close the quality gap.
FreeGAN's low latency and parameter count make it suitable for resource-constrained speech synthesis and coding applications.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond vocoding, the same amplitude-prior plus anti-wrapping phase-loss logic is a natural template for speech enhancement, bandwidth extension, and codec tasks that must estimate phase from magnitude-like conditioning; testing whether those domains also shed adversarial training would be a direct extension.
The experiments are on 16 kHz VCTK, so the most useful open test is high sampling rates and unseen acoustic conditions; the paper's ablation leaves open whether frequency-weighted anti-wrapping phase loss closes the high-frequency phase gap at 48 kHz.
A cheap falsifiable prediction follows from the paper's premise: any explicit-phase vocoder fed a strong-enough amplitude prior should improve monotonically as the prior gets closer to the natural amplitude spectrum, so feeding FreeGAN the oracle natural amplitude during phase prediction and measuring the remaining phase error would isolate how much quality comes from the prior versus the phase lo
If the result generalizes, the conventional generator-plus-discriminator pairing in neural audio may be separable: high quality can come from better phase objectives rather than adversarial supervision.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing to know: this paper likely gets the core empirical claim right. FreeGAN, a GAN-free vocoder built on serial amplitude-then-phase prediction, reaches MOS 4.12 against BigVGAN's 4.14 and APNet2's 4.13 on VCTK, with a fraction of the training cost and no discriminator to store. The ABX tests showing APNet2 and FreeV with our strategies perform no worse than their GAN versions is a genuinely useful generalization. That is a real finding, not a reshuffle of existing components.

What deserves credit: they did the hard comparative work. They reproduced GAN-free versions of five baselines, applied their tricks to those baselines, and tested explicitly which class of vocoders can drop GAN. The separation by explicit phase prediction is clean and is supported by the pattern in the UTMOS numbers. The three proposed pieces—amplitude prior, SNAKE-ConvNeXt v2, frequency-weighted anti-wrapping loss—each show a plausible contribution in ablation, with the amplitude prior pulling the most weight. The efficiency numbers are concrete. No code is released, which is annoying but not disqualifying for a conference submission.

The soft spots are real, but not fatal. The stress-test note is right: non-significant p-values are not evidence of equivalence. The MOS confidence intervals are all ±0.06, which makes a real difference of 0.2–0.3 MOS inconceivable, but the paper never pre-specifies an equivalence margin. A TOST or a reported 90% CI on the MOS difference would fix this. The same goes for the ABX tests: p=0.177 for FreeV might hide a small preference. The authors' conclusion 'GAN is unnecessary' is probably right, but the statistical support is weaker than the prose suggests. That is a minor-to-moderate issue in a paper whose other evidence (UTMOS, objective metrics, ABX patterns) points the same way.

Also: the 'Table II' ablation has no significance tests or repeated seeds. Dropping the prior drops UTMOS by 0.11 and MCD doubles; that is believable, but without variance we cannot know how robust it is. Hyperparameters rho and the loss weights are fixed, with no sensitivity analysis. Minor, but worth a revision.

The one thing I'd push back on in the stress-test: the weakest-assumption point about the amplitude prior is less load-bearing, because FreeGAN w/o Prior still gets UTMOS 3.81 and APNet2/FreeV don't use the serial framework. The inference issue is the real concern. The exclusion of BigVGAN from the GAN-free ablation is also not a flaw for the explicit-phase claim, since BigVGAN isn't explicit-phase; it's a limitation only if you read the title as claiming all mel vocoders.

Bottom line: this is a good paper with a slightly overstated conclusion. I'd send it to review. It deserves referee time, and the authors can fix the equivalence issue with a small statistical addition. I'd cite it in my own vocoder work, and bring it to reading group.

Recommendation: accept with revisions after a focused statistical review.

Referee Report

3 major / 4 minor

Summary. The paper asks whether GAN training is necessary for mel-spectrogram-based neural vocoders. It proposes FreeGAN, a GAN-free vocoder that predicts the amplitude spectrum first and then the phase spectrum, reconstructing the waveform via iSTFT. Three design elements are introduced to compensate for the absence of adversarial training: an amplitude prior obtained by Mel pseudo-inverse filtering, a SNAKE-ConvNeXt v2 backbone, and a frequency-weighted anti-wrapping phase loss. The authors report objective metrics, UTMOS, MOS, and ABX tests comparing FreeGAN with HiFi-GAN, BigVGAN, iSTFTNet, Vocos, APNet2, and FreeV, and also test GAN-free versions of these baselines. They conclude that FreeGAN matches GAN-based vocoders in subjective quality while being far more efficient, and that explicit-phase-prediction vocoders such as APNet2 and FreeV no longer need GAN when augmented with the proposed strategies.

Significance. If the central comparability claim can be established rigorously, the result is practically significant: it challenges the prevailing assumption that GAN-based adversarial training is required for high-fidelity mel-spectrogram vocoders, and it quantifies concrete savings in training time (e.g., roughly 15x faster than BigVGAN) and storage (no discriminator parameters). The paper's evidence is not circular: the central comparison is against external GAN-based baselines, and the ablation isolates the contribution of each proposed component. The objective metrics, while mostly point estimates, are reported for multiple baselines on a common evaluation setup, and the demo page is a useful complement. The main weakness is statistical: non-significant p-values are interpreted as evidence of equivalence, and the ABX p-values are under-specified. This statistical gap is load-bearing because it supports the headline 'GAN is unnecessary' claim, so the manuscript needs a revision that provides proper equivalence testing or decisive confidence-interval reasoning.

major comments (3)

[§IV-A, Table I] The central claim that FreeGAN is 'comparable' to BigVGAN and APNet2 is inferred from non-significant t-tests (p=0.61 and 0.76). Failing to reject equality is not evidence of equivalence; with 20 test utterances the tests may simply be underpowered. To support a positive comparability claim, please pre-specify an equivalence margin (e.g., ±0.1 MOS), report the 90% confidence interval for the mean difference, and apply a two one-sided tests (TOST) procedure or equivalent. The same requirement applies to the ABX comparisons in §IV-B and Figure 2, where 'no significant difference' is used to support the generalizability conclusion.
[§IV-A, Table I] The text states that 'for objective metrics such as SNR, MCD, F0-RMSE, and U/V error, FreeGAN significantly outperformed all other baseline vocoders.' This claim is not supported because Table I reports only point estimates; there are no error bars, confidence intervals, or significance tests for these metrics. Moreover, on UTMOS FreeGAN is numerically worse than BigVGAN, APNet2, and FreeV (3.92 vs 3.97–4.00), which the text acknowledges but does not reconcile with the word 'significantly.' Please provide variance measures and statistical tests for the objective metrics, or temper the claim to 'numerically better on some metrics.'
[§IV-B, Figure 2] The ABX result for APNet2 reports p=1.00 with preference shares of 42.40% for APNet2, 15.20% for APNet2* w/o GAN, and 42.40% 'no preference.' A p-value of 1.00 is not consistent with a significance test comparing the two preference proportions, which would be strongly non-significant in the opposite direction if tested naïvely. The sentence 'p denotes the p value of a t-test' is ambiguous: the unit of analysis (listeners? utterances?), the test being applied, and the degrees of freedom are not stated. Please clarify how p=1.00 is obtained; as written, the result is internally unexplained and weakens the conclusion that explicit-phase-prediction vocoders no longer need GAN.

minor comments (4)

[§II-A, Eq. (1)] The notation is inconsistent: the mel-spectrogram is denoted X in Eq. (1) but M in the surrounding text ('amplitude prior A+ as input, instead of the mel spectrogram M'). Please unify the notation.
[§II-C, Eq. (3)] The definition of the weight vector is written compactly. It should be explicit that w_i = ρ^(i/(N-1)) for i=0,...,N-1, and the operator E(·) in Eq. (3) is not defined. Adding parentheses around the expectation and the frequency-weighting term would improve readability.
[§IV-A, Table II] The ablation row 'FreeGAN w/o FW AWP' shows a decrease in SNR (7.59 vs 7.73) but identical UTMOS (3.92), and the text says the contribution is 'slightly smaller compared to the amplitude prior.' It would be helpful to report the phase-specific errors (e.g., phase distance) that the FW-AWP loss is designed to improve, rather than only global metrics.
[General] There are several typos: 'anti-warpping' (in the ablation description) should be 'anti-wrapping,' and 'Mel spectogram' in the Figure 1 caption should be 'Mel spectrogram.' The demo URL is given, but no code is released; providing training/evaluation code would ease reproduction.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim is tested against external GAN-based baselines, not derived from self-citations or construction.

full rationale

The paper's central claim—that GAN is unnecessary for explicit-phase-prediction mel-spectrogram vocoders—is evaluated through objective metrics (SNR, MCD, F0-RMSE, V/UV, UTMOS), subjective MOS tests against BigVGAN and APNet2, and ABX preference tests against external baselines (HiFi-GAN, iSTFTNet, Vocos, FreeV). These are external comparisons, not quantities forced by the model's definitions. The amplitude prior is a fixed pseudo-inverse filter, not a fitted parameter renamed as a prediction. The self-citations to prior work [22] and [24] supply architectural components and a motivating premise (phase can be predicted from amplitude), but the conclusion does not reduce to these citations; it rests on the measured performance of the newly proposed FreeGAN and on the GAN-free versions of APNet2 and FreeV. No equation in the paper is shown to be equivalent to another by construction, and no fitted value is presented as an independent prediction. The statistical interpretation of non-significant p-values as equivalence is a validity concern, not a circularity concern, because the comparisons are still external and falsifiable. Therefore, the derivation chain is self-contained with respect to circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the reliability of phase prediction from amplitude, the choice of several hyperparameters, and the transferability of the three strategies to other explicit-phase vocoders. No new physical entities are introduced.

free parameters (5)

rho = 2.5
Controls the frequency weighting in FWAWP loss; chosen without sensitivity analysis.
epsilon = 1e-5
Lower bound for amplitude prior to ensure non-negativity; small number set by hand.
lambda_A = 0.45
Weight for amplitude loss in the total loss.
lambda_S = 0.2
Weight for reconstructed STFT loss.
lambda_Mel = 0.45
Weight for mel-spectrogram loss.

assumptions (4)

domain assumption A high-quality amplitude spectrum contains enough information to predict the phase spectrum accurately.
Invoked in Section II-A; based on authors' prior work [24], not independently established here.
standard math STFT/iSTFT is a valid invertible reconstruction given amplitude and phase.
Used in the serial prediction framework and waveform reconstruction.
domain assumption The SNAKE-ConvNeXt v2 architecture can learn the amplitude-phase mapping with the given configuration.
Assumed in Section II-B; the configuration is borrowed from APNet2 and not independently justified.
domain assumption The mel pseudo-inverse filter is a fixed linear operator that provides a useful amplitude prior.
Used in Eq. (1); the filter is derived from the mel filterbank, assumed to be a meaningful inverse.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Is GAN Necessary for Mel-Spectrogram-based Neural Vocoder?." pith.science (2026). https://pith.science/paper/ECSPTYJF

@misc{pith2026250807711,
  author       = {Pith},
  title        = {Pith review of: Is GAN Necessary for Mel-Spectrogram-based Neural Vocoder?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ECSPTYJF}},
  note         = {Machine review of arXiv:2508.07711}
}

read the original abstract

Recently, mainstream mel-spectrogram-based neural vocoders rely on generative adversarial network (GAN) for high-fidelity speech generation, e.g., HiFi-GAN and BigVGAN. However, the use of GAN restricts training efficiency and model complexity. Therefore, this paper proposes a novel FreeGAN vocoder, aiming to answer the question of whether GAN is necessary for mel-spectrogram-based neural vocoders. The FreeGAN employs an amplitude-phase serial prediction framework, eliminating the need for GAN training. It incorporates amplitude prior input, SNAKE-ConvNeXt v2 backbone and frequency-weighted anti-wrapping phase loss to compensate for the performance loss caused by the absence of GAN. Experimental results confirm that the speech quality of FreeGAN is comparable to that of advanced GAN-based vocoders, while significantly improving training efficiency and complexity. Other explicit-phase-prediction-based neural vocoders can also work without GAN, leveraging our proposed methods.

Figures

Figures reproduced from arXiv: 2508.07711 by the authors.

**Figure 1.** Overview of the proposed FreeGAN vocoder and the strategies we proposed. The strategies we introduced are highlighted in red italics, including [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 2.** Average preference scores (%) of ABX tests on speech quality [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 27 canonical work pages

[1]

Natural TTS synthesis by conditioning wavenet on mel spectrogram predictions,

J. Shen, R. Pang, R. J. Weiss, M. Schuster, N. Jaitly, Z. Yang, Z. Chen, Y . Zhang, Y . Wang, R. Skerrv-Ryan et al. , “Natural TTS synthesis by conditioning wavenet on mel spectrogram predictions,” inProc. ICASSP, 2018, pp. 4779–4783

work page 2018
[2]

Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens,

Z. Du, Q. Chen, S. Zhang, K. Hu, H. Lu, Y . Yang, H. Hu, S. Zheng, Y . Gu, Z. Ma et al. , “Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens,” arXiv preprint arXiv:2407.05407, 2024

arXiv 2024
[3]

MP-SENet: A speech enhancement model with parallel denoising of magnitude and phase spectra,

Y .-X. Lu, Y . Ai, and Z.-H. Ling, “MP-SENet: A speech enhancement model with parallel denoising of magnitude and phase spectra,” in Proc. Interspeech, 2023, pp. 3834–3838

2023
[4]

AudioSR: Versatile audio super-resolution at scale,

H. Liu, K. Chen, Q. Tian, W. Wang, and M. D. Plumbley, “AudioSR: Versatile audio super-resolution at scale,” in Proc. ICASSP , 2024, pp. 1076–1080

2024
[5]

Towards high-quality and efficient speech bandwidth extension with parallel amplitude and phase prediction,

Y .-X. Lu, Y . Ai, H.-P. Du, and Z.-H. Ling, “Towards high-quality and efficient speech bandwidth extension with parallel amplitude and phase prediction,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, pp. 1–14, 2024

work page 2024
[6]

Spectral codecs: Spectrogram-based audio codecs for high quality speech synthesis,

R. Langman, A. Juki ´c, K. Dhawan, N. R. Koluguri, and B. Ginsburg, “Spectral codecs: Spectrogram-based audio codecs for high quality speech synthesis,” arXiv preprint arXiv:2406.05298 , 2024

arXiv 2024
[7]

A bitrate-scalable variational recurrent mel-spectrogram coder for real-time resynthesis-based speech coding,

B. Stahl, S. Windtner, and A. Sontacchi, “A bitrate-scalable variational recurrent mel-spectrogram coder for real-time resynthesis-based speech coding,” IEEE Access , 2024

work page 2024
[8]

Speaker-dependent WaveNet vocoder

A. Tamamori, T. Hayashi, K. Kobayashi, K. Takeda, and T. Toda, “Speaker-dependent WaveNet vocoder.” in Proc. Interspeech, 2017, pp. 1118–1122

work page 2017

Show all 33 references

[9]

SampleRNN-based neural vocoder for statistical parametric speech synthesis,

Y . Ai, H.-C. Wu, and Z.-H. Ling, “SampleRNN-based neural vocoder for statistical parametric speech synthesis,” in Proc. ICASSP, 2018, pp. 5659–5663

2018
[10]

A neural vocoder with hierarchical generation of amplitude and phase spectra for statistical parametric speech synthesis,

Y . Ai and Z.-H. Ling, “A neural vocoder with hierarchical generation of amplitude and phase spectra for statistical parametric speech synthesis,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 28, pp. 839–851, 2020

2020
[11]

WaveGlow: A flow-based generative network for speech synthesis,

R. Prenger, R. Valle, and B. Catanzaro, “WaveGlow: A flow-based generative network for speech synthesis,” in Proc. ICASSP 2019 , pp. 3617–3621

2019
[12]

WaveFlow: A compact flow- based model for raw audio,

W. Ping, K. Peng, K. Zhao, and Z. Song, “WaveFlow: A compact flow- based model for raw audio,” in Proc. ICML, 2020, pp. 7706–7716

2020
[13]

MelGAN: Generative adversarial networks for conditional waveform synthesis,

K. Kumar, R. Zhang, C. Fuegen, R. Puri, Y . Zhang, and B. Catanzaro, “MelGAN: Generative adversarial networks for conditional waveform synthesis,” in Proc. NeurIPS, 2019, pp. 14 881–14 892

2019
[14]

HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis,

J. Kong, J. Kim, and J. Bae, “HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis,” Advances in Neural Information Processing Systems , vol. 33, pp. 17 022–17 033, 2020

2020
[15]

APNet: An all-frame-level neural vocoder incor- porating direct prediction of amplitude and phase spectra,

Y . Ai and Z.-H. Ling, “APNet: An all-frame-level neural vocoder incor- porating direct prediction of amplitude and phase spectra,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 31, pp. 2145–2157, 2023

2023
[16]

BigVGAN: A universal neural vocoder with large-scale training,

S.-g. Lee, W. Ping, B. Ginsburg, B. Catanzaro, and S. Yoon, “BigVGAN: A universal neural vocoder with large-scale training,” in Proc. ICLR , 2023

2023
[17]

iSTFTNet: Fast and lightweight mel-spectrogram vocoder incorporating inverse short-time Fourier transform,

T. Kaneko, K. Tanaka, H. Kameoka, and S. Seki, “iSTFTNet: Fast and lightweight mel-spectrogram vocoder incorporating inverse short-time Fourier transform,” in Proc. ICASSP, 2022, pp. 6207–6211

2022
[18]

Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram,

R. Yamamoto, E. Song, and J.-M. Kim, “Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram,” in Proc. ICASSP , 2020, pp. 6199– 6203

2020
[19]

Fregrad: Lightweight and fast frequency-aware diffusion vocoder,

T. D. Nguyen, J.-H. Kim, Y . Jang, J. Kim, and J. S. Chung, “Fregrad: Lightweight and fast frequency-aware diffusion vocoder,” in Proc. ICASSP, 2024, pp. 10 736–10 740

2024
[20]

Direct modelling of magnitude and phase spectra for statistical parametric speech synthesis

F. Espic, C. Valentini-Botinhao, and S. King, “Direct modelling of magnitude and phase spectra for statistical parametric speech synthesis.” in Proc. Interspeech, 2017, pp. 1383–1387

2017
[21]

Speech acoustic modelling from raw phase spectrum,

E. Loweimi, Z. Cvetkovic, P. Bell, and S. Renals, “Speech acoustic modelling from raw phase spectrum,” in Proc. ICASSP, 2021, pp. 6738– 6742

2021
[22]

APNet2: High-quality and high-efficiency neural vocoder with direct prediction of amplitude and phase spectra,

H.-P. Du, Y .-X. Lu, Y . Ai, and Z.-H. Ling, “APNet2: High-quality and high-efficiency neural vocoder with direct prediction of amplitude and phase spectra,” in Proc. NCMMSC, 2023 , pp. 66–80

2023
[23]

FreeV: Free lunch for vocoders through pseudo inversed mel filter,

Y . Lv, H. Li, Y . Yan, J. Liu, D. Xie, and L. Xie, “FreeV: Free lunch for vocoders through pseudo inversed mel filter,” in Proc. Interspeech, 2024, pp. 3869–3873

2024
[24]

Neural speech phase prediction based on parallel estimation architecture and anti-wrapping losses,

Y . Ai and Z.-H. Ling, “Neural speech phase prediction based on parallel estimation architecture and anti-wrapping losses,” in Proc. ICASSP , 2023, pp. 1–5

2023
[25]

ConvNeXt v2: Co-designing and scaling convnets with masked autoencoders,

S. Woo, S. Debnath, R. Hu, X. Chen, Z. Liu, I. S. Kweon, and S. Xie, “ConvNeXt v2: Co-designing and scaling convnets with masked autoencoders,” in Proc. CVPR, 2023, pp. 16 133–16 142

2023
[26]

MDCTCodec: A lightweight MDCT-based neural audio codec towards high sampling rate and low bitrate scenarios,

X.-H. Jiang, Y . Ai, R.-C. Zheng, H.-P. Du, Y .-X. Lu, and Z.-H. Ling, “MDCTCodec: A lightweight MDCT-based neural audio codec towards high sampling rate and low bitrate scenarios,” in Proc. SLT, 2024, pp. 550–557

2024
[27]

Bivocoder: A bidirectional neural vocoder integrating feature extraction and waveform generation,

H.-P. Du, Y .-X. Lu, Y . Ai, and Z.-H. Ling, “Bivocoder: A bidirectional neural vocoder integrating feature extraction and waveform generation,” in Proc. Interspeech, 2024, pp. 3894–3898

2024
[28]

Gaussian error linear units (GELUs),

D. Hendrycks and K. Gimpel, “Gaussian error linear units (GELUs),” arXiv preprint arXiv:1606.08415 , 2016

2016 arXiv
[29]

Neural networks fail to learn periodic functions and how to fix it,

L. Ziyin, T. Hartwig, and M. Ueda, “Neural networks fail to learn periodic functions and how to fix it,” Advances in Neural Information Processing Systems, vol. 33, pp. 1583–1594, 2020

2020
[30]

Superseded-CSTR VCTK corpus: English multi-speaker corpus for cstr voice cloning toolkit,

C. Veaux, J. Yamagishi, K. MacDonald et al., “Superseded-CSTR VCTK corpus: English multi-speaker corpus for cstr voice cloning toolkit,” 2016

2016
[31]

Adam: A method for stochastic optimization,

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proc. ICLR, 2015

2015
[32]

V ocos: Closing the gap between time-domain and fourier- based neural vocoders for high-quality audio synthesis,

H. Siuzdak, “V ocos: Closing the gap between time-domain and fourier- based neural vocoders for high-quality audio synthesis,” in Proc. ICLR, 2024

2024
[33]

UTMOS: Utokyo-sarulab system for voiceMOS Chal- lenge 2022,

T. Saeki, D. Xin, W. Nakata, T. Koriyama, S. Takamichi, and H. Saruwatari, “UTMOS: Utokyo-sarulab system for voiceMOS Chal- lenge 2022,” in Proc. Interspeech, 2022, pp. 4521–4525

2022

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

Natural TTS synthesis by conditioning wavenet on mel spectrogram predictions,

J. Shen, R. Pang, R. J. Weiss, M. Schuster, N. Jaitly, Z. Yang, Z. Chen, Y . Zhang, Y . Wang, R. Skerrv-Ryan et al. , “Natural TTS synthesis by conditioning wavenet on mel spectrogram predictions,” inProc. ICASSP, 2018, pp. 4779–4783

work page 2018

[2] [2]

Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens,

Z. Du, Q. Chen, S. Zhang, K. Hu, H. Lu, Y . Yang, H. Hu, S. Zheng, Y . Gu, Z. Ma et al. , “Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens,” arXiv preprint arXiv:2407.05407, 2024

arXiv 2024

[3] [3]

MP-SENet: A speech enhancement model with parallel denoising of magnitude and phase spectra,

Y .-X. Lu, Y . Ai, and Z.-H. Ling, “MP-SENet: A speech enhancement model with parallel denoising of magnitude and phase spectra,” in Proc. Interspeech, 2023, pp. 3834–3838

2023

[4] [4]

AudioSR: Versatile audio super-resolution at scale,

H. Liu, K. Chen, Q. Tian, W. Wang, and M. D. Plumbley, “AudioSR: Versatile audio super-resolution at scale,” in Proc. ICASSP , 2024, pp. 1076–1080

2024

[5] [5]

Towards high-quality and efficient speech bandwidth extension with parallel amplitude and phase prediction,

Y .-X. Lu, Y . Ai, H.-P. Du, and Z.-H. Ling, “Towards high-quality and efficient speech bandwidth extension with parallel amplitude and phase prediction,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, pp. 1–14, 2024

work page 2024

[6] [6]

Spectral codecs: Spectrogram-based audio codecs for high quality speech synthesis,

R. Langman, A. Juki ´c, K. Dhawan, N. R. Koluguri, and B. Ginsburg, “Spectral codecs: Spectrogram-based audio codecs for high quality speech synthesis,” arXiv preprint arXiv:2406.05298 , 2024

arXiv 2024

[7] [7]

A bitrate-scalable variational recurrent mel-spectrogram coder for real-time resynthesis-based speech coding,

B. Stahl, S. Windtner, and A. Sontacchi, “A bitrate-scalable variational recurrent mel-spectrogram coder for real-time resynthesis-based speech coding,” IEEE Access , 2024

work page 2024

[8] [8]

Speaker-dependent WaveNet vocoder

A. Tamamori, T. Hayashi, K. Kobayashi, K. Takeda, and T. Toda, “Speaker-dependent WaveNet vocoder.” in Proc. Interspeech, 2017, pp. 1118–1122

work page 2017