REVIEW 3 major objections 6 minor 38 references
Revisiting Vocos: That Phasiness Business in Time-Frequency Neural Vocoding
T0 review · 3 major / 6 minor · reviewed 2026-07-31 · grok-4.5
Pith's one-line read Vocos still trails time-domain vocoders because its 1D convolutions cannot capture the time-frequency structure required for accurate phase reconstruction.
desk verdict Solid diagnostic ablations on Vocos; the 1D-vs-2D claim is overstated because Table 3 changes capacity and depth at once, but the rest of the evidence still holds. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Phase-difference prediction (baseband and frequency phase differences) used as a non-autoregressive diagnostic task. Training the Vocos backbone solely on these differences, then integrating them by least squares, tests whether the architecture can capture the local time-frequency structure that classical phase-reconstruction methods exploit.
What would settle it
A Vocos-style model that retains 1D convolutions yet matches BigVGAN MUSHRA scores on the same band-limited mel inputs and training recipe, or a 2D-convolution Vocos that closes the listening-test gap while still accepting arbitrary conditioning representations.
Extended reading notes
Core claim
The Vocos architecture is effective for magnitude modeling but substantially less so for phase. When the backbone is adapted to predict phase differences from ground-truth log-magnitude spectrograms, Conv1D layers perform poorly, whereas the identical design with Conv2D layers dramatically improves both wrapping-aware loss and log-spectral convergence. This isolates 1D convolutions as the component that prevents the network from modeling the time-frequency couplings needed for phase reconstruction.
Load-bearing premise
That success or failure at predicting phase differences from perfect magnitude spectrograms is a valid explanation for the phase errors that appear in the full end-to-end vocoder trained jointly on mel inputs with adversarial losses.
Editorial extensions
If this is right
- Time-frequency vocoders will not close the quality gap until their backbones better encode local time-frequency structure of speech.
- Scaling the existing 1D Vocos architecture or simply adding magnitude/phase spectrogram losses is unlikely to remove the residual phasiness.
- Allowing the network to predict inconsistent magnitudes currently compensates for phase errors; forcing consistency may hurt waveform quality.
- Future designs need inductive biases for harmonic and local TF structure that do not rely on pitch priors or restrict the input representation.
- Band-limited mel spectrograms remain a useful stress test that accentuates the domain gap between time-domain and time-frequency vocoders.
Reading between the lines
- The same 1D-versus-2D contrast may explain why other purely convolutional spectrogram-to-waveform models underperform relative to models that explicitly exploit time-frequency locality.
- A hybrid that keeps a cheap 1D path for arbitrary conditioning while adding a lightweight 2D refinement stage could preserve Vocos efficiency while fixing phase.
- The overlooked magnitude-clamping threshold suggests some published Vocos quality numbers may be limited by a hyperparameter rather than pure architecture.
- If phase-difference prediction is a reliable diagnostic, future TF-vocoder papers should report classical consistency metrics alongside perceptual scores.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript revisits the Vocos time-frequency vocoder to explain its residual quality gap relative to time-domain vocoders such as BigVGAN. The authors retrain both systems on LibriTTS with a shared, modern GAN recipe and a parameter-matched BigVGANv2-base, and confirm via objective metrics and a 16-listener MUSHRA test that the gap persists after controlling for loss functions, discriminators, and model size. Oracle ablations (Vocos-Mag / Vocos-Phase, Table 2) show the Vocos backbone predicts magnitude compatible with ground-truth phase almost perfectly, but cannot predict phase compatible with ground-truth magnitude — and that joint prediction beats phase-only prediction, implying Vocos's 'inconsistent' magnitude actively compensates its phase errors. Finally, the authors adapt the Vocos backbone to predict baseband/frequency phase differences (a non-autoregressive precursor of phase reconstruction) and find the Conv1D backbone fails (L_wa 0.646, LSC −20.24 dB) while a small Conv2D variant succeeds (0.112, −29.56 dB) and beats the Masuyama et al. benchmark, leading to the conclusion that 1D convolutions are the bottleneck for modeling time-frequency structure.
Significance. If the results hold, this is a useful diagnostic contribution to an actively debated question (why time-frequency vocoders lag time-domain ones). Particular strengths worth naming: matched retraining of both vocoders on the same data and recipe, a parameter-matched BigVGANv2-base control, the identification and correction of a concrete bug-level issue (the magnitude clamping threshold, Fig. 2), oracle ablations that yield a genuinely new and falsifiable insight (that Vocos's inconsistent magnitude prediction compensates phase error, implying STFT-consistency regularizers may be counterproductive), a MUSHRA test with corrected significance testing, and publicly posted stimuli. The phase-difference experiment is a clean idea and the 37K-parameter Conv2D model beating a 247K benchmark at ~6× fewer FLOPs is notable. The main limitation is that the causal attribution of the phase-diff failure specifically to 1D convolutions is not isolated by the current ablation design.
major comments (3)
- [§3.2, Table 3] The Conv1D-vs-Conv2D comparison that supports the paper's headline conclusion changes four things simultaneously: convolution dimensionality, channel widths (512/1536 → 32/64), block count (8 → 6), and parameter count (15.0M → 37.1K, a ~400× reduction). The causal attribution to '1D convolutional layers' (abstract) is therefore under-controlled. Live alternatives include (a) the 15M Conv1D backbone being mis-scaled or under-optimized for this task at 400k steps with the vocoding-derived training recipe of Table 1, and (b) depth/width interactions with the wrapping-aware loss. A capacity-matched Conv1D control (e.g., a ~37K–250K parameter Conv1D model with comparable receptive field) is a cheap experiment and is needed before the abstract's identification claim is warranted. Absent that control, the claim should be moderated to the level of the more careful §5.3 wording ('inductive biases
- [§3.2, Fig. 1] An alternative explanation for the Conv1D backbone's failure is the input layer itself: the first Conv1D (k=7) projects 513 log-magnitude channels to 512 feature channels, destroying frequency-bin identity before any convolution operates. The Conv2D variant and the Masuyama benchmark both preserve frequency as a spatial axis end-to-end. The failure may therefore lie in the channel-mixing projection rather than in 1D temporal convolutions per se — a distinction that matters for the paper's prescriptive conclusion about future architectures. A control retaining 1D temporal convolutions but with a frequency-preserving front end (or a Conv2D model with a fully mixing front end) would separate these hypotheses.
- [§3, §5.3] The argument that failure at non-autoregressive phase-difference prediction from ground-truth log-magnitude explains Vocos's end-to-end phase errors rests on an unstated bridge assumption: the two settings differ in input (513-bin magnitude vs 80-bin mel), supervision (wrapping-aware L1 vs adversarial + mel losses), and task (predict differences vs predict wrapped phase jointly with magnitude). It is plausible, and §3 motivates it via the signal model, but the manuscript should state it explicitly as an assumption and ideally provide one corroborating observation — e.g., whether the Conv2D phase-difference model's LSC improvement (−29.56 dB) transfers when its integrated phase is paired with Vocos-predicted magnitudes, or at minimum a discussion of why L_wa/LSC on the subtask is expected to predict LSC in Table 2.
minor comments (6)
- [Table 2] SCOREQ values for the oracle rows (Vocos-Mag: 0.073/0.147; Vocos-Phase: 0.495/0.614) sit outside the conventional MOS-like range; please state the scale/normalization and confirm the values, since Vocos-Mag is described as outperforming 'almost any other experiment' partly on this metric.
- [Table 1, §5.3] Phase-difference models are trained for 400k steps versus 2M for vocoders (Table 1). Please justify the budget or show convergence curves, since under-training the Conv1D model is one of the confounds in the Table 3 comparison.
- [§5.1, Fig. 4] The MUSHRA test uses 16 listeners rating 14 of 20 German utterances; please report listener screening (e.g., reference-identification criterion), confidence intervals in addition to the pairwise significance markers, and whether the female/male split in Fig. 4 shows any systematic condition-by-gender interaction.
- [Eq. (5), Table 2] LSC is defined (Eq. 5) as a phase-reconstruction metric given ground-truth magnitude, but in Table 2 it is computed from the vocoders' own output waveforms (where both magnitude and phase are predicted). A sentence clarifying this dual usage would help, since Vocos-Mag's −29.18 dB is not directly comparable to the end-to-end rows.
- [§2.2, Fig. 2] The magnitude-clamping observation (threshold 100 clipping up to 41% of training samples) is a nice practical finding; please state explicitly whether the 400 threshold is used in all retrained Vocos rows of Table 2, including 'Vocos (official†)'.
- [Eq. (3), References] Eq. (3): state the bin-index convention (m = 0 or 1) for the linear-phase term 2πmH/N. Also check reference [30] (SSW 2019 for a vocoder-comparison paper with these authors seems early) and the line-broken URL of the accompanying website.
Circularity Check
No circularity: purely experimental ablations against external baselines; no derivation reduces to its inputs by construction.
full rationale
The paper’s load-bearing claims are empirical comparisons (Vocos vs. BigVGAN under matched losses/discriminators; Vocos-Mag vs. Vocos-Phase oracle ablations; Conv1D vs. Conv2D phase-difference backbones vs. Masuyama et al.), evaluated with independent metrics (L_mel, SCOREQ, V/UV F1, periodicity RMSE, LSC, MUSHRA, wrapping-aware L_wa). No quantity is defined in terms of the quantity it is said to predict; no parameter is fitted to data and then re-presented as a first-principles prediction; no uniqueness theorem or ansatz is imported via self-citation to force the result. The magnitude-clamp threshold of 400 is an explicit hyper-parameter choice justified by a histogram of training STFT maxima, not a circular fit. Self-citations (e.g., [2]) are peripheral and not load-bearing for the architectural conclusions. Experimental-design confounds in the Conv1D/Conv2D comparison (channel width, depth, parameter count) affect causal attribution strength but are not circularity. The derivation chain is therefore self-contained against external benchmarks.
Assumptions & free parameters
free parameters (3)
- magnitude clamping threshold =
400
- generator loss weights (15 L_mel + L_adv + 2 L_fm) =
15 / 1 / 2
- Conv2D phase-diff channel widths (32, 64) and block count (6) =
6 blocks, (32,64) channels
assumptions (4)
- domain assumption STFT consistency / phase-magnitude coupling (time and frequency gradients of log-magnitude and phase are related)
- domain assumption Baseband phase differences (BPD) are easier for CNNs to model than raw instantaneous frequency because the linear-phase term is removed
- domain assumption Least-squares integration of predicted BPD/FPD yields a usable phase spectrogram for LSC evaluation
- ad hoc to paper Bandlimited 80-bin mel spectrograms (0–8 kHz at 22.05 kHz) are a fair and informative benchmark that accentuates time-frequency vs time-domain gaps
Cite this review
Pith. "Pith review of Revisiting Vocos: That Phasiness Business in Time-Frequency Neural Vocoding." pith.science (2026). https://pith.science/paper/WOVZU4B5
@misc{pith2026260724323,
author = {Pith},
title = {Pith review of: Revisiting Vocos: That Phasiness Business in Time-Frequency Neural Vocoding},
year = {2026},
howpublished = {\url{https://pith.science/paper/WOVZU4B5}},
note = {Machine review of arXiv:2607.24323}
}
read the original abstract
Recently, time-frequency neural vocoders have been approaching the state-of-the-art quality of time-domain neural vocoders. Vocos is a notable example due to its efficiency, but its audio quality lags behind the time-domain vocoders and the reasons remain debated. Thus, in this study, we revisit Vocos from a phase reconstruction perspective. First, we quantify the gap between time-domain and time-frequency domain vocoders using bandlimited mel spectrograms as inputs. Later, via an ablation study, we verify the Vocos architecture is effective for magnitude modeling, but less so for phase. We then adapt the Vocos backbone to predict phase differences, a precursor for phase reconstruction, and identify 1D convolutional layers are hindering their accurate prediction. Our findings indicate that future research needs to focus on inductive biases that allow the architecture to better model the time-frequency structure of speech signals, without sacrificing the support for arbitrary input representations.
Reference graph
Works this paper leans on
-
[1]
They play a key role in speech synthesis, such as in voice conversion and speaker anonymization systems [1, 2]
INTRODUCTION Neural vocoders synthesize time-domain audio signals from lossy representations, typically mel spectrograms. They play a key role in speech synthesis, such as in voice conversion and speaker anonymization systems [1, 2]. In these settings, time-domain neural vocoders can yield high-quality audio signals but are computa- tionally expensive due...
-
[2]
METHODOLOGY (VOCODING) 2.1. Input representation We use 80-band log mel spectrograms (for the frequency range 0– 8 kHz) as input, at a sampling rate of 22.05 kHz. Bandlimited spec- trograms could accentuate the gap between time-domain and time- frequency domain vocoders, since time-frequency domain vocoders are known to struggle with generating harmonic s...
arXiv 2026
-
[3]
METHODOLOGY (PHASE RECONSTRUCTION) In V ocos, the Head is the only block that imposes phase-related in- ductive bias, by (elementwise) wrapping the predicted values to the [−π, π)range. However, spectrograms are characterized by a much stronger inductive bias of consistency, in the form of a coupling be- tween the time and frequency gradients of log-magni...
-
[4]
Datasets For our trainings, we used LibriTTS [27] subsets (train-clean-100, train-clean-360, train-other-500), similar to BigVGAN and V ocos [3, 6]
EXPERIMENTAL SETUP 4.1. Datasets For our trainings, we used LibriTTS [27] subsets (train-clean-100, train-clean-360, train-other-500), similar to BigVGAN and V ocos [3, 6]. This dataset contains approximately 585 h of speech data from 2456 speakers. We employed the random amplitude augmentation as proposed for V ocos, namely, scaling the audio so that the...
-
[5]
RESULTS 5.1. Comparison of vocoders Objective evaluation.We report five objective metrics: the multi- resolution log-mel spectrogram lossL mel, SCOREQ using the ground-truth audio as reference [28], voiced-unvoiced F1 score, periodicity RMSE [9], and log-spectral convergence (LSC) [23] LSC( ˆϕ,A) = 20 log10 ∥A− |STFT(iSTFT(Aej ˆϕ))|∥F ro ∥A∥F ro ! , (5) w...
-
[6]
The results, reported in the bot- tom rows of Table 2, highlight that V ocos-Mag outperforms almost any other experiment in this paper, while V ocos-Phase performs the worst
V ocos-Phase predicts the phase spectrogramˆ p, andˆ mis set to the ground-truth log magnitudes. The results, reported in the bot- tom rows of Table 2, highlight that V ocos-Mag outperforms almost any other experiment in this paper, while V ocos-Phase performs the worst. We draw a number of conclusions from these results: • The V ocos architecture can eas...
-
[7]
not a bug, but a feature
CONCLUSION In this paper, we investigated the limitations of the V ocos architec- ture. We showed that using a bandlimited mel spectrogram as input is an informative benchmark for comparing vocoders, accentuating the gap between time-domain and time-frequency domain vocoders. We found that even after controlling for confounding factors, V ocos still lags ...
-
[8]
How should we extract discrete audio tokens from self-supervised models?
P. Mousavi et al., “How should we extract discrete audio tokens from self-supervised models?” InProc. Interspeech Conf., 2024
2024
Show all 38 references
-
[9]
Why disentanglement- based speaker anonymization systems fail at preserving emo- tions?
¨U. E. Gaznepoglu and N. Peters, “Why disentanglement- based speaker anonymization systems fail at preserving emo- tions?” InProc. IEEE Intl. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2025
2025
-
[10]
BigVGAN: A universal neural vocoder with large-scale training,
S.-g. Lee, W. Ping, B. Ginsburg, B. Catanzaro, and S. Yoon, “BigVGAN: A universal neural vocoder with large-scale training,” inProc. Intl. Conf. on Learning Representations (ICLR), 2023
2023
-
[11]
A streamwise GAN vocoder for wideband speech coding at very low bit rate,
A. Mustafa et al., “A streamwise GAN vocoder for wideband speech coding at very low bit rate,” inProc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2021
2021
-
[12]
Lightweight and high-fidelity end-to-end text-to-speech with multi-band generation and inverse short-time Fourier transform,
M. Kawamura, Y . Shirahata, R. Yamamoto, and K. Tachibana, “Lightweight and high-fidelity end-to-end text-to-speech with multi-band generation and inverse short-time Fourier transform,” inProc. IEEE Intl. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2023
2023
-
[13]
V ocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis,
H. Siuzdak, “V ocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis,” inProc. Intl. Conf. on Learning Representations (ICLR), 2024
2024
-
[14]
WaveNeXt: ConvNeXt-based fast neural vocoder without ISTFT layer,
T. Okamoto, H. Yamashita, Y . Ohtani, T. Toda, and H. Kawai, “WaveNeXt: ConvNeXt-based fast neural vocoder without ISTFT layer,” inProc. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2023
2023
-
[15]
Phase-vocoder: About this phasi- ness business,
J. Laroche and M. Dolson, “Phase-vocoder: About this phasi- ness business,” inProc. Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 1997
1997
-
[16]
Chunked autoregressive GAN for condi- tional waveform synthesis,
M. Morrison et al., “Chunked autoregressive GAN for condi- tional waveform synthesis,” inProc. Intl. Conf. on Learning Representations (ICLR), 2022
2022
-
[17]
Wavehax: Aliasing-free neural waveform synthesis based on 2d convolution and harmonic prior for reliable complex spectrogram estimation,
R. Yoneyama, A. Miyashita, R. Yamamoto, and T. Toda, “Wavehax: Aliasing-free neural waveform synthesis based on 2d convolution and harmonic prior for reliable complex spectrogram estimation,”IEEE Trans. Audio, Speech and Lang. Proc., vol. 33, 2025
2025
-
[18]
Signal estimation from modified short-time Fourier transform,
D. Griffin and J. Lim, “Signal estimation from modified short-time Fourier transform,”IEEE Trans. Audio, Speech and Sig. Proc., vol. 32, no. 2, 1984
1984
-
[19]
FreeV: Free lunch for vocoders through pseudo inversed mel filter,
Y . Lv et al., “FreeV: Free lunch for vocoders through pseudo inversed mel filter,” inProc. Interspeech Conf., 2024
2024
-
[20]
Is GAN necessary for mel-spectrogram-based neural vocoder?
H.-P. Du, Y . Ai, R.-C. Zheng, Y .-X. Lu, and Z.-H. Ling, “Is GAN necessary for mel-spectrogram-based neural vocoder?” IEEE Signal Process. Lett., vol. 32, 2025
2025
-
[21]
Learning neural vocoder from range-null space decomposition,
A. Li et al., “Learning neural vocoder from range-null space decomposition,” inProc. International Joint Conf. on Artifi- cial Intelligence, 2025
2025
-
[22]
A ConvNet for the 2020s,
Z. Liu et al., “A ConvNet for the 2020s,” inProc. IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2022
2022
-
[23]
High-fidelity audio compression with improved RVQGAN,
R. Kumar, P. Seetharaman, A. Luebs, I. Kumar, and K. Kumar, “High-fidelity audio compression with improved RVQGAN,” inProc. Neural Information Processing Sys- tems (NeurIPS), 2023
2023
-
[24]
High fi- delity neural audio compression,
A. D ´efossez, J. Copet, G. Synnaeve, and Y . Adi, “High fi- delity neural audio compression,”Trans. on Machine Learn- ing Research (TMLR), 2023
2023
-
[25]
Multi-scale sub- band constant-q transform discriminator for high-fidelity vocoder,
Y . Gu, X. Zhang, L. Xue, and Z. Wu, “Multi-scale sub- band constant-q transform discriminator for high-fidelity vocoder,” inProc. IEEE Intl. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2024
2024
-
[26]
On phase- magnitude relationships in the short-time Fourier transform,
F. Auger, ´E. Chassande-Mottin, and P. Flandrin, “On phase- magnitude relationships in the short-time Fourier transform,” IEEE Signal Processing Letters, vol. 19, no. 5, 2012
2012
-
[27]
A noniterative method for reconstruction of phase from STFT magnitude,
Z. Pr ˚uˇsa, P. Balazs, and P. L. Søndergaard, “A noniterative method for reconstruction of phase from STFT magnitude,” IEEE/ACM Trans. Audio, Speech and Lang. Proc., vol. 25, no. 5, 2017
2017
-
[28]
Adversarial generation of time-frequency features with ap- plication in audio synthesis,
A. Marafioti, N. Holighaus, N. Perraudin, and P. Majdak, “Adversarial generation of time-frequency features with ap- plication in audio synthesis,” inProc. Intl. Conf. on Machine Learning (ICML), 2019
2019
-
[29]
Mel spectrogram in- version with stable pitch,
B. Di Giorgi, M. Levy, and R. Sharp, “Mel spectrogram in- version with stable pitch,” inProc. Intl. Society for Music Information Retrieval Conference (ISMIR), 2022
2022
-
[30]
Online phase reconstruction via DNN-based phase differ- ences estimation,
Y . Masuyama, K. Yatabe, K. Nagatomo, and Y . Oikawa, “Online phase reconstruction via DNN-based phase differ- ences estimation,”IEEE/ACM Trans. Audio, Speech and Lang. Proc., vol. 31, 2023
2023
-
[31]
Efficient neural and numerical methods for high-quality on- line speech spectrogram inversion via gradient theorem,
A. Fernandez, J. Azcarreta, C ¸ . Bilen, and J. Monge Alvarez, “Efficient neural and numerical methods for high-quality on- line speech spectrogram inversion via gradient theorem,” in Proc. Interspeech Conf., 2025
2025
-
[32]
STFT phase reconstruc- tion in voiced speech for an improved single-channel speech enhancement,
M. Krawczyk and T. Gerkmann, “STFT phase reconstruc- tion in voiced speech for an improved single-channel speech enhancement,”IEEE/ACM Trans. Audio, Speech and Lang. Proc., vol. 22, no. 12, 2014
2014
-
[33]
Low-latency neural speech phase pre- diction based on parallel estimation architecture and anti- wrapping losses for speech generation tasks,
Y . Ai and Z.-H. Ling, “Low-latency neural speech phase pre- diction based on parallel estimation architecture and anti- wrapping losses for speech generation tasks,”IEEE/ACM Trans. on Audio, Speech and Lang. Proc., vol. 32, 2024
2024
-
[34]
LibriTTS: A corpus derived from LibriSpeech for text-to-speech,
H. Zen et al., “LibriTTS: A corpus derived from LibriSpeech for text-to-speech,” inProc. Interspeech Conf., 2019
2019
-
[35]
SCOREQ: Speech quality assessment with contrastive regression,
A. Ragano, J. Skoglund, and A. Hines, “SCOREQ: Speech quality assessment with contrastive regression,” inProc. Neural Information Processing Systems (NeurIPS), 2024
2024
-
[36]
webMUSHRA — a comprehensive framework for web-based listening tests,
M. Schoeffler et al., “webMUSHRA — a comprehensive framework for web-based listening tests,”Journal of Open Research Software, vol. 6, no. 1, 2018
2018
-
[37]
A com- parison of recent neural vocoders for speech signal recon- struction,
P. Govalkar, J. Fischer, F. Zalkow, and C. Dittmar, “A com- parison of recent neural vocoders for speech signal recon- struction,” inProc. ISCA Speech Synthesis Workshop (SSW), 2019
2019
-
[38]
APNet2: High- quality and high-efficiency neural vocoder with direct predic- tion of amplitude and phase spectra,
H.-P. Du, Y .-X. Lu, Y . Ai, and Z.-H. Ling, “APNet2: High- quality and high-efficiency neural vocoder with direct predic- tion of amplitude and phase spectra,” inProc. National Conf. Man-Machine Speech Communication (NCMMSC), 2024
2024
Reviewed July 31, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.