Pith. sign in

REVIEW 5 major objections 4 minor 48 references

Comparative Analysis of Fast and High-Fidelity Neural Vocoders for Low-Latency Streaming Synthesis in Resource-Constrained Environments

T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a multi-stream extension of the Wavehax vocoder delivers the best throughput among compared neural vocoders at sub-80 ms streaming latency on a single CPU, with near-non-causal quality at one frame of lookahead…

desk verdict MS-Wavehax is a genuinely compact, high-quality streaming vocoder, but the central throughput claim is partly confounded by an uneven ONNX STFT replacement across models. read the letter →

arxiv 2506.03554 v1 pith:Y5BH6M2N submitted 2025-06-04 cs.SD eess.AS

Reo Yoneyama , Masaya Kawamura , Ryo Terashima , Ryuichi Yamamoto , Tomoki Toda This is my paper
classification cs.SDeess.AS
keywords neuralvocoderstreamingsynthesislow-latencyaudiomulti-streamdecompositionWavehaxCPU-onlyinferencesubscalereal-timespeech
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a high-fidelity neural vocoder can be made small and fast enough for real-time streaming on a single CPU by decomposing the waveform into sub-bands. It introduces multi-stream Wavehax (MS-Wavehax), which extends the aliasing-free Wavehax vocoder with multi-stream synthesis that splits the signal into four subscale waveforms. The paper claims that MS-Wavehax achieves the highest streaming throughput among the compared vocoders at low latency (chunk sizes under 80 ms) and nearly matches its own non-causal quality with just one frame of lookahead, while using about 2.4% of HiFi-GAN V1's model size. It also identifies why the choice between 1D and 2D convolutions and the streaming chunk size dominate efficiency, through an im2col-based memory-bandwidth analysis. If correct, this gives a practical recipe for deploying neural vocoders on resource-constrained devices without sacrificing audio quality.

What carries the argument

The load-bearing mechanism is multi-stream synthesis with trainable filters combined with streaming adaptations. A signal analysis filter (length 63) decomposes the input waveform into four subscale signals; each subscale goes through STFT with 20 ms frames and a 10 ms shift, producing subscale spectrograms that are concatenated with projected mel-spectrogram features into a 12-channel representation. A series of 2D convolutional layers and residual blocks processes this representation, and a final pointwise convolution outputs eight spectrograms (real and imaginary parts of four subscale signals), which are converted back to the time domain via iSTFT and merged by a synthesis filter. Streaming feasibility comes from ring-buffer caching applied to all convolutional, STFT, and iSTFT layers, and from replacing layer normalization with batch normalization, since batch norm does not require computing statistics over the whole sequence at inference. The throughput analysis works through im2col matrix sizes: 2D convolutions produce input matrices that grow with chunk size T and the frequency dimension, increasing main-memory traffic at larger chunks, while 1D convolutions have large but constant filter matrices that are reused from cache.

What would settle it

Run the same streaming RTF comparison on a low-end mobile CPU (e.g., an ARM Cortex-A series core with small L1/L2 caches and no large shared L3) using the same ONNX Runtime pipeline and chunk sizes of 1, 4, 8, and 16 frames. If Vocos already matches or beats MS-Wavehax at chunk sizes below 8 frames, or if MS-Wavehax fails to keep the lowest RTF below 80 ms latency, then the paper's central 'highest throughput under low-latency' claim does not generalize beyond the tested EPYC configuration. A second check: switch the convolution backend from im2col-based GeMM to direct convolution on the same EPYC CPU and see whether the RTF crossover between MS-Wavehax and Vocos shifts or disappears, which would confirm or falsify the proposed memory-bandwidth mechanism.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that multi-stream decomposition of the harmonic-prior-based Wavehax architecture yields a vocoder that is both subjectively high-quality and uniquely suited to low-latency CPU streaming. MS-Wavehax splits the input waveform with an analysis filter into four subscale signals, converts each to a complex spectrogram, concatenates them with mel-spectrogram features, processes them with 2D convolutions, and recombines the streams via a synthesis filter. In listening tests for both analysis-synthesis and text-to-speech, non-causal MS-Wavehax and causal MS-Wavehax with one-frame lookahead score highest among the vocoders. With 0.332 million parameters and about 1.576 GMACs per second of audio in PyTorch, it achieves the best real-time factor at small chunk sizes and requires only 2.4% of HiFi-GAN V1's model size. The paper further shows that as chunk size grows, memory transfer in 2D convolutions becomes the bottleneck, which is why the small 2D-convolution-based MS-Wavehax wins at low latency while the 1D-convolution-based Vocos wins at large chunk sizes.

Load-bearing premise

The central throughput claim assumes that the im2col-based memory-bandwidth behavior measured on a single AMD EPYC 7302 CPU with ONNX Runtime generalizes to other resource-constrained devices; if a target device's cache hierarchy or convolution backend changes the balance between data transfer and compute, the reported throughput ranking and recommended chunk sizes could change.

Editorial extensions

If this is right

  • For chunk sizes corresponding to less than 80 ms of additional latency, MS-Wavehax gives the lowest real-time factor among the tested vocoders in a single-thread CPU environment, making it suitable for interactive real-time applications.
  • With one frame (10 ms) of lookahead, causal MS-Wavehax matches its non-causal quality in listening tests, indicating that near-zero-future-context streaming is feasible without audible degradation.
  • The model size of MS-Wavehax is about 2.4% of HiFi-GAN V1's parameter count (0.332M versus 13.82M), so it can fit into on-device memory budgets where larger vocoders are impractical.
  • The im2col memory-bandwidth analysis predicts that 2D-convolution vocoders lose throughput relative to 1D-convolution vocoders as chunk size grows, providing a concrete design rule for choosing streaming chunk sizes.
  • Replacing layer normalization with batch normalization and adding caching does not harm quality and, in the paper's internal comparisons, even improves objective metrics.
  • The relationship between latency and throughput depends on hardware cache hierarchy and memory bandwidth, so the optimal chunk size and vocoder architecture should be chosen according to the target device.
  • The paper's analysis is performed on a single AMD EPYC 7302 CPU using ONNX Runtime, and the practical recommendations assume this behavior generalizes to other resource-constrained CPUs and inference backends.
  • The multi-stream decomposition's benefit may extend beyond efficiency: partitioning the spectrogram into smaller sub-spectrograms makes spectral structures more locally consistent, which the paper argues improves how 2D convolutional kernels capture dependencies.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes multi-stream Wavehax (MS-Wavehax), a compact neural vocoder that extends Wavehax with four-stream subscale synthesis using trainable analysis/synthesis filters, and also introduces a multi-stream version of Vocos. The authors evaluate streaming synthesis on a single CPU with ONNX Runtime, reporting real-time factors (RTF) as a function of chunk size, and present a theoretical im2col-based analysis of why 1D ConvNeXt-based Vocos overtakes 2D Wavehax variants at larger chunk sizes. Subjective MOS evaluations in analysis-synthesis and text-to-speech tasks are reported for the baseline and multi-stream models, including causal MS-Wavehax with and without one-frame lookahead. The central claims are that MS-Wavehax offers high speech quality under causal and non-causal conditions, achieves the highest throughput at low latency (<80 ms), and requires only 2.4% of HiFi-GAN V1's parameter count.

Significance. If the central claims hold, this is a practically valuable result: a vocoder of 0.332M parameters that nearly matches non-causal quality with one frame of lookahead would be directly useful for on-device streaming TTS and voice conversion. The paper also provides a useful methodological discussion of how streaming overhead, parameter loading, and cache behavior interact with 1D versus 2D convolution design, and the im2col matrix-size analysis in Section 3.2 and Table 2 is a clear, concrete contribution. The experimental setup uses public corpora (JVS, JSUT), a standard MOS protocol with 30 listeners, and detailed hyperparameters, which are strengths. However, as detailed below, the throughput comparison is confounded by the ONNX STFT/iSTFT replacement, and the subjective comparison lacks causal baselines and statistical testing, so the main claims are not yet fully established.

major comments (5)
  1. [Section 3.2, Table 1, Figure 2] The central throughput claim is confounded by the ONNX STFT/iSTFT replacement. Table 1 shows that replacing torch.stft/torch.istft with convolution-based implementations changes per-second MACs from 1.348B to 46.96B for Vocos (a 35x increase), from 1.787B to 13.03B for Wavehax (a 7.3x increase), and from 1.576B to 2.291B for MS-Wavehax (a 1.45x increase). Since Figure 2 reports RTF measured with ONNX Runtime, the ranking at small chunk sizes may reflect the relative overhead of the conv-based STFT implementation rather than the multi-stream architecture or the 2D-versus-1D convolution behavior analyzed in the im2col discussion. The manuscript does not provide a per-layer RTF breakdown or an ablation that holds the STFT implementation fixed across all models. To support the claim that MS-Wavehax has the highest streaming throughput, the authors should either report RTF with a common STFT/iSTFT implementation (e.g., native PyTorch or a backend optimized for all models), or explicitly isolate and compare the STFT overhead per model.
  2. [Section 4.2, Figure 4] The MOS evaluation does not include causal baselines. The non-causal set includes HiFi-GAN, iSTFTNet, Vocos, and Wavehax, but the only causal models evaluated are MS-Wavehax with lookahead 0 and 1. Thus the claim that MS-Wavehax 'delivers high speech quality under causal conditions' cannot be compared against causal variants of any other vocoder. For example, a causal HiFi-GAN or causal Vocos would be a natural baseline to determine whether the quality gap under causality is specific to MS-Wavehax or a general property of causal streaming. The authors should add at least one strong causal baseline or soften the claim accordingly.
  3. [Section 4.2, Figure 4] No statistical significance testing is reported for the MOS results. The figure shows 95% confidence intervals, but with 30 listeners and 10 samples per model, overlapping intervals do not establish equivalent or superior quality. The manuscript states that MS-Wavehax 'achieved the highest MOS in both tasks' and that 'the performance gap between MS-Wavehax with LA=1 and LA=0 disappeared in TTS,' but these conclusions are not supported by any pairwise significance test (e.g., matched bootstrap, Wilcoxon, or ANOVA with a post-hoc test). The authors should add an appropriate significance test or explicitly present the comparison as descriptive rather than inferential.
  4. [Section 2.2 and Section 2.3] Two load-bearing design choices are justified only by 'internal experiments' that are never shown: the selection of trainable analysis/synthesis filters over PQMF and DWT, and the replacement of layer normalization with batch normalization. No objective metric values, architectures, or ablations are provided for these choices, which makes the proposed method difficult to reproduce and means the reader cannot assess whether these choices are robust or tuned to the specific corpora used. The authors should report the numerical results of these internal experiments or omit the claims and treat the choices as fixed design decisions.
  5. [Section 3.2] The generalizability of the throughput analysis to 'resource-constrained environments' is supported by measurements on a single AMD EPYC 7302 CPU with a 16 MiB L3 cache, which is a server-class processor rather than a typical edge or mobile CPU. The paper acknowledges cache and SIMD dependencies in the discussion, but no data is provided for a range of hardware (e.g., ARM mobile cores, lower-power x86, or devices with smaller caches). The practical recommendations about chunk sizes and 1D vs 2D convolution design should either be explicitly scoped to this one CPU or validated on at least one additional representative device.
minor comments (4)
  1. [Section 2.2] The sentence 'These spectrograms are then concatenated with the projected mel-spectrogram features, resulting in a 12-channel representation' appears twice verbatim; one copy should be removed.
  2. [Section 3.2 and Table 1] The table caption lists 'MACs' for PyTorch and ONNX, but the figure and text never clarify whether the PyTorch numbers use the native torch.stft/torch.istft or the same convolution-based replacement; this should be stated explicitly for both columns.
  3. [Section 4.2] The explanation for MS-Vocos underperforming Vocos is speculative ('We speculate that estimating inter-stream dependencies is particularly challenging...') and is not tested. If this claim is retained, it would benefit from a simple diagnostic, such as varying the latent channel count in MS-Vocos.
  4. [General] There are several typographical artifacts, such as 'V ocoder' in the title and repeated 'V ocos' spacing in the references; a final proofreading pass would be helpful.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central throughput and quality claims are empirical comparisons against external baselines and corpora; self-citations are architectural precedents, not load-bearing evidence.

full rationale

The paper's derivation chain is empirical rather than definitional. MS-Wavehax's low-latency throughput is measured via ONNX Runtime RTF experiments (Section 3.2, Figure 2), its compactness is reported as measured parameter counts and MACs (Table 1), and its quality claims rest on listener MOS evaluations (Section 4.2, Figure 4). No fitted constant is renamed as a prediction, and no equation reduces the claimed result to its own inputs by construction. The im2col analysis (Table 2, Figure 3) is an explanatory account of the observed RTF reversal, not a derivation that forces the outcome. The paper does cite prior work by overlapping authors: Wavehax [12] and multi-stream variants [13, 27]. These self-citations establish the architectural lineage from which MS-Wavehax is built, but the paper's new claims are not justified solely by those citations; they are supported by new measurements against external baselines (HiFi-GAN, iSTFTNet, Vocos) and external corpora (JVS, JSUT). The Table 1 note that torch.stft/torch.istft were replaced by convolution-based implementations for ONNX conversion is a potential confound for the cross-model throughput comparison, since MACs change disproportionately across models (e.g., Vocos 1.348B to 46.96B vs. MS-Wavehax 1.576B to 2.291B). This is a validity or generalization concern, not circularity: the reported RTF ranking may depend on the STFT implementation choice, but nothing in the argument assumes its own conclusion by construction. Section 3.2's footnote also acknowledges that the im2col analysis assumes a particular backend and that other implementations may behave differently. These caveats lower confidence in the generality of the central claim but do not make the claim circular. Overall, the paper is self-contained against external benchmarks, so the appropriate finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. Its claims rest on empirical ML evaluation; the main ledger entries are untested design choices and internal experiment claims that are not shown.

free parameters (4)
  • Number of subscale streams = 4
    Chosen based on internal experiments; not tuned per target. Affects model size and throughput.
  • MS-Wavehax STFT frame length/frame shift = 20 ms / 10 ms
    Design choice in Figure 1; affects spectral resolution and causal latency.
  • Analysis/synthesis filter length = 63
    Figure 1 caption; hand-chosen filter length.
  • Hidden channels in MS-Wavehax = 64
    Twice Wavehax's channels while frequency dimension is reduced; design choice.
assumptions (4)
  • domain assumption Trainable subscale filters outperform PQMF and DWT for the tested vocoders
    Section 2.2 states internal experiments, but no results are shown. MS-Wavehax's quality relies on this choice.
  • domain assumption Batch normalization is a suitable replacement for layer normalization in streaming and improves objective metrics
    Section 2.3 states this from internal experiments; not independently verified or shown.
  • domain assumption ONNX Runtime with im2col-based convolutions is representative of streaming inference on resource-constrained CPUs
    Section 3.2 and footnotes 1-2; the throughput analysis is based on this specific backend and hardware.
  • domain assumption MOS from 30 listeners on 10 samples per model can distinguish model quality
    Section 4.2; no significance tests are reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Comparative Analysis of Fast and High-Fidelity Neural Vocoders for Low-Latency Streaming Synthesis in Resource-Constrained Environments." pith.science (2026). https://pith.science/paper/Y5BH6M2N

@misc{pith2026250603554,
  author       = {Pith},
  title        = {Pith review of: Comparative Analysis of Fast and High-Fidelity Neural Vocoders for Low-Latency Streaming Synthesis in Resource-Constrained Environments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y5BH6M2N}},
  note         = {Machine review of arXiv:2506.03554}
}
read the original abstract

In real-time speech synthesis, neural vocoders often require low-latency synthesis through causal processing and streaming. However, streaming introduces inefficiencies absent in batch synthesis, such as limited parallelism, inter-frame dependency management, and parameter loading overhead. This paper proposes multi-stream Wavehax (MS-Wavehax), an efficient neural vocoder for low-latency streaming, by extending the aliasing-free neural vocoder Wavehax with multi-stream decomposition. We analyze the latency-throughput trade-off in a CPU-only environment and identify key bottlenecks in streaming neural vocoders. Our findings provide practical insights for optimizing chunk sizes and designing vocoders tailored to specific application demands and hardware constraints. Furthermore, our subjective evaluations show that MS-Wavehax delivers high speech quality under causal and non-causal conditions while being remarkably compact and easily deployable in resource-constrained environments.

Figures

Figures reproduced from arXiv: 2506.03554 by the authors.

Figure 1
Figure 1. An overview of multi-stream Wavehax. The harmonic prior waveform is generated from the input fundamental fre￾quencies via signal processing. The lengths of the analysis and synthesis filters are both 63. The frame length and frame shift for STFT/iSTFT are 20 ms and 10 ms, respectively. The num￾bers on the right represent the tensor shapes at each processing stage, where C, F, T, and t denote the number of channels, … view at source ↗
Figure 2
Figure 2. Real-time factors for batch and streaming synthesis, measured using ONNX Runtime on a single thread of an AMD EPYC 7302 processor (maximum clock speed: 3 GHz). The results were averaged over 30 runs, each lasting 10 seconds, following 5 initial warm-up iterations. The chunk size T is the number of time frames generated per synthesis step. For exam￾ple, a chunk size of 8 introduces an additional latency of 70 ms rela… view at source ↗
Figure 4
Figure 4. presents the MOS on speech quality in the A/S and TTS tasks. We evaluated non-causal HiFi-GAN, iSTFTNet, Vo￾cos, and Wavehax, along with their multi-stream variants, as well as causal MS-Wavehax with or without one-frame looka￾head (LA) [9, 10, 41]. As illustrated, non-causal MS-Wavehax and causal MS-Wavehax with LA achieved the highest MOS in both tasks. For TTS, differences among all methods, including natural spe… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 44 canonical work pages

  1. [1]

    One key component of this progress is neural vocoders, which synthesize audio waveforms from acoustic fea- tures

    Introduction Advances in deep learning have led to remarkable improve- ments in speech synthesis, enabling highly natural speech generation. One key component of this progress is neural vocoders, which synthesize audio waveforms from acoustic fea- tures. Specifically, neural vocoders based on generative adver- sarial networks [1], such as MelGAN [2] and H...

    work page

  2. [2]

    Comparative Analysis of Fast and High-Fidelity Neural Vocoders for Low-Latency Streaming Synthesis in Resource-Constrained Environments

    Proposed method This section introduces V ocos [7] and Wavehax [12], along with our proposed MS-V ocos and MS-Wavehax, which inte- grate multi-stream synthesis [13]. Additionally, we elaborate on techniques to facilitate streaming feasibility. 2.1. Overview of V ocos and Wavehax V ocos repurposes the ConvNeXt [20] model, originally devel- oped for image p...

    work page Pith review arXiv 2025

  3. [3]

    throughput We analyze the relationship between latency and throughput via block streaming synthesis using several neural vocoders

    Analysis of latency vs. throughput We analyze the relationship between latency and throughput via block streaming synthesis using several neural vocoders. 3.1. Model details We compared four neural vocoders, HiFi-GAN (V1) [3], iSTFT- Net (V1) [4], V ocos [7], and Wavehax [12], along with their multi-stream variants, each configured with four streams. All ...

    work page

  4. [4]

    Wavehax and MS-Wavehax utilized F0 for generating prior signals, whereas other models concate- nate it with the mel-spectrogram, resulting in a 101-dimensional input feature

    All models assumed a 100-dimensional mel-spectrogram and a 1-dimensionalF 0 input. Wavehax and MS-Wavehax utilized F0 for generating prior signals, whereas other models concate- nate it with the mel-spectrogram, resulting in a 101-dimensional input feature. The sampling rate and frame shift are fixed at 24 kHz and 10 ms, respectively. Each model was adapt...

    work page

  5. [5]

    Next, we evaluate its speech quality under causal and non-causal condi- tions, compared to the vocoders described in Section 3.1

    Speech quality evaluation In the experiments in Section 3, MS-Wavehax achieved the highest throughput under low-latency conditions. Next, we evaluate its speech quality under causal and non-causal condi- tions, compared to the vocoders described in Section 3.1. 1Although our analysis focuses on a particular scenario where backend-level optimizations depen...

    work page

  6. [6]

    Acknowledgements This work was supported in part by JST AIP Acceleration Re- search JPMJCR25U5, Japan, and in part by the Japan Society for the Promotion of Science (JSPS) Grants-in-Aid for Scien- tific Research (KAKENHI) under Grant 24KJ1236

    work page

  7. [7]

    Our analysis revealed that streaming throughput de- pends on overhead from data and parameter loading as well as computational complexity

    Conclusion This study examined neural vocoders for streaming applica- tions. Our analysis revealed that streaming throughput de- pends on overhead from data and parameter loading as well as computational complexity. This overhead varies with the streaming unit sizes and the choice of 1D and 2D convolutions. These findings provide insights into designing s...

    work page

  8. [9]

    Generative Ad- versarial Nets,

    I. Goodfellow, J. Pouget-Abadie, M. Mirzaet al., “Generative Ad- versarial Nets,” inProc. NeurIPS, vol. 27, 2014, pp. 2672–2680

    work page 2014

Show all 48 references
  1. [10]

    MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis,

    K. Kumar, R. Kumar, T. de Boissiereet al., “MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis,” in Proc. NeurIPS, 2019, pp. 14 910–14 921

    2019

  2. [11]

    HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis,

    J. Kong, J. Kim, and J. Bae, “HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis,” in Proc. NeurIPS, vol. 33, 2020, pp. 17 022–17 033

    2020

  3. [12]

    iSTFTNet: Fast and Lightweight Mel-Spectrogram V ocoder Incorporating Inverse Short-Time Fourier Transform,

    T. Kaneko, K. Tanaka, H. Kameokaet al., “iSTFTNet: Fast and Lightweight Mel-Spectrogram V ocoder Incorporating Inverse Short-Time Fourier Transform,” inProc. ICASSP, 2022, pp. 6207–6211

    2022

  4. [13]

    iSTFTNet2: Faster and More Lightweight iSTFT-Based Neural V ocoder Using 1D- 2D CNN,

    T. Kaneko, H. Kameoka, K. Tanakaet al., “iSTFTNet2: Faster and More Lightweight iSTFT-Based Neural V ocoder Using 1D- 2D CNN,” inProc. Interspeech, 2023, pp. 4369–4373

    2023

  5. [14]

    APNet: An All-Frame-Level Neural V ocoder Incorporating Direct Prediction of Amplitude and Phase Spectra,

    Y . Ai and Z.-H. Ling, “APNet: An All-Frame-Level Neural V ocoder Incorporating Direct Prediction of Amplitude and Phase Spectra,”IEEE/ACM TASLP, 2023

    2023

  6. [15]

    V ocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis,

    H. Siuzdak, “V ocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis,” inProc. ICLR, 2024

    2024

  7. [16]

    AC-VC: Non-Parallel Low La- tency Phonetic Posteriorgrams Based V oice Conversion,

    D. Ronssin and M. Cernak, “AC-VC: Non-Parallel Low La- tency Phonetic Posteriorgrams Based V oice Conversion,” inProc. ASRU, 2021, pp. 710–716

    2021

  8. [17]

    Low-latency real-time non-parallel voice conversion based on cyclic variational autoencoder and multiband WaveRNN with data-driven linear prediction,

    P. L. Tobing and T. Toda, “Low-latency real-time non-parallel voice conversion based on cyclic variational autoencoder and multiband WaveRNN with data-driven linear prediction,” inProc. SSW, 2021, pp. 142–147

    2021

  9. [18]

    An Investigation of Streaming Non-Autoregressive sequence-to-sequence V oice Con- version,

    T. Hayashi, K. Kobayashi, and T. Toda, “An Investigation of Streaming Non-Autoregressive sequence-to-sequence V oice Con- version,” inProc. ICASSP, 2022, pp. 6802–6806

    2022

  10. [19]

    Streaming non-autoregressive model for any-to-many voice conversion,

    Z. Chen, H. Miao, and P. Zhang, “Streaming non-autoregressive model for any-to-many voice conversion,”arXiv:2206.07288, 2022

    2022 arXiv

  11. [20]

    Wavehax: Aliasing-Free Neural Waveform Synthesis Based on 2D Convo- lution and Harmonic Prior for Reliable Complex Spectrogram Es- timation,

    R. Yoneyama, A. Miyashita, R. Yamamotoet al., “Wavehax: Aliasing-Free Neural Waveform Synthesis Based on 2D Convo- lution and Harmonic Prior for Reliable Complex Spectrogram Es- timation,”arXiv:2411.06807, 2024

    2024

  12. [21]

    Multi-Stream HiFi-GAN with Data-Driven Waveform Decomposition,

    T. Okamoto, T. Toda, and H. Kawai, “Multi-Stream HiFi-GAN with Data-Driven Waveform Decomposition,” inProc. ASRU, 2021, pp. 610–617

    2021

  13. [22]

    Implementation of DNN-based real-time voice conversion and its improvements by audio data augmentation and mask-shaped device,

    R. Arakawa, S. Takamichi, and H. Saruwatari, “Implementation of DNN-based real-time voice conversion and its improvements by audio data augmentation and mask-shaped device,” inProc. SSW, 2019, pp. 93–98

    2019

  14. [23]

    Real-Time, Full-Band, Online DNN-Based V oice Conversion System Using a Single CPU,

    T. Saeki, Y . Saito, S. Takamichiet al., “Real-Time, Full-Band, Online DNN-Based V oice Conversion System Using a Single CPU,” inProc. Interspeech, 2020, pp. 1021–1022

    2020

  15. [24]

    Fregrad: Lightweight and Fast Frequency-Aware Diffusion V ocoder,

    T. D. Nguyen, J.-H. Kim, Y . Janget al., “Fregrad: Lightweight and Fast Frequency-Aware Diffusion V ocoder,” inProc. ICASSP, 2024, pp. 10 736–10 740

    2024

  16. [25]

    Incremental Text-to-Speech Syn- thesis with Prefix-to-Prefix Framework,

    M. Ma, B. Zheng, K. Liuet al., “Incremental Text-to-Speech Syn- thesis with Prefix-to-Prefix Framework,” inProc. EMNLP, 2020, pp. 3886–3896

    2020

  17. [26]

    Neural iTTS: Toward Synthesizing Speech in Real-time with End-to-end Neural Text- to-Speech Framework,

    T. Yanagita, S. Sakti, and S. Nakamura, “Neural iTTS: Toward Synthesizing Speech in Real-time with End-to-end Neural Text- to-Speech Framework,” inProc. SSW, 2019, pp. 183–188

    2019

  18. [27]

    High Qual- ity Streaming Speech Synthesis with Low, Sentence-Length- Independent Latency,

    N. Ellinas, G. Vamvoukakis, K. Markopouloset al., “High Qual- ity Streaming Speech Synthesis with Low, Sentence-Length- Independent Latency,” inProc. Interspeech, 2020, pp. 2022–2026

    2020

  19. [28]

    A ConvNet for the 2020s,

    Z. Liu, H. Mao, C. Wuet al., “A ConvNet for the 2020s,” inProc. CVPR, 2022, pp. 11 966–11 976

    2022

  20. [29]

    Design and evaluation of parallel quadrature mirror filters (PQMF),

    C. Galand and D. Esteban, “Design and evaluation of parallel quadrature mirror filters (PQMF),” inProc. ICASSP, vol. 8, 1983, pp. 224–227

    1983

  21. [30]

    Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speech,

    G. Yang, S. Yang, K. Liuet al., “Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speech,” inProc. SLT, 2021

    2021

  22. [31]

    Fre-GAN: Adversar- ial Frequency-Consistent Audio Synthesis,

    J.-H. Kim, S.-H. Lee, J.-H. Leeet al., “Fre-GAN: Adversar- ial Frequency-Consistent Audio Synthesis,” inProc. Interspeech, 2021, pp. 2197–2201

    2021

  23. [32]

    Harvest: A High-Performance Fundamental Fre- quency Estimator from Speech Signals,

    M. Morise, “Harvest: A High-Performance Fundamental Fre- quency Estimator from Speech Signals,” inProc. Interspeech, 2017, pp. 2321–2325

    2017

  24. [33]

    Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assess- ment of telephone networks and codecs,

    A. Rix, J. Beerends, M. Hollieret al., “Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assess- ment of telephone networks and codecs,” inProc. ICASSP, vol. 2, 2001, pp. 749–752 vol.2

    2001

  25. [34]

    UTMOS: UTokyo-SaruLab System for V oiceMOS Challenge 2022,

    T. Saeki, D. Xin, W. Nakataet al., “UTMOS: UTokyo-SaruLab System for V oiceMOS Challenge 2022,” inProc. Interspeech, 2022, pp. 4521–4525

    2022

  26. [35]

    Lightweight and High-Fidelity End-to-End Text-to-Speech with Multi-Band Generation and Inverse Short-Time Fourier Transform,

    M. Kawamura, Y . Shirahata, R. Yamamotoet al., “Lightweight and High-Fidelity End-to-End Text-to-Speech with Multi-Band Generation and Inverse Short-Time Fourier Transform,” inProc. ICASSP, 2023, pp. 1–5

    2023

  27. [36]

    Aggregate

    (Mel-model), which combines a Transformer encoder [37] with a decoder based on optimal-transport conditional flow matching [38]. We used the official implementation [39] and set the mel-spectrogram hyperparameters to match those of the A/S task. We developed anF 0 prediction m...

  28. [37]

    Developing Real-Time Stream- ing Transformer Transducer for Speech Recognition on Large- Scale Dataset,

    X. Chen, Y . Wu, Z. Wanget al., “Developing Real-Time Stream- ing Transformer Transducer for Speech Recognition on Large- Scale Dataset,” inProc. ICASSP, 2021, pp. 5904–5908

    2021

  29. [38]

    Layer Normalization,

    J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer Normalization,” arXiv:1607.06450, 2016

    2016 arXiv

  30. [39]

    Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift,

    S. Ioffe and C. Szegedy, “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift,” in Proc. ICML, 2015, pp. 448–456

    2015

  31. [40]

    JVS corpus: free Japanese multi-speaker voice corpus,

    S. Takamichi, K. Mitsui, Y . Saitoet al., “JVS corpus: free Japanese multi-speaker voice corpus,”arXiv:1908.06248, 2019

    1908 arXiv

  32. [41]

    BigVGAN: A Universal Neural V ocoder with Large-Scale Training,

    S. gil Lee, W. Ping, B. Ginsburget al., “BigVGAN: A Universal Neural V ocoder with Large-Scale Training,” inProc. ICLR, 2023

    2023

  33. [42]

    UnivNet: A Neural V ocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generation,

    W. Jang, D. Lim, J. Yoonet al., “UnivNet: A Neural V ocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generation,” inProc. Interspeech, 2021, pp. 2207– 2211

    2021

  34. [43]

    JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthe- sis,

    R. Sonobe, S. Takamichi, and H. Saruwatari, “JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthe- sis,”arXiv:1711.00354, 2017

    2017 arXiv

  35. [44]

    Matcha-TTS: A Fast TTS Architecture with Conditional Flow Matching,

    S. Mehta, R. Tu, J. Beskowet al., “Matcha-TTS: A Fast TTS Architecture with Conditional Flow Matching,” inProc. ICASSP, 2024, pp. 11 341–11 345

    2024

  36. [45]

    Attention is All you Need,

    A. Vaswani, N. Shazeer, N. Parmaret al., “Attention is All you Need,” inProc. NeurIPS, vol. 30, 2017

    2017

  37. [46]

    Flow Matching for Generative Modeling,

    Y . Lipman, R. T. Q. Chen, H. Ben-Hamuet al., “Flow Matching for Generative Modeling,” inProc. ICLR, 2023

    2023

  38. [47]

    Matcha-tts

    shivammehta25. Matcha-tts. [Online]. Available: https://github. com/shivammehta25/Matcha-TTS

  39. [48]

    jsut-label

    sarulab-speech. jsut-label. [Online]. Available: https://github. com/sarulab-speech/jsut-label

  40. [49]

    What the Future Brings: Investigating the Impact of Lookahead for Incremental Neural TTS,

    B. Stephenson, L. Besacier, L. Girinet al., “What the Future Brings: Investigating the Impact of Lookahead for Incremental Neural TTS,” inProc. Interspeech, 2020, pp. 215–219

    2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.