WavTTS is the first raw-waveform diffusion TTS model using DiT flow matching and multi-scale mel supervision that approaches SOTA latent zero-shot performance while beating prior end-to-end models.
hub Mixed citations
DiffWave: A Versatile Diffusion Model for Audio Synthesis
Mixed citation behavior. Most common role is background (67%).
43
Pith papers citing it
121
external citations · Pith
Background
67% of classified citations
abstract
In this work, we propose DiffWave, a versatile diffusion probabilistic model for conditional and unconditional waveform generation. The model is non-autoregressive, and converts the white noise signal into structured waveform through a Markov chain with a constant number of steps at synthesis. It is efficiently trained by optimizing a variant of variational bound on the data likelihood. DiffWave produces high-fidelity audios in different waveform generation tasks, including neural vocoding conditioned on mel spectrogram, class-conditional generation, and unconditional generation. We demonstrate that DiffWave matches a strong WaveNet vocoder in terms of speech quality (MOS: 4.44 versus 4.43), while synthesizing orders of magnitude faster. In particular, it significantly outperforms autoregressive and GAN-based waveform models in the challenging unconditional generation task in terms of audio quality and sample diversity from various automatic and human evaluations.
hub tools
citation-role summary
background 6
baseline 1
dataset 1
other 1
citation-polarity summary
representative citing papers
Flux Matching generalizes score-based generative modeling by using a weaker objective that admits infinitely many non-conservative vector fields with the data as stationary distribution, enabling new design choices beyond traditional score matching.
Consistency models achieve fast one-step generation with SOTA FID of 3.55 on CIFAR-10 and 6.20 on ImageNet 64x64 by directly mapping noise to data, outperforming prior distillation techniques.
Introduces an SDE-based framework for score-based generative modeling that unifies prior methods, enables predictor-corrector sampling and neural ODE likelihoods, and achieves SOTA unconditional image generation on CIFAR-10.
MM-SOLD is a training-free particle sampler whose large-particle limit converges to a moment-matched Gibbs distribution obtained by exponentially tilting a score-smoothed target.
MelShield adds keyed low-energy spread-spectrum perturbations to Mel-spectrograms inside TTS pipelines before vocoding to enable robust extraction of user-specific attribution signals even after compression or noise.
LatentFT uses latent-space Fourier transforms and frequency masking in diffusion autoencoders to enable timescale-specific manipulation of musical structure in generative models.
SynHAT uses a novel two-stage spatio-temporal diffusion framework with Latent Spatio-Temporal U-Net to synthesize realistic human activity traces, outperforming baselines by 52% on spatial and 33% on temporal metrics across four cities.
Shortcut models enable high-quality single or few-step sampling in diffusion models with one network and training phase by conditioning on desired step size.
Diffusion models with architecture improvements and classifier guidance achieve superior FID scores to GANs on unconditional and conditional ImageNet image synthesis.
Causal Spectral Policy decomposes actions spectrally into coarse motion from obs/language and conditional fine corrections, outperforming baselines on precision manipulation tasks.
Proves global GD convergence on reverse Fisher divergence for GMM score matching to single-Gaussian targets from arbitrary init and to separated GMM targets under random init.
DiffErase removes black-box audio watermarks via diffusion priors by adding intermediate noise and regenerating with a pretrained model, preserving quality across audio domains.
EventSpeech is a text-conditioned neural framework that uses neuromorphic event cameras and a new EVT-SPK benchmark to generate expressive speech, claiming to outperform RGB baselines by preserving fine-grained emotions without motion blur.
PrismFlow augments flow matching with residual dynamical experts and a winner-take-all objective to reduce spectral distortion and improve mode coverage in time-series generation.
WavFlow performs direct waveform audio generation via flow matching on 2D token grids from raw patches plus amplitude lifting, matching latent-based methods on VGGSound and AudioCaps without intermediate compression.
Dual-Rate Diffusion interleaves sparse heavy context encoding with frequent light denoising to cut diffusion sampling cost by 2-4x on ImageNet while matching baseline quality and remaining compatible with distillation.
DAD4TS trains a diffusion-based generator jointly with a forecaster under RL control and geometric projections to produce augmentation samples that boost accuracy on small-scale time-series data, with validation reported on five of six real-world datasets.
DiRotQ uses PCA-based rotation-aware activation quantization combined with GPTQ to achieve better FID and PSNR in 4-bit diffusion transformers than prior methods like SVDQuant.
DSL provides a continuous embedding framework where one denoiser supports a family of SNR paths for discrete sequences, improving MAUVE scores on OpenWebText and allowing random-order and hybrid sampling from a fine-tuned MDLM checkpoint.
TrajDLM applies block diffusion language models to discrete road-segment sequences with topology constraints to generate realistic trajectories up to 2.8 times faster than prior methods while supporting zero-shot transfer.
DiffATS trains diffusion models directly on aligned Tucker tensor primitives that are proven to be homeomorphisms, delivering efficient unconditional and conditional generation across images, videos, and PDE data with high compression.
Anisotropic SPDEs preserve geometric data structure over longer timescales in score-based generative modeling, yielding better image quality than standard SDE baselines and flow matching in unconditional and conditional tasks.
SDFlow learns a global transport map via similarity-driven flow matching in VQ latent space, using low-rank manifold decomposition and a categorical posterior to handle discreteness, yielding SOTA long-horizon performance and inference speedups.
citing papers explorer
-
WavTTS: Towards High-Quality Zero-Shot TTS via Direct Raw Waveform Modeling
WavTTS is the first raw-waveform diffusion TTS model using DiT flow matching and multi-scale mel supervision that approaches SOTA latent zero-shot performance while beating prior end-to-end models.
-
Generative Modeling with Flux Matching
Flux Matching generalizes score-based generative modeling by using a weaker objective that admits infinitely many non-conservative vector fields with the data as stationary distribution, enabling new design choices beyond traditional score matching.
-
Consistency Models
Consistency models achieve fast one-step generation with SOTA FID of 3.55 on CIFAR-10 and 6.20 on ImageNet 64x64 by directly mapping noise to data, outperforming prior distillation techniques.
-
Score-Based Generative Modeling through Stochastic Differential Equations
Introduces an SDE-based framework for score-based generative modeling that unifies prior methods, enables predictor-corrector sampling and neural ODE likelihoods, and achieves SOTA unconditional image generation on CIFAR-10.
-
Training-Free Generative Sampling via Moment-Matched Score Smoothing
MM-SOLD is a training-free particle sampler whose large-particle limit converges to a moment-matched Gibbs distribution obtained by exponentially tilting a score-smoothed target.
-
MelShield: Robust Mel-Domain Audio Watermarking for Provenance Attribution of AI Generated Synthesized Speech
MelShield adds keyed low-energy spread-spectrum perturbations to Mel-spectrograms inside TTS pipelines before vocoding to enable robust extraction of user-specific attribution signals even after compression or noise.
-
Latent Fourier Transform
LatentFT uses latent-space Fourier transforms and frequency masking in diffusion autoencoders to enable timescale-specific manipulation of musical structure in generative models.
-
SynHAT: A Two-stage Coarse-to-Fine Diffusion Framework for Synthesizing Human Activity Traces
SynHAT uses a novel two-stage spatio-temporal diffusion framework with Latent Spatio-Temporal U-Net to synthesize realistic human activity traces, outperforming baselines by 52% on spatial and 33% on temporal metrics across four cities.
-
One Step Diffusion via Shortcut Models
Shortcut models enable high-quality single or few-step sampling in diffusion models with one network and training phase by conditioning on desired step size.
-
Diffusion Models Beat GANs on Image Synthesis
Diffusion models with architecture improvements and classifier guidance achieve superior FID scores to GANs on unconditional and conditional ImageNet image synthesis.
-
Hierarchical Policy Learning via Spectral Decomposition
Causal Spectral Policy decomposes actions spectrally into coarse motion from obs/language and conditional fine corrections, outperforming baselines on precision manipulation tasks.
-
Global Convergence of Gradient Descent for Score Matching in Gaussian Mixtures via Reverse Fisher Divergence
Proves global GD convergence on reverse Fisher divergence for GMM score matching to single-Gaussian targets from arbitrary init and to separated GMM targets under random init.
-
Audio Pirates: Black-box Audio Watermark Removal via Diffusion Priors
DiffErase removes black-box audio watermarks via diffusion priors by adding intermediate noise and regenerating with a pretrained model, preserving quality across audio domains.
-
Can We Hear from Events? Generating Speech from Event Camera
EventSpeech is a text-conditioned neural framework that uses neuromorphic event cameras and a new EVT-SPK benchmark to generate expressive speech, claiming to outperform RGB baselines by preserving fine-grained emotions without motion blur.
-
PrismFlow: Residual Dynamics for Flow Matching in Time-Series Generation
PrismFlow augments flow matching with residual dynamical experts and a winner-take-all objective to reduce spectral distortion and improve mode coverage in time-series generation.
-
WavFlow: Audio Generation in Waveform Space
WavFlow performs direct waveform audio generation via flow matching on 2D token grids from raw patches plus amplitude lifting, matching latent-based methods on VGGSound and AudioCaps without intermediate compression.
-
Dual-Rate Diffusion: Accelerating diffusion models with an interleaved heavy-light network
Dual-Rate Diffusion interleaves sparse heavy context encoding with frequent light denoising to cut diffusion sampling cost by 2-4x on ImageNet while matching baseline quality and remaining compatible with distillation.
-
DAD4TS: Data-Augmentation-Oriented Diffusion Model for Time-Series Forecasting with Small-Scale Data
DAD4TS trains a diffusion-based generator jointly with a forecaster under RL control and geometric projections to produce augmentation samples that boost accuracy on small-scale time-series data, with validation reported on five of six real-world datasets.
-
DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers
DiRotQ uses PCA-based rotation-aware activation quantization combined with GPTQ to achieve better FID and PSNR in 4-bit diffusion transformers than prior methods like SVDQuant.
-
Discrete Stochastic Localization for Non-autoregressive Generation
DSL provides a continuous embedding framework where one denoiser supports a family of SNR paths for discrete sequences, improving MAUVE scores on OpenWebText and allowing random-order and hybrid sampling from a fine-tuned MDLM checkpoint.
-
TrajDLM: Topology-Aware Block Diffusion Language Model for Trajectory Generation
TrajDLM applies block diffusion language models to discrete road-segment sequences with topology constraints to generate realistic trajectories up to 2.8 times faster than prior methods while supporting zero-shot transfer.
-
DiffATS: Diffusion in Aligned Tensor Space
DiffATS trains diffusion models directly on aligned Tucker tensor primitives that are proven to be homeomorphisms, delivering efficient unconditional and conditional generation across images, videos, and PDE data with high compression.
-
Score-Based Generative Modeling through Anisotropic Stochastic Partial Differential Equations
Anisotropic SPDEs preserve geometric data structure over longer timescales in score-based generative modeling, yielding better image quality than standard SDE baselines and flow matching in unconditional and conditional tasks.
-
SDFlow: Similarity-Driven Flow Matching for Time Series Generation
SDFlow learns a global transport map via similarity-driven flow matching in VQ latent space, using low-rank manifold decomposition and a categorical posterior to handle discreteness, yielding SOTA long-horizon performance and inference speedups.
-
Interests Burn-down Diffusion Process for Personalized Collaborative Filtering
A new interests burn-down diffusion process models decaying user interests for personalized collaborative filtering and outperforms prior generative methods in the StageCF implementation.
-
Interpolating Discrete Diffusion Models with Controllable Resampling
IDDM interpolates diffusion transitions with a resampling mechanism to lessen dependence on intermediate latents and improve sample quality over masked and uniform discrete diffusion models.
-
Diffusion Model for Manifold Data: Score Decomposition, Curvature, and Statistical Complexity
Diffusion models on manifold-supported data admit score decompositions whose statistical rates are controlled by intrinsic dimension and curvature.
-
Flow Map Language Models: One-step Language Modeling via Continuous Denoising
Continuous flows on token embeddings with flow-map distillation produce one-step language models whose quality exceeds recent 8-step discrete diffusion baselines on LM1B and OpenWebText.
-
GCDance: Genre-Controlled Music-Driven 3D Full Body Dance Generation
GCDance is a text-and-music-conditioned diffusion framework that generates genre-consistent 3D dance sequences and reports better results than prior methods on FineDance and AIST++.
-
Mechanisms of Multimodal Synchronization: Insights from Decoder-Based Video-Text-to-Speech Synthesis
Experiments with a video-text-to-speech transformer show co-temporal positional indexing enables synchronization without timestamps, text and video supply complementary signals, and modality ordering creates a trade-off between in-domain accuracy and cross-domain generalization.
-
Diffusion Policy Policy Optimization
DPPO fine-tunes diffusion policies via policy gradients and outperforms prior RL approaches for diffusion policies and PG-tuned alternatives on robot benchmarks while enabling stable training and hardware deployment.
-
ARIA: Adaptive Region-Based Importance Allocation for Conditional Diffusion Distillation
ARIA adaptively focuses distillation updates on conditioning-space regions with high ongoing teacher-student misalignment while preserving the base objective.
-
RFM-Editing 2: Text-Guided Audio Editing with Rectified Flow Matching and Coarse-to-Fine Diffusion Transformers
A coarse-to-fine hybrid MMDiT/DiT audio editor trained with rectified flow matching improves fidelity and cuts edit time versus prior instruction-guided baselines on synthetic overlapping-event tasks.
-
Flicker-DDPM: Accelerating Denoising Diffusion via 1/f Colored Noise Injection
Flicker-DDPM accelerates DDPM sampling by injecting 1/f colored noise matched to image spectra, achieving similar quality with 3.33 times fewer steps on CIFAR-10.
-
Functional MRI Time Series Generation via Wavelet-Based Image Transform and Spectral Flow Matching for Brain Disorder Identification
DSFM generates synthetic fMRI time series via DWT-DCT dual transforms and spectral flow matching to improve downstream brain network classification.
-
Spectro-Temporal Modulation Representation Framework for Human-Imitated Speech Detection
STM representations from auditory filterbanks detect human-imitated speech at or above human listener accuracy levels.
-
EmDT: Embedding Diffusion Transformer for Tabular Data Generation in Fraud Detection
EmDT combines UMAP clustering with a Transformer-based diffusion process to create synthetic fraud samples that improve XGBoost classification on credit card fraud data while preserving correlations and privacy.
-
Diffusion Models for Solving Inverse Problems via Posterior Sampling with Piecewise Guidance
Piecewise guidance in diffusion posterior sampling cuts inference time 23-25% on inpainting and super-resolution with negligible PSNR/SSIM loss while handling measurement noise.
-
MSDformer: Multi-scale Discrete Transformer For Time Series Generation
MSDformer introduces a multi-scale discrete transformer that tokenizes time series at multiple scales and models them autoregressively in discrete space, claiming superior performance over prior DTM methods with rate-distortion theoretical support.
-
DanceDuo: Bridging Human Movement and AI Choreography
DanceDuo applies diffusion models for music-synchronized dance generation and pose estimation for user-AI performance comparison, with a user study reporting positive feedback on usability.
-
Missing Pattern Recognized Diffusion Imputation Model for Missing Not At Random
PRDIM is a diffusion model using a pattern recognizer to impute MNAR missing data by maximizing joint likelihood of observed values and missing mask via EM.
-
Elucidating Representation Degradation Problem in Diffusion Model Training
Diffusion models suffer representation degradation at high noise due to recoverability mismatch; ERD mitigates this by dynamic optimization reallocation, accelerating convergence across backbones.
-
Personalized Federated Distillation Assisted Vehicle Edge Caching Strategy
A personalized federated distillation approach for vehicle edge caching reduces communication overhead and shows robustness to changes in vehicle speed.