NFDRL models return distributions via continuous normalizing flows paired with a geometry-aware Cramér surrogate distance, delivering fixed-size parameters, a sqrt(gamma) contraction, unbiased gradients, and competitive Atari-5 performance.
The Cramer Distance as a Solution to Biased Wasserstein Gradients
6 Pith papers cite this work. Polarity classification is still indexing.
6
Pith papers citing it
abstract
The Wasserstein probability metric has received much attention from the machine learning community. Unlike the Kullback-Leibler divergence, which strictly measures change in probability, the Wasserstein metric reflects the underlying geometry between outcomes. The value of being sensitive to this geometry has been demonstrated, among others, in ordinal regression and generative modelling. In this paper we describe three natural properties of probability divergences that reflect requirements from machine learning: sum invariance, scale sensitivity, and unbiased sample gradients. The Wasserstein metric possesses the first two properties but, unlike the Kullback-Leibler divergence, does not possess the third. We provide empirical evidence suggesting that this is a serious issue in practice. Leveraging insights from probabilistic forecasting we propose an alternative to the Wasserstein metric, the Cram\'er distance. We show that the Cram\'er distance possesses all three desired properties, combining the best of the Wasserstein and Kullback-Leibler divergences. To illustrate the relevance of the Cram\'er distance in practice we design a new algorithm, the Cram\'er Generative Adversarial Network (GAN), and show that it performs significantly better than the related Wasserstein GAN.
citation-role summary
method 2
background 1
citation-polarity summary
representative citing papers
A harmonized probabilistic model with adaptive feature conditioning and high-frequency prompt modules disentangles acquisition artifacts from rater variability to produce personalized yet consistent multi-rater segmentations, showing SOTA results on LIDC-IDRI and NPC-170.
BigGANs achieve state-of-the-art class-conditional synthesis on ImageNet 128x128 with Inception Score 166.5 and FID 7.4 by scaling GANs and applying orthogonal regularization plus truncation.
MMD GANs have unbiased critic gradients but biased generator gradients from sample-based learning, and the Kernel Inception Distance provides a practical new measure for GAN convergence and dynamic learning rate adaptation.
A one-step text-to-audio model using energy-distance training and contextual distillation outperforms prior fast baselines on AudioCaps and achieves up to 8.5x faster inference than the multi-step IMPACT system with competitive quality.
citing papers explorer
-
Parameter-Efficient Distributional RL via Normalizing Flows and a Geometry-Aware Cram\'er Surrogate
NFDRL models return distributions via continuous normalizing flows paired with a geometry-aware Cramér surrogate distance, delivering fixed-size parameters, a sqrt(gamma) contraction, unbiased gradients, and competitive Atari-5 performance.
-
Harmonized Feature Conditioning and Frequency-Prompt Personalization for Multi-Rater Medical Segmentation
A harmonized probabilistic model with adaptive feature conditioning and high-frequency prompt modules disentangles acquisition artifacts from rater variability to produce personalized yet consistent multi-rater segmentations, showing SOTA results on LIDC-IDRI and NPC-170.
-
Large Scale GAN Training for High Fidelity Natural Image Synthesis
BigGANs achieve state-of-the-art class-conditional synthesis on ImageNet 128x128 with Inception Score 166.5 and FID 7.4 by scaling GANs and applying orthogonal regularization plus truncation.
-
Demystifying MMD GANs
MMD GANs have unbiased critic gradients but biased generator gradients from sample-based learning, and the Kernel Inception Distance provides a practical new measure for GAN convergence and dynamic learning rate adaptation.
-
Fast Text-to-Audio Generation with One-Step Sampling via Energy-Scoring and Auxiliary Contextual Representation Distillation
A one-step text-to-audio model using energy-distance training and contextual distillation outperforms prior fast baselines on AudioCaps and achieves up to 8.5x faster inference than the multi-step IMPACT system with competitive quality.
- A Kernel Score Perspective on Forecast Disagreement and the Linear Pool