REVIEW 4 cited by
Minibatch optimal transport distances; analysis and applications
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Optimal transport distances have become a classic tool to compare probability distributions and have found many applications in machine learning. Yet, despite recent algorithmic developments, their complexity prevents their direct use on large scale datasets. To overcome this challenge, a common workaround is to compute these distances on minibatches i.e. to average the outcome of several smaller optimal transport problems. We propose in this paper an extended analysis of this practice, which effects were previously studied in restricted cases. We first consider a large variety of Optimal Transport kernels. We notably argue that the minibatch strategy comes with appealing properties such as unbiased estimators, gradients and a concentration bound around the expectation, but also with limits: the minibatch OT is not a distance. To recover some of the lost distance axioms, we introduce a debiased minibatch OT function and study its statistical and optimisation properties. Along with this theoretical analysis, we also conduct empirical experiments on gradient flows, generative adversarial networks (GANs) or color transfer that highlight the practical interest of this strategy.
Forward citations
Cited by 4 Pith papers
-
Distributional Determinantal Point Process for Repulsive Clustering of Distributions
A dDPP prior built on a sliced Wasserstein kernel provides a repulsive distribution-valued point process that, in a generalized Bayesian mixture model, clusters distributions into better-separated groups than a Dirich...
-
Convergence of empirical subgradients for optimal transport-based objectives
Under smooth unit costs and models, empirical subdifferentials of parameterized transport objectives converge graphically almost surely to the population subdifferential, so subgradient methods approach population cri...
-
Hierarchical Rectified Flow Matching with Mini-Batch Couplings
Mini-batch couplings in data and velocity space simplify the hierarchy of velocity distributions in hierarchical rectified flow matching, improving low-step generation quality.
-
Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation
Training on the best of K generated candidates improves image, video, and language generative models, with the reported gains growing with scale and enabling single-pass end-to-end generation.
Discussion (0). Sign in to comment.