Rectified flow learns straight-path neural ODEs for distribution transport, yielding efficient generative models and domain transfers that work well even with a single simulation step.
hub Canonical reference
NICE: Non-linear Independent Components Estimation
Canonical reference. 75% of citing Pith papers cite this work as background.
43
Pith papers citing it
Background
75% of classified citations
abstract
We propose a deep learning framework for modeling complex high-dimensional densities called Non-linear Independent Component Estimation (NICE). It is based on the idea that a good representation is one in which the data has a distribution that is easy to model. For this purpose, a non-linear deterministic transformation of the data is learned that maps it to a latent space so as to make the transformed data conform to a factorized distribution, i.e., resulting in independent latent variables. We parametrize this transformation so that computing the Jacobian determinant and inverse transform is trivial, yet we maintain the ability to learn complex non-linear transformations, via a composition of simple building blocks, each based on a deep neural network. The training criterion is simply the exact log-likelihood, which is tractable. Unbiased ancestral sampling is also easy. We show that this approach yields good generative models on four image datasets and can be used for inpainting.
hub tools
citation-role summary
background 6
baseline 1
extension 1
citation-polarity summary
representative citing papers
Introduces an SDE-based framework for score-based generative modeling that unifies prior methods, enables predictor-corrector sampling and neural ODE likelihoods, and achieves SOTA unconditional image generation on CIFAR-10.
Denoising diffusion probabilistic models generate high-quality images by learning to reverse a fixed forward diffusion process, achieving FID 3.17 on CIFAR10.
Neural networks are redefined as continuous dynamical systems by learning the derivative of the hidden state with a neural network and integrating it with an ODE solver.
Real NVP uses affine coupling layers to create invertible transformations that support exact density estimation, sampling, and latent inference without approximations.
A forward diffusion process adds noise iteratively to data until it is unstructured, and a neural network learns the reverse process to generate new samples from the original distribution.
SCREAM adapts the CATHODE method to treat stellar streams as feature-space over-densities, incorporates measurement uncertainties into neural network training, and achieves F1=0.745 on GD-1 while recovering faint members and a diffuse cocoon missed by prior methods.
Latent Heuristic Search performs continuous optimization over learned embeddings of heuristics, using normalizing flows and LLM prompting to discover competitive solvers for TSP, CVRP, KSP, and OBP.
StAD distills divergence of PF-ODEs via the Langevin-Stein operator for faster, lower-variance likelihood estimation in generative models without Jacobian costs.
The Sinkhorn treatment effect is a new entropic optimal transport measure of divergence between counterfactual distributions that admits first- and second-order pathwise differentiability, debiased estimators, and asymptotically valid tests for distributional treatment effects.
NTM models each generative reverse step as a conditional normalizing flow with a hybrid shallow-deep architecture, enabling exact-likelihood training and strong four-step sampling performance on text-to-image tasks.
TRACE creates valid conformal prediction sets for complex generative models by scoring outputs via averaged denoising or velocity errors along stochastic transport paths instead of likelihoods.
ARDIS enables arbitrary-resolution deep image steganography via frequency decoupling in hiding and latent-guided implicit reconstruction for blind recovery.
A data-driven framework using normalizing flows predicts the rate and kinematic distributions of dark photon and millicharged particle production directly from measured dilepton events.
Iso-Riemannian descent algorithm with convergence analysis under iso-convexity, iso-monotonicity and iso-Lipschitz conditions for optimization on learned Riemannian manifolds from data.
Neural autoregressive flows enable flexible high-dimensional spatial warpings for nonstationary anisotropic processes, with simulations showing greater representational capacity than standard models and an application to 3D Argo Floats data.
Moonwalk enables memory-efficient training of deep networks via mixed-mode gradient computation with vector-inverse-Jacobian products for submersive layers and fragmental checkpointing otherwise, matching backprop runtime at over twice the depth.
Proposes cINN architecture for conditional image generation that by construction yields diverse sharp samples, demonstrated on MNIST digit generation and image colorization with latent space manipulation.
Generative models for cosmological field-level inference can reproduce posterior means and cross-correlations yet fail to capture correct uncertainty geometry when validated against HMC reference samples.
A flow-matching generative model trained on CoLBT-hydro data conditionally generates marginal final-state hadron spectra from jet-induced hydro responses in 0-10% Pb+Pb collisions at 5.02 TeV, matching training data statistics with approximately six orders of magnitude computational speedup.
TVRN combines invertible wavelet-based networks with a surrogate gradient approximator and compression-aware asymmetric design to improve frame-rate rescaling quality under real codecs.
CONTRA generates sharp multi-dimensional conformal prediction regions by defining nonconformity scores as distances from the center in the latent space of a normalizing flow.
STARFlow2 presents an autoregressive flow-based architecture for unified multimodal text-image generation by interleaving a VLM stream with a TarFlow stream via residual skips and a unified latent space.
ARFP is a key-conditioned reversible face cloaking method that resists unauthorized restoration attacks while enabling authorized recovery with tamper indication.
citing papers explorer
-
Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
Rectified flow learns straight-path neural ODEs for distribution transport, yielding efficient generative models and domain transfers that work well even with a single simulation step.
-
Score-Based Generative Modeling through Stochastic Differential Equations
Introduces an SDE-based framework for score-based generative modeling that unifies prior methods, enables predictor-corrector sampling and neural ODE likelihoods, and achieves SOTA unconditional image generation on CIFAR-10.
-
Denoising Diffusion Probabilistic Models
Denoising diffusion probabilistic models generate high-quality images by learning to reverse a fixed forward diffusion process, achieving FID 3.17 on CIFAR10.
-
Neural Ordinary Differential Equations
Neural networks are redefined as continuous dynamical systems by learning the derivative of the hidden state with a neural network and integrating it with an ODE solver.
-
Density estimation using Real NVP
Real NVP uses affine coupling layers to create invertible transformations that support exact density estimation, sampling, and latent inference without approximations.
-
Deep Unsupervised Learning using Nonequilibrium Thermodynamics
A forward diffusion process adds noise iteratively to data until it is unstructured, and a neural network learns the reverse process to generate new samples from the original distribution.
-
Moonwalk: Inverse-Forward Differentiation
Moonwalk enables memory-efficient training of deep networks via mixed-mode gradient computation with vector-inverse-Jacobian products for submersive layers and fragmental checkpointing otherwise, matching backprop runtime at over twice the depth.
-
Lookahead Drifting Model
The lookahead drifting model improves upon the drifting model by sequentially computing multiple drifting terms that incorporate higher-order gradient information, leading to better performance on toy examples and CIFAR10.
-
Multivariate Time Series Anomaly Detection via Dual-Branch Reconstruction and Autoregressive Flow-based Residual Density Estimation
DBR-AF decouples cross-variable correlations in reconstruction and applies autoregressive flows to model residual densities for improved anomaly detection in multivariate time series.
-
Reversible Deep Learning for 13C NMR in Chemoinformatics: On Structures and Spectra
A conditional invertible neural network unifies forward prediction of 13C NMR spectra from structures and inverse generation of structure candidates from spectra.
-
Generative Modeling by Estimating Gradients of the Data Distribution
Score-based generative modeling via multi-noise-level score matching and annealed Langevin dynamics produces samples on par with GANs and sets a new inception score record on CIFAR-10.
-
Deep Adaptive Dimension Reduction for Bayesian Inference in Inverse Problems
A new variational flow model with iterative prior updating and adaptive FNO surrogate for dimension-reduced Bayesian inference in high-dimensional PDE-governed inverse problems, reporting competitive accuracy versus MCMC, UKI, and SVGD on test cases.
-
Venom: A PyTorch Generative Modeling Toolkit
Venom is an educational PyTorch toolkit that packages multiple generative modeling families under a single MNIST-first interface with reproducible scripts and tutorials.