Rectified flow learns straight-path neural ODEs for distribution transport, yielding efficient generative models and domain transfers that work well even with a single simulation step.
hub Canonical reference
NICE: Non-linear Independent Components Estimation
Canonical reference. 75% of citing Pith papers cite this work as background.
43
Pith papers citing it
Background
75% of classified citations
abstract
We propose a deep learning framework for modeling complex high-dimensional densities called Non-linear Independent Component Estimation (NICE). It is based on the idea that a good representation is one in which the data has a distribution that is easy to model. For this purpose, a non-linear deterministic transformation of the data is learned that maps it to a latent space so as to make the transformed data conform to a factorized distribution, i.e., resulting in independent latent variables. We parametrize this transformation so that computing the Jacobian determinant and inverse transform is trivial, yet we maintain the ability to learn complex non-linear transformations, via a composition of simple building blocks, each based on a deep neural network. The training criterion is simply the exact log-likelihood, which is tractable. Unbiased ancestral sampling is also easy. We show that this approach yields good generative models on four image datasets and can be used for inpainting.
hub tools
citation-role summary
background 6
baseline 1
extension 1
citation-polarity summary
representative citing papers
Introduces an SDE-based framework for score-based generative modeling that unifies prior methods, enables predictor-corrector sampling and neural ODE likelihoods, and achieves SOTA unconditional image generation on CIFAR-10.
Denoising diffusion probabilistic models generate high-quality images by learning to reverse a fixed forward diffusion process, achieving FID 3.17 on CIFAR10.
Neural networks are redefined as continuous dynamical systems by learning the derivative of the hidden state with a neural network and integrating it with an ODE solver.
Real NVP uses affine coupling layers to create invertible transformations that support exact density estimation, sampling, and latent inference without approximations.
A forward diffusion process adds noise iteratively to data until it is unstructured, and a neural network learns the reverse process to generate new samples from the original distribution.
SCREAM adapts the CATHODE method to treat stellar streams as feature-space over-densities, incorporates measurement uncertainties into neural network training, and achieves F1=0.745 on GD-1 while recovering faint members and a diffuse cocoon missed by prior methods.
Latent Heuristic Search performs continuous optimization over learned embeddings of heuristics, using normalizing flows and LLM prompting to discover competitive solvers for TSP, CVRP, KSP, and OBP.
StAD distills divergence of PF-ODEs via the Langevin-Stein operator for faster, lower-variance likelihood estimation in generative models without Jacobian costs.
The Sinkhorn treatment effect is a new entropic optimal transport measure of divergence between counterfactual distributions that admits first- and second-order pathwise differentiability, debiased estimators, and asymptotically valid tests for distributional treatment effects.
NTM models each generative reverse step as a conditional normalizing flow with a hybrid shallow-deep architecture, enabling exact-likelihood training and strong four-step sampling performance on text-to-image tasks.
TRACE creates valid conformal prediction sets for complex generative models by scoring outputs via averaged denoising or velocity errors along stochastic transport paths instead of likelihoods.
ARDIS enables arbitrary-resolution deep image steganography via frequency decoupling in hiding and latent-guided implicit reconstruction for blind recovery.
A data-driven framework using normalizing flows predicts the rate and kinematic distributions of dark photon and millicharged particle production directly from measured dilepton events.
Iso-Riemannian descent algorithm with convergence analysis under iso-convexity, iso-monotonicity and iso-Lipschitz conditions for optimization on learned Riemannian manifolds from data.
Neural autoregressive flows enable flexible high-dimensional spatial warpings for nonstationary anisotropic processes, with simulations showing greater representational capacity than standard models and an application to 3D Argo Floats data.
Moonwalk enables memory-efficient training of deep networks via mixed-mode gradient computation with vector-inverse-Jacobian products for submersive layers and fragmental checkpointing otherwise, matching backprop runtime at over twice the depth.
Proposes cINN architecture for conditional image generation that by construction yields diverse sharp samples, demonstrated on MNIST digit generation and image colorization with latent space manipulation.
Generative models for cosmological field-level inference can reproduce posterior means and cross-correlations yet fail to capture correct uncertainty geometry when validated against HMC reference samples.
A flow-matching generative model trained on CoLBT-hydro data conditionally generates marginal final-state hadron spectra from jet-induced hydro responses in 0-10% Pb+Pb collisions at 5.02 TeV, matching training data statistics with approximately six orders of magnitude computational speedup.
TVRN combines invertible wavelet-based networks with a surrogate gradient approximator and compression-aware asymmetric design to improve frame-rate rescaling quality under real codecs.
CONTRA generates sharp multi-dimensional conformal prediction regions by defining nonconformity scores as distances from the center in the latent space of a normalizing flow.
STARFlow2 presents an autoregressive flow-based architecture for unified multimodal text-image generation by interleaving a VLM stream with a TarFlow stream via residual skips and a unified latent space.
ARFP is a key-conditioned reversible face cloaking method that resists unauthorized restoration attacks while enabling authorized recovery with tamper indication.
citing papers explorer
-
Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
Rectified flow learns straight-path neural ODEs for distribution transport, yielding efficient generative models and domain transfers that work well even with a single simulation step.
-
Score-Based Generative Modeling through Stochastic Differential Equations
Introduces an SDE-based framework for score-based generative modeling that unifies prior methods, enables predictor-corrector sampling and neural ODE likelihoods, and achieves SOTA unconditional image generation on CIFAR-10.
-
Denoising Diffusion Probabilistic Models
Denoising diffusion probabilistic models generate high-quality images by learning to reverse a fixed forward diffusion process, achieving FID 3.17 on CIFAR10.
-
Neural Ordinary Differential Equations
Neural networks are redefined as continuous dynamical systems by learning the derivative of the hidden state with a neural network and integrating it with an ODE solver.
-
Density estimation using Real NVP
Real NVP uses affine coupling layers to create invertible transformations that support exact density estimation, sampling, and latent inference without approximations.
-
Deep Unsupervised Learning using Nonequilibrium Thermodynamics
A forward diffusion process adds noise iteratively to data until it is unstructured, and a neural network learns the reverse process to generate new samples from the original distribution.
-
Characterizing Stellar Streams with Error-Aware Machine Learning
SCREAM adapts the CATHODE method to treat stellar streams as feature-space over-densities, incorporates measurement uncertainties into neural network training, and achieves F1=0.745 on GD-1 while recovering faint members and a diffuse cocoon missed by prior methods.
-
Latent Heuristic Search: Continuous Optimization for Automated Algorithm Design
Latent Heuristic Search performs continuous optimization over learned embeddings of heuristics, using normalizing flows and LLM prompting to discover competitive solvers for TSP, CVRP, KSP, and OBP.
-
StAD: Stein Amortized Divergence for Fast Likelihoods with Diffusion and Flow
StAD distills divergence of PF-ODEs via the Langevin-Stein operator for faster, lower-variance likelihood estimation in generative models without Jacobian costs.
-
Sinkhorn Treatment Effects: A Causal Optimal Transport Measure
The Sinkhorn treatment effect is a new entropic optimal transport measure of divergence between counterfactual distributions that admits first- and second-order pathwise differentiability, debiased estimators, and asymptotically valid tests for distributional treatment effects.
-
Normalizing Trajectory Models
NTM models each generative reverse step as a conditional normalizing flow with a hybrid shallow-deep architecture, enabling exact-likelihood training and strong four-step sampling performance on text-to-image tasks.
-
TRACE: Transport Alignment Conformal Prediction via Diffusion and Flow Matching Models
TRACE creates valid conformal prediction sets for complex generative models by scoring outputs via averaged denoising or velocity errors along stochastic transport paths instead of likelihoods.
-
Breaking the Resolution Barrier: Arbitrary-resolution Deep Image Steganography Framework
ARDIS enables arbitrary-resolution deep image steganography via frequency decoupling in hiding and latent-guided implicit reconstruction for blind recovery.
-
Data-Driven Predictions for Dark Photon and Millicharged Particle Production
A data-driven framework using normalizing flows predicts the rate and kinematic distributions of dark photon and millicharged particle production directly from measured dilepton events.
-
Iso-Riemannian Optimization on Learned Data Manifolds
Iso-Riemannian descent algorithm with convergence analysis under iso-convexity, iso-monotonicity and iso-Lipschitz conditions for optimization on learned Riemannian manifolds from data.
-
Modeling nonstationary spatial processes with normalizing flows
Neural autoregressive flows enable flexible high-dimensional spatial warpings for nonstationary anisotropic processes, with simulations showing greater representational capacity than standard models and an application to 3D Argo Floats data.
-
Moonwalk: Inverse-Forward Differentiation
Moonwalk enables memory-efficient training of deep networks via mixed-mode gradient computation with vector-inverse-Jacobian products for submersive layers and fragmental checkpointing otherwise, matching backprop runtime at over twice the depth.
-
Guided Image Generation with Conditional Invertible Neural Networks
Proposes cINN architecture for conditional image generation that by construction yields diverse sharp samples, demonstrated on MNIST digit generation and image colorization with latent space manipulation.
-
Learning the Universe: Posterior Reliability of Neural Generative Models in High-Dimensional Field-Level Inference of Cosmic Initial Conditions
Generative models for cosmological field-level inference can reproduce posterior means and cross-correlations yet fail to capture correct uncertainty geometry when validated against HMC reference samples.
-
A flow-matching generative model for event-by-event jet-induced hydro response in high-energy heavy-ion collisions
A flow-matching generative model trained on CoLBT-hydro data conditionally generates marginal final-state hadron spectra from jet-induced hydro responses in 0-10% Pb+Pb collisions at 5.02 TeV, matching training data statistics with approximately six orders of magnitude computational speedup.
-
TVRN: Invertible Neural Networks for Compression-Aware Temporal Video Rescaling
TVRN combines invertible wavelet-based networks with a surrogate gradient approximator and compression-aware asymmetric design to improve frame-rate rescaling quality under real codecs.
-
CONTRA: Conformal Prediction Region via Normalizing Flow Transformation
CONTRA generates sharp multi-dimensional conformal prediction regions by defining nonconformity scores as distances from the center in the latent space of a normalizing flow.
-
STARFlow2: Bridging Language Models and Normalizing Flows for Unified Multimodal Generation
STARFlow2 presents an autoregressive flow-based architecture for unified multimodal text-image generation by interleaving a VLM stream with a TarFlow stream via residual skips and a unified latent space.
-
Asymmetric Invertible Threat: Learning Reversible Privacy Defense for Face Recognition
ARFP is a key-conditioned reversible face cloaking method that resists unauthorized restoration attacks while enabling authorized recovery with tamper indication.
-
8DNA: 8D Neural Asset Light Transport by Distribution Learning
8DNA learns the complete 8D light transport function from path-traced samples via distribution learning to support accurate near-field global illumination rendering of complex 3D assets.
-
REZE: Representation Regularization for Domain-adaptive Text Embedding Pre-finetuning
REZE controls representation shifts in contrastive pre-finetuning of text embeddings via eigenspace decomposition of anchor-positive pairs and adaptive soft-shrinkage on task-variant directions.
-
Lookahead Drifting Model
The lookahead drifting model improves upon the drifting model by sequentially computing multiple drifting terms that incorporate higher-order gradient information, leading to better performance on toy examples and CIFAR10.
-
Monocular Depth Estimation From the Perspective of Feature Restoration: A Diffusion Enhanced Depth Restoration Approach
Monocular depth estimation is recast as indirect feature restoration via an invertible diffusion module plus auxiliary viewpoint enhancement, delivering 4-38% RMSE gains on KITTI over baselines.
-
Dartmouth Stellar Evolution Emulator (DSEE) 1: Generative Stellar Evolution Model Database
DSEE is a flow-based emulator that generates stellar evolution tracks and isochrones as probabilistic outputs from a single model trained on millions of simulations, enabling fast interpolation and uncertainty-aware analyses.
-
Monte Carlo Event Generation with Continuous Normalizing Flows
Continuous normalizing flows improve unweighting efficiency in Monte Carlo event generation for high-jet-multiplicity collider processes by factors up to 184, with wall-time gains of about ten when combined with coupling-layer flows.
-
Multivariate Time Series Anomaly Detection via Dual-Branch Reconstruction and Autoregressive Flow-based Residual Density Estimation
DBR-AF decouples cross-variable correlations in reconstruction and applies autoregressive flows to model residual densities for improved anomaly detection in multivariate time series.
-
Conditional flow matching for physics-constrained inverse problems with finite training data
Conditional flow matching learns a velocity field to sample from measurement-conditioned posteriors in physics inverse problems, with early stopping to prevent variance collapse and selective memorization under finite training data.
-
Reversible Deep Learning for 13C NMR in Chemoinformatics: On Structures and Spectra
A conditional invertible neural network unifies forward prediction of 13C NMR spectra from structures and inverse generation of structure candidates from spectra.
-
RefTon: Reference person shot assist virtual Try-on
RefTon is a flux-based virtual try-on method that uses unpaired reference images of the target garment on different people to guide texture and detail preservation in a streamlined person-to-person pipeline without body parsing or masks.
-
T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models
T2I-Adapters are lightweight modules that enable fine-grained control over color and structure in text-to-image diffusion models by aligning external conditions with the frozen model's internal knowledge.
-
VideoGPT: Video Generation using VQ-VAE and Transformers
VideoGPT generates competitive natural videos by learning discrete latents with VQ-VAE and modeling them autoregressively with a transformer.
-
Generative Modeling by Estimating Gradients of the Data Distribution
Score-based generative modeling via multi-noise-level score matching and annealed Langevin dynamics produces samples on par with GANs and sets a new inception score record on CIFAR-10.
-
Deep Adaptive Dimension Reduction for Bayesian Inference in Inverse Problems
A new variational flow model with iterative prior updating and adaptive FNO surrogate for dimension-reduced Bayesian inference in high-dimensional PDE-governed inverse problems, reporting competitive accuracy versus MCMC, UKI, and SVGD on test cases.
-
Pulse profile modelling of the 2024 outburst of the accreting millisecond pulsar SRGA J144459.2-604207
Joint NICER+IXPE pulse-profile modeling of SRGA J144459.2-604207 favors large neutron-star mass and radius with two independent hotspots but shows strong sensitivity to joint-analysis methodology.
-
Generative Design of a Gas Turbine Combustor Using Invertible Neural Networks
Invertible Neural Networks are used to generate gas turbine combustor designs that meet specified performance criteria from a training database of parameterized designs and simulations.
-
Align Generative Artificial Intelligence with Human Preferences: A Novel Large Language Model Fine-Tuning Method for Online Review Management
A preference fine-tuning method for LLMs that combines context augmentation, theory-driven preference pair construction, curriculum learning, and a density estimation support constraint to produce domain-aligned review responses with reduced hallucinations and over-conservatism.
-
Venom: A PyTorch Generative Modeling Toolkit
Venom is an educational PyTorch toolkit that packages multiple generative modeling families under a single MNIST-first interface with reproducible scripts and tutorials.
-
Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models
The paper reviews the background, technology, applications, limitations, and future directions of OpenAI's Sora text-to-video generative model based on public information.