Diffusion posterior samplers produce biased outputs that can be expressed as an Ornstein-Uhlenbeck path expectation via a surrogate Gaussian path and Feynman-Kac representation, with STSL flattening the spatially varying bias term.
hub Canonical reference
A Survey on Diffusion Models for Inverse Problems
Canonical reference. 100% of citing Pith papers cite this work as background.
31
Pith papers citing it
Background
100% of classified citations
abstract
Diffusion models have become increasingly popular for generative modeling due to their ability to generate high-quality samples. This has unlocked exciting new possibilities for solving inverse problems, especially in image restoration and reconstruction, by treating diffusion models as unsupervised priors. This survey provides a comprehensive overview of methods that utilize pre-trained diffusion models to solve inverse problems without requiring further training. We introduce taxonomies to categorize these methods based on both the problems they address and the techniques they employ. We analyze the connections between different approaches, offering insights into their practical implementation and highlighting important considerations. We further discuss specific challenges and potential solutions associated with using latent diffusion models for inverse problems. This work aims to be a valuable resource for those interested in learning about the intersection of diffusion models and inverse problems.
hub tools
citation-role summary
background 8
citation-polarity summary
fields
cs.CV 11 cs.LG 7 stat.ML 3 eess.IV 2 eess.SP 2 physics.med-ph 2 cs.AI 1 math.NA 1 math.OC 1 physics.flu-dyn 1roles
background 8polarities
background 8representative citing papers
UOTIP learns an unbalanced optimal transport map from noisy to clean distributions for unpaired inverse problems, incorporating a likelihood cost and proving existence/uniqueness via quadratic cost satisfying the twist condition.
Equivalence between Gaussian processes and linear diffusion models enables general conditioning on arbitrary pointwise likelihoods via ODE dynamics and Monte Carlo guidance approximation.
Dynamic resolution priors enable faster diffusion-based image restoration by operating in lower-dimensional subspaces, with adapted methods outperforming prior DM approaches on most tasks.
FlowADMM replaces stochastic renoise-denoise steps in flow-based plug-and-play methods with a deterministic expectation operator inside ADMM, yielding convergence guarantees under weak Lipschitz conditions and state-of-the-art results on standard inverse problems.
Diffusion model priors enable training-free Bayesian sampling for more accurate rain field reconstruction from path-integrated commercial microwave link measurements than Gaussian process baselines.
A Conditional Diffusion Transformer recovers full MIMO-OFDM channels from sparse noisy pilots, delivering over 5 dB gain versus baselines even at 1/32 pilot density and completing inference in 10 steps.
A new decoupled diffusion guidance method enables efficient zero-shot inpainting by avoiding backpropagation through the denoiser while maintaining observation consistency and quality.
Diff-ANO uses conditional consistency models and adjoint neural operator surrogates to enable fast, high-quality USCT reconstructions under sparse and partial views by replacing slow PDE solvers and enabling few-step sampling.
Diffusion models reconstruct high-resolution 3D cardiac ultrasound volumes from heavily undersampled elevation planes and outperform traditional interpolation and supervised deep learning baselines.
CrystalBoltz performs experiment-guided posterior sampling with diffusion models on structure-factor amplitudes for protein structure determination, reporting lower RMSD and R-factors than baselines with 33x faster runtime.
DVD treats voxel occupancy as a discrete variable in a diffusion framework to generate, assess, and edit sparse 3D voxels without continuous thresholding.
A delighting network trained via Dataset Latent Modulation on heterogeneous OLAT and Light Stage data enables high-quality in-the-wild facial reflectance capture from video and produces the NeRSemble-Scan dataset.
Hallucinations in diffusion models are driven by local intrinsic dimension instabilities on the manifold, which Intrinsic Quenching corrects by deflating it.
FMRG is a training-free single-trajectory guidance framework for flow-based models that matches or exceeds baselines on reward-guided tasks and inverse problems using as few as 3 NFEs.
DiffSRDA uses denoising diffusion models to perform uncertainty-aware spatiotemporal super-resolution data assimilation, achieving EnKF-like quality from low-resolution forecasts on an ocean jet testbed.
Noise injection into plug-and-play algorithms using pretrained score-based diffusion denoisers optimizes a Gaussian-smoothed objective and yields better reconstructions for severely ill-posed imaging tasks.
Conditional flow matching learns a velocity field to sample from measurement-conditioned posteriors in physics inverse problems, with early stopping to prevent variance collapse and selective memorization under finite training data.
Preconditioned ULA with exact likelihood enables faster, higher-quality posterior sampling for Cartesian and non-Cartesian MRI reconstructions than annealed sampling or DPS.
FMPlug adapts foundation flow-matching models into practical priors for inverse problems by combining instance-guided warm-start with sharp Gaussianity regularization, showing superior results on image restoration and scientific tasks with limited samples.
NPN introduces a neural-network-based regularization that promotes reconstructions lying in a low-dimensional projection of the sensing operator's null-space, with claimed theoretical guarantees and improved empirical performance across compressive sensing, deblurring, super-resolution, CT, and MRI.
Nuclear Diffusion combines low-rank temporal models with diffusion posterior sampling to suppress background artifacts in videos and shows gains over RPCA on cardiac ultrasound dehazing metrics.
A wavelet diffusion model combined with diffusion posterior sampling enables joint 3D activity-attenuation reconstruction in PET from emission data alone, outperforming MLAA on simulated TOF data.
Derives the conditional score exactly from an unconditional score via affine maps for linear inverse problems in infinite dimensions, shifting computation to offline training.
citing papers explorer
-
Diffusion-Based Posterior Sampling: A Feynman-Kac Analysis of Bias and Stability
Diffusion posterior samplers produce biased outputs that can be expressed as an Ornstein-Uhlenbeck path expectation via a surrogate Gaussian path and Feynman-Kac representation, with STSL flattening the spatially varying bias term.
-
UOTIP: Unbalanced Optimal Transport Map for Unpaired Inverse Problems
UOTIP learns an unbalanced optimal transport map from noisy to clean distributions for unpaired inverse problems, incorporating a likelihood cost and proving existence/uniqueness via quadratic cost satisfying the twist condition.
-
Conditioning Gaussian Processes on Almost Anything
Equivalence between Gaussian processes and linear diffusion models enables general conditioning on arbitrary pointwise likelihoods via ODE dynamics and Monte Carlo guidance approximation.
-
Image Restoration via Diffusion Models with Dynamic Resolution
Dynamic resolution priors enable faster diffusion-based image restoration by operating in lower-dimensional subspaces, with adapted methods outperforming prior DM approaches on most tasks.
-
FlowADMM: Plug-and-play ADMM with Flow-based Renoise-Denoise Priors
FlowADMM replaces stochastic renoise-denoise steps in flow-based plug-and-play methods with a deterministic expectation operator inside ADMM, yielding convergence guarantees under weak Lipschitz conditions and state-of-the-art results on standard inverse problems.
-
Bayesian Rain Field Reconstruction using Commercial Microwave Links and Diffusion Model Priors
Diffusion model priors enable training-free Bayesian sampling for more accurate rain field reconstruction from path-integrated commercial microwave link measurements than Gaussian process baselines.
-
Diffusion Inpainting MIMO-OFDM Channels with Limited Noisy Observations
A Conditional Diffusion Transformer recovers full MIMO-OFDM channels from sparse noisy pilots, delivering over 5 dB gain versus baselines even at 1/32 pilot density and completing inference in 10 steps.
-
Efficient Zero-Shot Inpainting with Decoupled Diffusion Guidance
A new decoupled diffusion guidance method enables efficient zero-shot inpainting by avoiding backpropagation through the denoiser while maintaining observation consistency and quality.
-
Diff-ANO: Towards Fast High-Resolution Ultrasound Computed Tomography via Conditional Consistency Models and Adjoint Neural Operators
Diff-ANO uses conditional consistency models and adjoint neural operator surrogates to enable fast, high-quality USCT reconstructions under sparse and partial views by replacing slow PDE solvers and enabling few-step sampling.
-
High Volume Rate 3D Ultrasound Reconstruction with Diffusion Models
Diffusion models reconstruct high-resolution 3D cardiac ultrasound volumes from heavily undersampled elevation planes and outperform traditional interpolation and supervised deep learning baselines.
-
CrystalBoltz: End-to-End Protein Structure Determination via Experiment-Guided Diffusion for X-Ray Crystallography
CrystalBoltz performs experiment-guided posterior sampling with diffusion models on structure-factor amplitudes for protein structure determination, reporting lower RMSD and R-factors than baselines with 33x faster runtime.
-
DVD: Discrete Voxel Diffusion for 3D Generation and Editing
DVD treats voxel occupancy as a discrete variable in a diffusion framework to generate, assess, and edit sparse 3D voxels without continuous thresholding.
-
Learning a Delighting Prior for Facial Appearance Capture in the Wild
A delighting network trained via Dataset Latent Modulation on heterogeneous OLAT and Light Stage data enables high-quality in-the-wild facial reflectance capture from video and produces the NeRSemble-Scan dataset.
-
Local Intrinsic Dimension Unveils Hallucinations in Diffusion Models
Hallucinations in diffusion models are driven by local intrinsic dimension instabilities on the manifold, which Intrinsic Quenching corrects by deflating it.
-
How to Guide Your Flow: Few-Step Alignment via Flow Map Reward Guidance
FMRG is a training-free single-trajectory guidance framework for flow-based models that matches or exceeds baselines on reward-guided tasks and inverse problems using as few as 3 NFEs.
-
Uncertainty-Aware Spatiotemporal Super-Resolution Data Assimilation with Diffusion Models
DiffSRDA uses denoising diffusion models to perform uncertainty-aware spatiotemporal super-resolution data assimilation, achieving EnKF-like quality from low-resolution forecasts on an ocean jet testbed.
-
Stochastic Generative Plug-and-Play Priors
Noise injection into plug-and-play algorithms using pretrained score-based diffusion denoisers optimizes a Gaussian-smoothed objective and yields better reconstructions for severely ill-posed imaging tasks.
-
Conditional flow matching for physics-constrained inverse problems with finite training data
Conditional flow matching learns a velocity field to sample from measurement-conditioned posteriors in physics inverse problems, with early stopping to prevent variance collapse and selective memorization under finite training data.
-
Fast and Robust Diffusion Posterior Sampling for MR Image Reconstruction Using the Preconditioned Unadjusted Langevin Algorithm
Preconditioned ULA with exact likelihood enables faster, higher-quality posterior sampling for Cartesian and non-Cartesian MRI reconstructions than annealed sampling or DPS.
-
Saving Foundation Flow-Matching Priors for Inverse Problems
FMPlug adapts foundation flow-matching models into practical priors for inverse problems by combining instance-guided warm-start with sharp Gaussianity regularization, showing superior results on image restoration and scientific tasks with limited samples.
-
NPN: Non-Linear Projections of the Null-Space for Imaging Inverse Problems
NPN introduces a neural-network-based regularization that promotes reconstructions lying in a low-dimensional projection of the sensing operator's null-space, with claimed theoretical guarantees and improved empirical performance across compressive sensing, deblurring, super-resolution, CT, and MRI.
-
Nuclear Diffusion Models for Low-Rank Background Suppression in Videos
Nuclear Diffusion combines low-rank temporal models with diffusion posterior sampling to suppress background artifacts in videos and shows gains over RPCA on cardiac ultrasound dehazing metrics.
-
Joint Reconstruction of Activity and Attenuation in PET by Diffusion Posterior Sampling in Wavelet Coefficient Space
A wavelet diffusion model combined with diffusion posterior sampling enables joint 3D activity-attenuation reconstruction in PET from emission data alone, outperforming MLAA on simulated TOF data.
-
An Unconditional Representation of the Conditional Score in Infinite-Dimensional Linear Inverse Problems
Derives the conditional score exactly from an unconditional score via affine maps for linear inverse problems in infinite dimensions, shifting computation to offline training.
-
Principled Design of Diffusion-based Optimizers for Inverse Problems
Reparameterizations create invariances in diffusion inverse-problem solvers, enabling hyperparameter reuse and accelerated inference via the OptDiff optimization framework.
-
A Stability Benchmark of Generative Regularizers for Inverse Problems
Numerical benchmarks indicate generative regularizers deliver strong reconstructions in some imaging inverse problem settings but can be unstable or problematic under imperfect conditions compared to variational methods.
-
Dual Ascent Diffusion for Inverse Problems
A dual ascent optimization framework is introduced for MAP estimation with diffusion priors, claimed to outperform prior methods on image restoration in quality, noise robustness, speed, and data fidelity.
-
Harnessing AI for Inverse Partial Differential Equation Problems: Past, Present, and Prospects
A survey organizing AI methods for inverse PDE problems into inverse problems, inverse design, and control categories, covering applications and future challenges like physics-informed models and uncertainty quantification.
-
Generative AI Meets 6G and Beyond: Diffusion Models for Semantic Communications
The tutorial synthesizes diffusion model techniques for generative semantic communications to achieve high compression while preserving meaning in wireless transmission.
- Proximal-Based Generative Modeling for Bayesian Inverse Problems
- The Principles of Diffusion Models