Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model
read the original abstract
Most existing Image Restoration (IR) models are task-specific, which can not be generalized to different degradation operators. In this work, we propose the Denoising Diffusion Null-Space Model (DDNM), a novel zero-shot framework for arbitrary linear IR problems, including but not limited to image super-resolution, colorization, inpainting, compressed sensing, and deblurring. DDNM only needs a pre-trained off-the-shelf diffusion model as the generative prior, without any extra training or network modifications. By refining only the null-space contents during the reverse diffusion process, we can yield diverse results satisfying both data consistency and realness. We further propose an enhanced and robust version, dubbed DDNM+, to support noisy restoration and improve restoration quality for hard tasks. Our experiments on several IR tasks reveal that DDNM outperforms other state-of-the-art zero-shot IR methods. We also demonstrate that DDNM+ can solve complex real-world applications, e.g., old photo restoration.
This paper has not been read by Pith yet.
Forward citations
Cited by 22 Pith papers
-
PolarVSR: A Unified Framework and Benchmark for Continuous Space-Time Polarization Video Reconstruction
PolarVSR is the first unified architecture for continuous space-time polarization video reconstruction from DoFP captures, using polarization-aware implicit neural representations, a flow-guided variation loss, and a ...
-
FlowBender: Feedback-Aware Training for Self-Correcting Conditional Flows
FlowBender introduces closed-loop training that lets conditional flow models learn correction policies from their own task-specific alignment errors, outperforming supervised and guidance baselines on fidelity and pla...
-
Reflection Separation from a Single Image via Joint Latent Diffusion
A joint latent diffusion model with cross-layer self-attention and disjoint sampling separates reflection and transmission layers from single images more effectively than prior methods on real-world benchmarks.
-
Proximal-Based Generative Modeling for Bayesian Inverse Problems
PGM replaces the intractable likelihood score in diffusion models with a closed-form Moreau score computed via proximal operators, enabling non-asymptotic sampling for inverse problems trained only on prior data.
-
Proximal-Based Generative Modeling for Bayesian Inverse Problems
PGM framework links diffusion to proximal regularization for closed-form Moreau-score sampling in Bayesian inverse problems, learned only from prior samples.
-
Invertible Diffusion for Low-Memory Channel Gain Map Construction in Wireless Communication Networks
InvDiff-CGM uses invertible architectures in diffusion and U-Net plus a multi-scale prior injector to construct CGMs with 85% lower peak training memory and 38.02 dB PSNR on RadioMap3DSeer.
-
Diff-ANO: Towards Fast High-Resolution Ultrasound Computed Tomography via Conditional Consistency Models and Adjoint Neural Operators
Diff-ANO uses conditional consistency models and adjoint neural operator surrogates to enable fast, high-quality USCT reconstructions under sparse and partial views by replacing slow PDE solvers and enabling few-step ...
-
COCO-Inpaint: A Benchmark for Detecting and Localizing Inpainting-Based Image Manipulations
COCO-Inpaint supplies a large-scale dataset and evaluation protocol focused on inpainting-based image forgeries to benchmark existing detection methods.
-
From Sparse X-rays to 3D CT: Training-Free Reconstruction with Diffusion Priors
TF-PRDiT uses a frozen 3D diffusion transformer prior with task-specific forward operators and predictor-corrector sampling to solve X-ray-to-CT reconstruction and other volumetric inverse problems without any retraining.
-
Moebius: 0.2B Lightweight Image Inpainting Framework with 10B-Level Performance
Moebius introduces a compressed diffusion inpainting model using Local-λ Mix Interaction blocks and latent-space multi-granularity distillation to reach 10B-level quality with 0.22B parameters.
-
Measurement Geometry and Design for Trustworthy Generative Inverse Problems
Introduces a local measurement-manifold compatibility measure that bounds stable reconstruction error under generative priors and motivates fixed and adaptive acquisition rules.
-
Conservative Flows: A New Paradigm of Generative Models
Conservative flows generate by running probability-preserving stochastic dynamics initialized at data points rather than noise, using corrected Langevin or predictor-corrector mechanisms on top of any pretrained flow ...
-
Plug-and-Play Label Map Diffusion for Universal Goal-Oriented Navigation
PLMD applies a denoising diffusion model to predict labels for unknown map regions, allowing goal localization in unexplored environments by substituting completed labels into existing navigation pipelines.
-
Conditional Diffusion Under Linear Constraints: Langevin Mixing and Information-Theoretic Guarantees
Error in approximating the tangent conditional score by the unconditional score in diffusion models is bounded by dimension-free conditional mutual information, with a projected-Langevin method outperforming baselines...
-
Degradation-Robust Fusion: An Efficient Degradation-Aware Diffusion Framework for Multimodal Image Fusion in Arbitrary Degradation Scenarios
A degradation-aware diffusion framework fuses multimodal images under arbitrary degradations by directly regressing the fused image and applying joint degradation-fusion constraints during limited-step sampling.
-
Analyzing and Guiding Zero-Shot Posterior Sampling in Diffusion Models
Under a Gaussian prior assumption, zero-shot diffusion posterior samplers for inverse problems admit closed-form spectral representations that enable a new parameter-selection framework balancing perceptual quality an...
-
Protein Autoregressive Modeling via Multiscale Structure Generation
PAR is a multi-scale autoregressive transformer framework for protein backbone generation that uses coarse-to-fine prediction, noisy context learning, and flow-based decoding to achieve high-quality unconditional and ...
-
NPN: Non-Linear Projections of the Null-Space for Imaging Inverse Problems
NPN introduces a neural-network-based regularization that promotes reconstructions lying in a low-dimensional projection of the sensing operator's null-space, with claimed theoretical guarantees and improved empirical...
-
Separating Intrinsic Ambiguity from Estimation Uncertainty in Deep Generative Models for Linear Inverse Problems
Introduces a cascade-based structural decomposition of posterior uncertainty to isolate intrinsic ambiguity from estimation uncertainty in deep generative models for linear inverse problems.
-
Unifying Deep Stochastic Processes for Image Enhancement
Stochastic image enhancement methods are shown to be variants of a shared SDE differing in drift, diffusion, terminal distributions and boundary conditions, with controlled experiments revealing no single dominant fam...
-
Dual Ascent Diffusion for Inverse Problems
A dual ascent optimization framework is introduced for MAP estimation with diffusion priors, claimed to outperform prior methods on image restoration in quality, noise robustness, speed, and data fidelity.
-
A Survey on Diffusion Models for Inverse Problems
A survey that introduces taxonomies for categorizing pre-trained diffusion model methods applied to inverse problems and analyzes their connections and challenges.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.