REVIEW 7 cited by
Inference-Time Alignment of Diffusion Models with Direct Noise Optimization
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In this work, we focus on the alignment problem of diffusion models with a continuous reward function, which represents specific objectives for downstream tasks, such as increasing darkness or improving the aesthetics of images. The central goal of the alignment problem is to adjust the distribution learned by diffusion models such that the generated samples maximize the target reward function. We propose a novel alignment approach, named Direct Noise Optimization (DNO), that optimizes the injected noise during the sampling process of diffusion models. By design, DNO operates at inference-time, and thus is tuning-free and prompt-agnostic, with the alignment occurring in an online fashion during generation. We rigorously study the theoretical properties of DNO and also propose variants to deal with non-differentiable reward functions. Furthermore, we identify that naive implementation of DNO occasionally suffers from the out-of-distribution reward hacking problem, where optimized samples have high rewards but are no longer in the support of the pretrained distribution. To remedy this issue, we leverage classical high-dimensional statistics theory to an effective probability regularization technique. We conduct extensive experiments on several important reward functions and demonstrate that the proposed DNO approach can achieve state-of-the-art reward scores within a reasonable time budget for generation.
Forward citations
Cited by 7 Pith papers
-
Optimization under Persistent State-Dependent Bias: Gradient-based Method and Complexity Analysis
Residual Learning, a proposed bilevel gradient method, claims exact convergence under state-dependent analog-hardware bias with rate O~(kappa1*kappa2^4*sigma^2/(mu*K)).
-
Self-Improving Diffusion Classifiers with Minority Preference Optimization
Fine-tuning a diffusion model with a reconstruction-error minority reward via LoRA+GRPO improves zero-shot diffusion classification by expanding low-density coverage.
-
Na\"ive PAINE: Lightweight Text-to-Image Generation Improvement with Prompt Evaluation
A lightweight predictor ranks initial noises by expected human-preference score for a prompt, selecting the best few for diffusion generation and reporting prompt difficulty.
-
Diffusion Sampling Path Tells More: An Efficient Plug-and-Play Strategy for Sample Filtering
CFG-Rejection filters low-quality diffusion samples early using the accumulated norm of the classifier-free guidance vector, improving quality scores without external reward models.
-
Test-Time Scaling of Diffusion Models via Noise Trajectory Search
An epsilon-greedy search over per-step noise trajectories improves proxy rewards in diffusion image generation without retraining.
-
Scaling Image and Video Generation via Test-Time Evolutionary Search
Evolutionary search over denoising trajectories improves image and video generation quality and diversity as test-time compute increases, without retraining the generative model.
-
Ctrl-Z Sampling: Scaling Diffusion Sampling with Controlled Random Zigzag Explorations
Ctrl-Z Sampling improves text-to-image outputs by adaptively rolling back and re-exploring when a reward model flags a quality plateau, at roughly 3 to 9 times the usual compute.
Discussion (0). Sign in to comment.