Pith. sign in

REVIEW 4 major objections 5 minor 4 cited by

Optimizing the starting noise makes generic models out-inpaint specialized ones.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 20:21 UTC pith:IBG6JJ6D

load-bearing objection Optimizing the initial noise with a linearized trajectory and spectral-domain Adam is a simple, effective trick that consistently beats strong baselines on three benchmarks, though the load-bearing linearization is never directly validated. the 4 major comments →

arxiv 2511.19985 v3 pith:IBG6JJ6D submitted 2025-11-25 cs.CV

SONIC: Spectral Optimization of Noise for Inpainting with Consistency

Seungyeon Baek , Erqun Dong , Shadan Namazifard , Mark J. Matthews , Kwang Moo Yi This is my paper
classification cs.CV
keywords image inpaintingdiffusion modelsinitial seed noisetraining-freespectral optimizationlinearizationflow modelsinverse problems
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that training-free inpainting with generic text-to-image diffusion models fails not because the model lacks task-specific training, but because the initial noise seed is chosen randomly. It proposes to optimize that seed so that, once denoised, the unmasked parts of the image match the input. To make this feasible, it approximates the entire denoising trajectory as a straight line, avoiding backpropagation through the denoiser, and optimizes in the spectral (Fourier) domain to stabilize convergence. On three standard benchmarks, the method outperforms existing training-free and trained inpainting methods on perceptual metrics (SSIM, LPIPS, FID). If correct, generic models could replace specialized inpainting models with no extra training.

Core claim

The central discovery is that the initial seed noise largely determines the final image structure, and that for inpainting this noise can be directly optimized to match the observed region. The key technical move is to linearize the denoising trajectory—treating the difference between the final clean estimate and the initial noise as constant—so the gradient flows to the noise without unrolling the denoiser. The authors then optimize this noise in the Fourier domain with a standard adaptive optimizer, masking updates inside the masked region to keep the noise on the valid seed manifold. Empirically, this yields state-of-the-art SSIM, LPIPS, and FID on FFHQ, DIV2K, and BrushBench with an off-

What carries the argument

The central object is the initial seed noise x_T, optimized against the masked-observation loss L_linear = ||y − A([D_T(x_T) − x_T]_sg + x_T)||². The linearization replaces the entire T-step denoising path with a straight line from x_T to the predicted clean image D_T(x_T), with a stop-gradient on the endpoint, making the loss differentiable with respect to x_T without backpropagating through the denoiser. Spectral-domain optimization, where x_T = F⁻¹(X_T) and the Fourier coefficients X_T are updated, levels the learning pace across spatial frequencies and stabilizes convergence. Masking gradient updates in the masked region preserves the noise's validity as an initial seed.

Load-bearing premise

The whole method rests on the assumption that a diffusion model's denoising path is nearly a straight line, so the shortcut gradient points toward a noise seed that really does match the visible image.

What would settle it

Take a few masked images, find the optimal seed using the paper's linear method, then run the full denoiser on that seed and measure how well the unmasked region matches the input; also compare against seeds found by true backpropagation through the denoiser. If the linear seed's reconstruction error is much worse than the true-gradient seed's, the linearity assumption is the weak link.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A single off-the-shelf text-to-image model can produce inpainting results that beat specialized trained models on perceptual metrics, without any training or fine-tuning.
  • The optimized seed can be plugged into any downstream training-free inpainting method, such as blending-based approaches, improving its consistency with the visible image.
  • The linearization makes optimization cheap enough to run in tens of steps, with runtime comparable to existing inverse-problem solvers.
  • The formulation is generic and, as the authors state, could extend to other linear inverse problems such as super-resolution and deblurring.
  • Ablations show that spectral-domain optimization and gradient masking are both critical; removing either degrades results substantially.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Inference: If the linear-trajectory assumption holds for modern flow models, this opens a route to solving other inverse problems by optimizing only the seed, without guidance or training; the paper only demonstrates inpainting.
  • Inference: The spectral-domain stabilization suggests that optimizing other latent-space parameters (e.g., prompts or guidance scales) might also benefit from frequency-domain preconditioning.
  • Inference: The linearity assumption implies a testable boundary: for very curved trajectories or long sampling schedules, the optimized seed's actual denoised output could drift from the target; measuring that drift would indicate when the method breaks.
  • Inference: Combining seed optimization with posterior-sampling guidance, rather than simple blending, could yield further gains; the paper uses a simple blending-based inpainter on top.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SONIC, a training-free image inpainting method that optimizes the initial seed noise of a pre-trained flow model (Stable Diffusion 3.5) so that the denoised output matches the known (unmasked) image content. The key technical ideas are: (i) a linearization of the denoising trajectory (Eq. 2) that allows the gradient of the reconstruction loss to be computed without backpropagating through the denoiser; (ii) optimization in the spectral domain (Fourier coefficients) with Adam; and (iii) masking out gradient updates in the masked region to keep the noise on the manifold. The method is evaluated on FFHQ, DIV2K, and BrushBench, and reported to outperform strong baselines (BLD-SD3.5, FlowChef, FlowDPS, FLAIR, BrushNet) in SSIM, LPIPS, and FID, with ablations showing the importance of spectral optimization and gradient masking.

Significance. If the results hold, this is a valuable contribution: it enables training-free inpainting with an off-the-shelf text-to-image model at a computational cost comparable to existing guidance methods, while achieving state-of-the-art perceptual quality. The paper ships several strong components: a concrete optimization objective (Eq. 3), a linearization that avoids unrolling, an ablation study isolating the key design choices, and a clear experimental protocol with three diverse datasets. The main reservation is that the central linearization assumption is not directly validated, which leaves the mechanism of the method less certain than the empirical gains suggest.

major comments (4)
  1. [§3.2, Eq. (2)-(3)] The linearization of the denoising trajectory is load-bearing but unvalidated. At the current iterate, L_linear equals the true loss L_true = ||y - A D_T(x_T)||², but the gradient is computed with the D_T(x_T)-x_T term frozen, i.e., the Jacobian of D_T is replaced by the identity. This is a heuristic; the claim that 'in theory this should be a perfect approximation of flow models' (Sec. 3.2) is not correct, because even for a perfectly rectified flow the endpoint D_T(x_T) depends on x_T through the whole trajectory, so the Jacobian is not generally identity. The paper provides no evidence that the optimization actually decreases L_true: it does not report L_true before/after optimization, nor does it compare against unrolled gradients (even for a small subset or for a single denoising step). Quantitative support is needed to verify that the improved inpainting results arise from matching
  2. [Tables 1-3, §4.2] All quantitative results are reported as single numbers with no variance or statistical significance. Some differences are modest, e.g., Table 3 LPIPS: ours 0.153 vs. FlowChef 0.161, SSIM 0.861 vs. FLAIR 0.855. With a single run per method, it is unclear whether these gaps are reliable or within run-to-run variability. Since the method starts from random noise, multiple random seeds are expected to produce different outcomes. Please report mean ± standard deviation over at least three seeds for the main tables and the ablation (Table 4), or provide a clear deterministic-evaluation statement if a fixed seed is used.
  3. [§3.3-3.4] The implementation of gradient masking in the spectral domain is underspecified. The optimization is performed on the Fourier coefficients X_T, but the paper states that gradient updates are masked out in the masked region of the latent. Since a Fourier coefficient is non-local, it is not clear how a spatial mask is applied to the gradient. The authors should describe the exact procedure: e.g., compute the gradient w.r.t. x_T, apply a binary mask in the spatial domain, then transform to the spectral domain to update X_T. Without this detail, the method is not fully reproducible.
  4. [§3.2, second paragraph] The statement 'in theory this should be a perfect approximation of flow models' is misleading and should be corrected. Even for an ideal rectified flow, the mapping from initial noise to output is not affine with identity Jacobian; the linearization is a practical approximation whose error should be analyzed or at least empirically bounded. Please rephrase and provide a discussion of the conditions under which the approximation is expected to be good, or a measurement of the approximation error.
minor comments (5)
  1. [§3.4] The meaning of '400 NFE, which we use for our method' is ambiguous. A clearer description of the computational budget for the proposed method (number of denoiser calls in the optimization loop plus final denoising) would help the reader compare runtime fairly with baselines.
  2. [Tables 1-3] Consider bolding the best value in each row/column for readability, and adding a footnote on the number of runs if single-run evaluation is retained.
  3. [§4.1] The prompt generation via a VLM is a potentially significant variable; please provide in the main text a short summary of the prompt template and state whether the same prompts are used for all methods (this is mentioned, but a one-sentence clarification near the baselines would help).
  4. [Figure 4] The diagram is dense; please add a legend or explanation for the stop-gradient symbol and the 'nearest-pixel fill' step in the caption.
  5. [General] There are a few typos and grammatical slips (e.g., 'the denoising trajectory x(t) is well approximated by a straight line', 'we find empirically in Sec. 4.3 produces inferior results'). A careful proofread is advised.

Circularity Check

0 steps flagged

No significant circularity: the derivation is self-contained and the reported gains come from an externally evaluated optimization procedure.

full rationale

The paper's core derivation is an optimization scheme, not a fitted prediction. Equation (3) is a standard masked reconstruction loss; the stop-gradient in Eq. (2) makes the gradient computation cheap but does not change the loss value relative to Eq. (1). The method optimizes the initial noise for each test image using only the unmasked region, then evaluates the resulting inpainted image on held-out benchmarks. No constant or parameter is fitted to the test outputs and then renamed as a prediction. The linear-trajectory approximation is a real correctness/robustness concern, but it is not circular: the paper does not define the objective in terms of the final evaluation metric, and it does not claim that the approximation is exact. The citations to prior work on seed-noise influence and frequency-domain guidance are external, not self-citations, and none is used to forbid alternatives or to supply the method's core result. Test-time optimization on the unmasked region is intrinsic to the inpainting task, and the masked-region content is not forced by construction. The mild issue of hyperparameters being selected on the same datasets used for reporting affects generalizability but is not a circular-derivation issue. Overall, the derivation chain from objective to optimization to evaluation does not reduce to its own inputs.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 0 invented entities

The method introduces no new physical or architectural entities. The free parameters are optimization hyperparameters chosen by hand on the target datasets; the axioms are empirical assumptions about flow-model trajectory shape and optimization stability.

free parameters (4)
  • Adam learning rate = 3.0
    Hand-picked; critical for convergence of the seed optimization (Sec. 3.4).
  • Number of optimization iterations = ~20 (400 NFE total / T=20)
    Chosen; the paper says 'tens of optimization steps' and reports using 400 NFE.
  • Classifier-free guidance scale = 2.0
    Follows FLAIR [10], but still a choice that affects results.
  • Denoising steps T = 20
    Chosen for the denoiser; affects the linearization quality and compute.
axioms (3)
  • domain assumption Denoising trajectories of rectified flow models are approximately linear.
    Used to justify Eq. (2) and the entire linearized optimization; the paper acknowledges approximation error but relies on it.
  • domain assumption Optimizing the seed noise in the Fourier domain keeps the iterate near the manifold of valid initial noises.
    Empirical heuristic motivating Sec. 3.3; no theoretical guarantee is provided.
  • domain assumption VLM-generated prompts provide a fair and equivalent conditioning signal for all compared methods.
    Prompts are generated from the masked image and used for all methods; differences in prompt-phrasing sensitivity are not controlled.

pith-pipeline@v1.3.0-alltime-deepseek · 14450 in / 13005 out tokens · 129387 ms · 2026-08-03T20:21:27.038088+00:00 · methodology

0 comments
read the original abstract

We propose a novel training-free method for inpainting with off-the-shelf text-to-image models. While guidance-based methods in theory allow generic models to be used for inverse problems such as inpainting -- in practice their effectiveness is limited, leading to the necessity of specialized inpainting-specific models. In this work, we argue the missing ingredient for training-free generic model usage is proper optimization of the initial noise sample. We optimize the initial noise to approximately reproduce the unmasked image, in as few as tens of optimization steps, then use it with a conventional training-free inpainting method. Critically, we propose two core ideas that make this possible: (i) we perform linear approximation that avoids the costly and often impractical unrolling required to relate the initial noise sample to model output -- which potentially is why this relationship was previously overlooked; and (ii) perform spectral preconditioning by optimizing the initial noise sample in the spectral domain with Adam, which stabilizes the optimization. We demonstrate our method on various inpainting tasks, outperforming the state of the art. Project website: https://ubc-vision.github.io/sonic/

Figures

Figures reproduced from arXiv: 2511.19985 by Erqun Dong, Kwang Moo Yi, Mark J. Matthews, Seungyeon Baek, Shadan Namazifard.

Figure 1
Figure 1. Figure 1: Teaser – We propose a novel training-free method of inpainting that focuses exclusively on the initial seed noise. (Top row) We show the denoising result of an initial seed noise, as we optimize the seed noise using our method. We optimize the seed noise to faithfully regenerate the non-masked regions of the input image, so as to obtain more consistent inpainting results. (Bottom row) Inpainting results of… view at source ↗
Figure 3
Figure 3. Figure 3: Optimizing in the spectral domain is important – We show examples of how the optimized initial seed noise de￾noises during the optimization process, when optimized to match the non-masked regions in [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Method overview – We optimize the initial seed noise in the spectral domain XT , starting from a random noise xT , such that our denoised latent matches the masked observation y in the latent space. To allow partial observations to be encoded, we use nearest-pixel filling before passing it into the encoder. We then compute the masked mean square error in the latent space, comparing it with a fully denoised… view at source ↗
Figure 5
Figure 5. Figure 5: Optimizing in the spectral domain – (Left) We show an example convergence graph of Eq. (3) when optimizing in the spatial domain vs spectral, for the same example in [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative results – We show results across different methods and datasets. We denote the masked areas with black rectangles on the ground truth images. Our method provides the best inpainting results, especially aligning well with the layout and also the lighting conditions of the input image. Zoom in for better detail [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative comparison of inpainting results on FFHQ dataset. 15 samples drawn at random from the [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative comparison of inpainting results on DIV2K dataset. 15 samples drawn at random from [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Qualitative comparison of inpainting results on BrushBench dataset. 15 samples drawn at random [PITH_FULL_IMAGE:figures/full_fig_p015_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Map2World: Segment Map Conditioned Text to 3D World Generation

    cs.CV 2026-05 unverdicted novelty 6.0

    Map2World produces scale-consistent 3D worlds from text and arbitrary segment maps via a detail enhancer that incorporates global structure information.

  2. InpaintSLat: Inpainting Structured 3D Latents via Initial Noise Optimization

    cs.CV 2026-05 unverdicted novelty 6.0

    Optimizing initial noise via backpropagation approximation and spectral parameterization in structured 3D latent diffusion yields higher contextual consistency and prompt alignment in training-free inpainting.

  3. Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint

    cs.LG 2026-05 unverdicted novelty 4.0

    In one Parkinson's patient, six occlusal probes produce overlapping gait scores and UMAP embeddings, so observable performance does not uniquely identify adaptive system state under VDO constraint.

  4. Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint

    cs.LG 2026-05 unverdicted novelty 3.0

    In a single Parkinson's patient, gait conditions with comparable linear performance metrics showed different temporal organizations in dynamical state space and unsupervised latent embeddings when vertical occlusion d...

Reference graph

Works this paper leans on

57 extracted references · cited by 3 Pith papers

  1. [1]

    NTIRE 2017 chal- lenge on single image super-resolution: Dataset and study

    Eirikur Agustsson and Radu Timofte. NTIRE 2017 chal- lenge on single image super-resolution: Dataset and study. InIEEE Conf. Comput. Vis. Pattern Recog. Worksh., 2017. 2, 6, 7, 8

    2017

  2. [2]

    A noise is worth diffusion guidance.arXiv, 2024

    Donghoon Ahn, Jiwon Kang, Sanghyun Lee, Jaewon Min, Minjae Kim, Wooseok Jang, Hyoungwon Cho, Sayak Paul, SeonHwa Kim, Eunju Cha, Kyong Hwan Jin, and Seungry- ong Kim. A noise is worth diffusion guidance.arXiv, 2024. 3

    2024

  3. [3]

    Stable diffusion 3.5 medium: A 2.5 billion parameter multimodal diffusion transformer.https : / / huggingface

    Stability AI. Stable diffusion 3.5 medium: A 2.5 billion parameter multimodal diffusion transformer.https : / / huggingface . co / stabilityai / stable - diffusion-3.5-medium, 2024. 1, 2, 4, 5, 6, 7, 8

    2024

  4. [4]

    Claude code.https://claude.com/ claude-code, 2025

    Anthropic. Claude code.https://claude.com/ claude-code, 2025. 6, 12

    2025

  5. [5]

    Blended latent diffusion.ACM Trans

    Omri Avrahami, Ohad Fried, and Dani Lischinski. Blended latent diffusion.ACM Trans. Graph., 42(4), 2023. 1, 2, 3, 5, 6, 7, 8

    2023

  6. [6]

    Universal guidance for diffusion models

    Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geip- ing, and Tom Goldstein. Universal guidance for diffusion models. InInt. Conf. Learn. Represent., 2024. 3

    2024

  7. [7]

    Improving text-guided object inpainting with semantic pre-inpainting

    Yifu Chen, Jingwen Chen, Yingwei Pan, Yehao Li, Ting Yao, Zhineng Chen, and Tao Mei. Improving text-guided object inpainting with semantic pre-inpainting. InEur. Conf. Com- put. Vis., 2024. 3

    2024

  8. [8]

    Mar- tinez

    Ciprian Corneanu, Raghudeep Gadde, and Aleix M. Mar- tinez. Latentpaint: Image inpainting in latent space with dif- fusion models. InIEEE Winter Conf. Appl. Comput. Vis.,

  9. [9]

    Di- makis, and Mauricio Delbracio

    Giannis Daras, Hyungjin Chung, Chieh-Hsin Lai, Yuki Mit- sufuji, Jong Chul Ye, Peyman Milanfar, Alexandros G. Di- makis, and Mauricio Delbracio. A survey on diffusion mod- els for inverse problems.arXiv, 2024. 1

    2024

  10. [10]

    Solving inverse problems with flair

    Julius Erbach, Dominik Narnhofer, Andreas Dombos, Bernt Schiele, Jan Eric Lenssen, and Konrad Schindler. Solving inverse problems with flair. InAdv. Neural Inform. Process. Syst., 2025. 1, 2, 3, 5, 6, 7, 8

    2025

  11. [11]

    Scaling rec- tified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. Scaling rec- tified flow transformers for high-resolution image synthesis. InInternational Conference on Machine Learning, 2024. 4

    2024

  12. [12]

    Frequency-guided diffusion for training- free text-driven image translation

    Zheng Gao, Jifei Song, Zhensong Zhang, Jiankang Deng, and Ioannis Patras. Frequency-guided diffusion for training- free text-driven image translation. InInt. Conf. Comput. Vis.,

  13. [13]

    Zico Kolter, Ruslan Salakhutdinov, and Stefano Ermon

    Yutong He, Naoki Murata, Chieh-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Dongjun Kim, Wei-Hsiang Liao, Yuki Mitsufuji, J. Zico Kolter, Ruslan Salakhutdinov, and Stefano Ermon. Manifold preserving guided diffusion. InInt. Conf. Learn. Represent., 2024. 3

    2024

  14. [14]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. InAdv. Neural Inform. Process. Syst., 2017. 2, 6

    2017

  15. [15]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. InNeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021. 3, 5

    2021

  16. [16]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. InAdv. Neural Inform. Process. Syst., 2020. 1, 3

    2020

  17. [17]

    Diffusion model-based image editing: A survey.IEEE Trans

    Yi Huang, Jiancheng Huang, Yifan Liu, Mingfu Yan, Jiaxi Lv, Jianzhuang Liu, Wei Xiong, He Zhang, Liangliang Cao, and Shifeng Chen. Diffusion model-based image editing: A survey.IEEE Trans. Pattern Anal. Mach. Intell., 47(6), 2025. 1, 3

    2025

  18. [18]

    Uniedit-flow: Unleashing inversion and editing in the era of flow models.arXiv, 2025

    Guanlong Jiao, Biqing Huang, Kuan-Chieh Wang, and Ren- jie Liao. Uniedit-flow: Unleashing inversion and editing in the era of flow models.arXiv, 2025. 3

    2025

  19. [19]

    Brushnet: A plug-and-play image inpainting model with decomposed dual-branch diffusion

    Xuan Ju, Xian Liu, Xintao Wang, Yuxuan Bian, Ying Shan, and Qiang Xu. Brushnet: A plug-and-play image inpainting model with decomposed dual-branch diffusion. InEur. Conf. Comput. Vis., 2024. 1, 2, 3, 6, 7, 8

    2024

  20. [20]

    A style-based generator architecture for generative adversarial networks

    Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In IEEE Conf. Comput. Vis. Pattern Recog., 2019. 2, 5, 6, 7, 8

    2019

  21. [21]

    Denoising diffusion restoration models

    Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. InAdv. Neu- ral Inform. Process. Syst., 2022. 3

    2022

  22. [22]

    Flowdps: Flow-driven posterior sampling for inverse prob- lems

    Jeongsol Kim, Bryan Sangwoo Kim, and Jong Chul Ye. Flowdps: Flow-driven posterior sampling for inverse prob- lems. InInt. Conf. Comput. Vis., 2025. 2, 3, 6, 7, 8

    2025

  23. [23]

    Rad: Region-aware diffusion models for image inpainting

    Sora Kim, Sungho Suh, and Minsik Lee. Rad: Region-aware diffusion models for image inpainting. InIEEE Conf. Com- put. Vis. Pattern Recog., 2025. 3

    2025

  24. [24]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. InInt. Conf. Learn. Represent.,

  25. [25]

    Enhancing compositional text-to-image generation with reli- able random seeds

    Shuangqi Li, Hieu Le, Jingyi Xu, and Mathieu Salzmann. Enhancing compositional text-to-image generation with reli- able random seeds. InInt. Conf. Learn. Represent., 2025. 2, 3

    2025

  26. [26]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maxi- milian Nickel, and Matt Le. Flow matching for generative modeling. InInt. Conf. Learn. Represent., 2023. 1, 4

    2023

  27. [27]

    Corrfill: Enhancing faithfulness in reference-based inpainting with correspondence guidance in diffusion models

    Kuan-Hung Liu, Cheng-Kun Yang, Min-Hung Chen, Yu-Lun Liu, and Yen-Yu Lin. Corrfill: Enhancing faithfulness in reference-based inpainting with correspondence guidance in diffusion models. InIEEE Winter Conf. Appl. Comput. Vis.,

  28. [28]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. InInt. Conf. Learn. Represent., 2023. 1, 4

    2023

  29. [29]

    Repaint: Inpaint- ing using denoising diffusion probabilistic models

    Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpaint- ing using denoising diffusion probabilistic models. InIEEE Conf. Comput. Vis. Pattern Recog., 2022. 3

    2022

  30. [30]

    Is-diff: Im- proving diffusion-based inpainting with better initial seed

    Yongzhe Lyu, Yu Wu, Yutian Lin, and Bo Du. Is-diff: Im- proving diffusion-based inpainting with better initial seed. arXiv, 2025. 3

    2025

  31. [31]

    Hd-painter: High-resolution and prompt-faithful text-guided image inpainting with diffusion models

    Hayk Manukyan, Andranik Sargsyan, Barsegh Atanyan, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Hd-painter: High-resolution and prompt-faithful text-guided image inpainting with diffusion models. InInt. Conf. Learn. Represent., 2025. 3

    2025

  32. [32]

    Metaxas, and Yezhou Yang

    Maitreya Patel, Song Wen, Dimitris N. Metaxas, and Yezhou Yang. Steering rectified flow models in the vector field for controlled image generation. InInt. Conf. Comput. Vis.,

  33. [33]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. InInternational Conference on Machine Learning, 2021. 6

    2021

  34. [34]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. InIEEE Conf. Comput. Vis. Pattern Recog., 2022. 3, 6

    2022

  35. [35]

    Flowopt: Fast optimization through whole flow processes for training- free editing.arXiv, 2025

    Or Ronai, Vladimir Kulikov, and Tomer Michaeli. Flowopt: Fast optimization through whole flow processes for training- free editing.arXiv, 2025. 3

    2025

  36. [36]

    Dimakis, and Sanjay Shakkottai

    Litu Rout, Negin Raoof, Giannis Daras, Constantine Cara- manis, Alexandros G. Dimakis, and Sanjay Shakkottai. Solv- ing linear inverse problems provably via posterior sampling with latent diffusion models. InAdv. Neural Inform. Process. Syst., 2023. 3

    2023

  37. [37]

    Seyedmorteza Sadat, Tobias V ontobel, Farnood Salehi, and Romann M. Weber. Guidance in the frequency domain en- ables high-fidelity sampling at low cfg scales.arXiv, 2025. 2, 4

    2025

  38. [38]

    Laion-5b: an open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Laion-5b: an open large-scale dataset for training next generation image-text model...

    2022

  39. [39]

    From missing pieces to masterpieces: Image completion with context-adaptive diffusion.IEEE Trans

    Pourya Shamsolmoali, Masoumeh Zareapoor, Huiyu Zhou, Michael Felsberg, Dacheng Tao, and Xuelong Li. From missing pieces to masterpieces: Image completion with context-adaptive diffusion.IEEE Trans. Pattern Anal. Mach. Intell., 2025. 3

    2025

  40. [40]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. InInt. Conf. Learn. Repre- sent., 2021. 1, 3

    2021

  41. [41]

    Resolution-robust large mask inpainting with fourier convolutions

    Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. Resolution-robust large mask inpainting with fourier convolutions. InIEEE Winter Conf. Appl. Comput. Vis., 2022. 3

    2022

  42. [42]

    Diffusers: State-of-the-art diffu- sion models.https://github.com/huggingface/ diffusers, 2022

    Patrick von Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj, Dhruv Nair, Sayak Paul, William Berman, Yiyi Xu, Steven Liu, and Thomas Wolf. Diffusers: State-of-the-art diffu- sion models.https://github.com/huggingface/ diffusers, 2022. 6

    2022

  43. [43]

    Seeds of structure: Patch pca reveals universal com- positional cues in diffusion models

    Qingsong Wang, Zhengchao Wan, Mikhail Belkin, and Yusu Wang. Seeds of structure: Patch pca reveals universal com- positional cues in diffusion models. InAdv. Neural Inform. Process. Syst., 2025. 2, 3

    2025

  44. [44]

    Zero-shot image restoration using denoising diffusion null-space model

    Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. In Int. Conf. Learn. Represent., 2023. 3

    2023

  45. [45]

    Towards enhanced image inpainting: Mitigating unwanted object insertion and preserving color consistency

    Yikai Wang, Chenjie Cao, Junqiu Yu, Ke Fan, Xiangyang Xue, and Yanwei Fu. Towards enhanced image inpainting: Mitigating unwanted object insertion and preserving color consistency. InIEEE Conf. Comput. Vis. Pattern Recog.,

  46. [46]

    Image quality assessment: From error visibility to structural similarity.IEEE Trans

    Zhou Wang, Alan Bovik, Hamid Sheikh, and Eero Simon- celli. Image quality assessment: From error visibility to structural similarity.IEEE Trans. Image Process., 13, 2004. 2, 6

    2004

  47. [47]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis.arXiv, 2023

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis.arXiv, 2023. 2, 6

    2023

  48. [48]

    Turbofill: Adapting few-step text-to-image model for fast image inpainting

    Liangbin Xie, Daniil Pakhomov, Zhonghao Wang, Zongze Wu, Ziyan Chen, Yuqian Zhou, Haitian Zheng, Zhifei Zhang, Zhe Lin, Jiantao Zhou, and Chao Dong. Turbofill: Adapting few-step text-to-image model for fast image inpainting. In IEEE Conf. Comput. Vis. Pattern Recog., 2025. 3

    2025

  49. [49]

    Smartbrush: Text and shape guided object inpainting with diffusion model

    Shaoan Xie, Zhifei Zhang, Zhe Lin, Tobias Hinz, and Kun Zhang. Smartbrush: Text and shape guided object inpainting with diffusion model. InIEEE Conf. Comput. Vis. Pattern Recog., 2022. 3

    2022

  50. [50]

    Imagere- ward: Learning and evaluating human preferences for text- to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation. InAdv. Neural Inform. Process. Syst.,

  51. [51]

    Paint by example: Exemplar-based image editing with diffusion mod- els

    Binxin Yang, Shuyang Gu, Bo Zhang, Ting Zhang, Xuejin Chen, Xiaoyan Sun, Dong Chen, and Fang Wen. Paint by example: Exemplar-based image editing with diffusion mod- els. InIEEE Conf. Comput. Vis. Pattern Recog., 2023. 3

    2023

  52. [52]

    Uni-paint: A unified framework for multimodal image inpainting with pretrained diffusion model

    Shiyuan Yang, Xiaodong Chen, and Jing Liao. Uni-paint: A unified framework for multimodal image inpainting with pretrained diffusion model. InACM Int. Conf. Multimedia,

  53. [53]

    Freedom: Training-free energy-guided condi- tional diffusion model

    Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang. Freedom: Training-free energy-guided condi- tional diffusion model. InInt. Conf. Comput. Vis., 2023. 2, 4

    2023

  54. [54]

    Towards coherent image in- painting using denoising diffusion implicit models

    Guanhua Zhang, Jiabao Ji, Yang Zhang, Mo Yu, Tommi Jaakkola, and Shiyu Chang. Towards coherent image in- painting using denoising diffusion implicit models. InIn- ternational Conference on Machine Learning, 2023. 3

    2023

  55. [55]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. InInt. Conf. Comput. Vis., 2023. 2

    2023

  56. [56]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. InIEEE Conf. Comput. Vis. Pattern Recog., 2018. 2, 6

    2018

  57. [57]

    mask" or

    Junhao Zhuang, Yanhong Zeng, Wenran Liu, Chun Yuan, and Kai Chen. A task is worth one word: Learning with task prompts for high-quality versatile image inpainting. In Eur. Conf. Comput. Vis., 2024. 1, 3 5.1. Prompt for example figures The following prompt was used for Figs. 2, 3, 5 and 6. A high quality, 4K, realistic image of bright red mushrooms with wh...

    2024