{"id":"d708147c-2d79-4167-b21c-e8d05d5757ba","arxiv_id":"2511.19985","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A training-free inpainting method optimizes the initial seed noise in the spectral domain under a linearized denoising approximation, beating specialist inpainting models on FFHQ, DIV2K, and BrushBench.","lead":"This paper optimizes the random seed noise of a text-to-image diffusion model so that inpainting results better match the un-masked parts of the image. The method is training-free and uses a linearized denoising path plus spectral-domain optimization, reporting improved inpainting quality on three benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Linear-trajectory approximation in Eq. (2) is unvalidated; if the flow is curved, the optimized seed may not actually match the unmasked region, undermining the claimed mechanism.","rationale":"The Reader's weakest_assumption identifies exactly this: the straight-line approximation in Eq. (2) is not validated against unrolled gradients. My analysis confirms that this is the most load-bearing point. The paper is honest about acknowledging the approximation error, but it does not quantify it or show that the optimized seed actually reduces the true inpainting loss. Without this, the method's stated mechanism—that optimizing the seed via the linearized objective matches the unmasked region—remains an untested hypothesis. The strong quantitative results in Tabs. 1-3 are still empirical evidence for the method's usefulness, but they do not confirm the proposed reason. The conditional verdict is appropriate: the paper should be accepted only after this validation. I do not see a reason to move the verdict to reject or accept unconditionally. The other concerns (no error bars, missing code, narrower SOTA claim) are secondary and addressable. Thus the reader's CONDITIONAL verdict remains unchanged.","tokens_in":14713,"tokens_out":3337,"duration_ms":37339,"concrete_test":"On a random subset of, say, 20 images from FFHQ and 20 from DIV2K: (1) Record the full denoising trajectory x(t) for t=0..T for both the initial seed and the optimized seed. (2) Compute the mean/max deviation between the actual trajectory and the linear interpolation of Eq. (2): |x(t) - (x_T + (t/T)(D_T(x_T)-x_T))|. (3) Compute L_linear and L_true = ||y - A D_T(x_T)||^2 before and after optimization. (4) For a smaller subset (5 images), run full unrolled gradient descent with gradient checkpointing (or using a smaller T) and compare the achieved L_true against SONIC's optimized seed. If L_true does not decrease substantially for SONIC while the linearized loss does, or if trajectory curvature exceeds a small threshold relative to the update step, the linearization is the bottleneck and the central claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The method's core enabler is the linearization in Eq. (2): the denoising trajectory x(t) is approximated as a straight line from x_T to D_T(x_T), allowing gradients to flow through the additive x_T term while treating D_T(x_T) as constant. This replaces the true Jacobian of D_T with the identity. The loss value at the current iterate equals the true inpainting loss, but the gradient direction is correct only if D_T is approximately the identity plus a constant offset along the optimized direction. In practice, SD3.5 is a rectified flow model, not a perfectly straight flow, and the paper acknowledges approximation error exists but never measures it. If the trajectory has significant curvature, the optimized seed may reduce L_linear while leaving the true objective L_true = ||y - A D_T(x_T)||^2 unchanged or even worse. Since the paper's central claim is that optimizing the seed to match the unmasked region improves inpainting, a failure of this approximation would break the proposed mechanism: the gains observed in Tabs. 1-3 might then stem from incidental properties of spectral-domain Adam or seed perturbation rather than from actually matching the unmasked region. The paper does not report L_true before/after optimization, nor does it compare against full unrolled gradient descent on any subset. This is the single most load-bearing concern because it directly questions whether the stated innovation does what it claims.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SONIC, a training-free image inpainting method that optimizes the initial seed noise of a pre-trained flow model (Stable Diffusion 3.5) so that the denoised output matches the known (unmasked) image content. The key technical ideas are: (i) a linearization of the denoising trajectory (Eq. 2) that allows the gradient of the reconstruction loss to be computed without backpropagating through the denoiser; (ii) optimization in the spectral domain (Fourier coefficients) with Adam; and (iii) masking out gradient updates in the masked region to keep the noise on the manifold. The method is evaluated on FFHQ, DIV2K, and BrushBench, and reported to outperform strong baselines (BLD-SD3.5, FlowChef, FlowDPS, FLAIR, BrushNet) in SSIM, LPIPS, and FID, with ablations showing the importance of spectral optimization and gradient masking.","tokens_in":15017,"tokens_out":5737,"duration_ms":60045,"significance":"If the results hold, this is a valuable contribution: it enables training-free inpainting with an off-the-shelf text-to-image model at a computational cost comparable to existing guidance methods, while achieving state-of-the-art perceptual quality. The paper ships several strong components: a concrete optimization objective (Eq. 3), a linearization that avoids unrolling, an ablation study isolating the key design choices, and a clear experimental protocol with three diverse datasets. The main reservation is that the central linearization assumption is not directly validated, which leaves the mechanism of the method less certain than the empirical gains suggest.","major_comments":[{"comment":"The linearization of the denoising trajectory is load-bearing but unvalidated. At the current iterate, L_linear equals the true loss L_true = ||y - A D_T(x_T)||², but the gradient is computed with the D_T(x_T)-x_T term frozen, i.e., the Jacobian of D_T is replaced by the identity. This is a heuristic; the claim that 'in theory this should be a perfect approximation of flow models' (Sec. 3.2) is not correct, because even for a perfectly rectified flow the endpoint D_T(x_T) depends on x_T through the whole trajectory, so the Jacobian is not generally identity. The paper provides no evidence that the optimization actually decreases L_true: it does not report L_true before/after optimization, nor does it compare against unrolled gradients (even for a small subset or for a single denoising step). Quantitative support is needed to verify that the improved inpainting results arise from matching","section":"§3.2, Eq. (2)-(3)"},{"comment":"All quantitative results are reported as single numbers with no variance or statistical significance. Some differences are modest, e.g., Table 3 LPIPS: ours 0.153 vs. FlowChef 0.161, SSIM 0.861 vs. FLAIR 0.855. With a single run per method, it is unclear whether these gaps are reliable or within run-to-run variability. Since the method starts from random noise, multiple random seeds are expected to produce different outcomes. Please report mean ± standard deviation over at least three seeds for the main tables and the ablation (Table 4), or provide a clear deterministic-evaluation statement if a fixed seed is used.","section":"Tables 1-3, §4.2"},{"comment":"The implementation of gradient masking in the spectral domain is underspecified. The optimization is performed on the Fourier coefficients X_T, but the paper states that gradient updates are masked out in the masked region of the latent. Since a Fourier coefficient is non-local, it is not clear how a spatial mask is applied to the gradient. The authors should describe the exact procedure: e.g., compute the gradient w.r.t. x_T, apply a binary mask in the spatial domain, then transform to the spectral domain to update X_T. Without this detail, the method is not fully reproducible.","section":"§3.3-3.4"},{"comment":"The statement 'in theory this should be a perfect approximation of flow models' is misleading and should be corrected. Even for an ideal rectified flow, the mapping from initial noise to output is not affine with identity Jacobian; the linearization is a practical approximation whose error should be analyzed or at least empirically bounded. Please rephrase and provide a discussion of the conditions under which the approximation is expected to be good, or a measurement of the approximation error.","section":"§3.2, second paragraph"}],"minor_comments":[{"comment":"The meaning of '400 NFE, which we use for our method' is ambiguous. A clearer description of the computational budget for the proposed method (number of denoiser calls in the optimization loop plus final denoising) would help the reader compare runtime fairly with baselines.","section":"§3.4"},{"comment":"Consider bolding the best value in each row/column for readability, and adding a footnote on the number of runs if single-run evaluation is retained.","section":"Tables 1-3"},{"comment":"The prompt generation via a VLM is a potentially significant variable; please provide in the main text a short summary of the prompt template and state whether the same prompts are used for all methods (this is mentioned, but a one-sentence clarification near the baselines would help).","section":"§4.1"},{"comment":"The diagram is dense; please add a legend or explanation for the stop-gradient symbol and the 'nearest-pixel fill' step in the caption.","section":"Figure 4"},{"comment":"There are a few typos and grammatical slips (e.g., 'the denoising trajectory x(t) is well approximated by a straight line', 'we find empirically in Sec. 4.3 produces inferior results'). A careful proofread is advised.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper studies a sound empirical question and reports consistent gains, but the central mechanism—that the linearized objective actually makes the optimized seed match the unmasked region—is not directly verified. The missing validation is the main obstacle to accepting as-is. I do not see circularity: the method is evaluated on test splits and no constants are fit to test data. The single-run reporting is a standard but important weakness. If the authors supply the requested experiments (L_true tracking, comparison with unrolled gradients, multiple seeds), the paper would be a strong candidate for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is worth reading, and I'd send it to reviewers. The idea is refreshingly simple: for inpainting with a frozen text-to-image model, don't touch the denoiser—just optimize the initial noise so the unmasked region of the denoised output matches the input. They make it tractable by linearizing the trajectory (treating the flow endpoint as a stop-gradient constant) and by doing the optimization in the Fourier domain with Adam, plus masking gradient updates in the masked region. That combination consistently beats BLD, FLAIR, FlowChef, FlowDPS, and even trained BrushNet on SSIM, LPIPS, and FID across FFHQ, DIV2K, and BrushBench, and the ablations show both spectral optimization and gradient masking are doing real work. That's a genuine contribution.\n\nThe closest prior work is the concurrent FlowOpt, and the authors are upfront about it; their spectral-domain preconditioning is the differentiator, and they show spatial optimization degrades results.\n\nThe soft spots are real but addressable. The biggest is the linearization: they approximate the whole denoising path as a straight line, which is the core enabler, but they never measure how much the true objective changes versus the linearized one. If the flow is curved, the optimized seed may reduce the linear loss without actually matching the unmasked region. The stress-test note is right to flag this; it's a load-bearing assumption, not a footnote. They acknowledge in Sec. 3.2 that approximation error exists, but they never quantify it, and the limitations section omits it entirely. A simple experiment—track L_true before/after optimization on a subset, or compare against a few unrolled steps—would settle it. Second, all numbers are single runs; for stochastic generative baselines, no seed variance or error bars. That matters when claims are about small metric gaps. Third, the 'state of the art' claim is slightly over-broad: on BrushBench they're third on IR/HPS/AS, so the paper should say 'state of the art on structure/perceptual metrics' rather than flatly outperforming. Fourth, the code is promised but not released; the exact masking and spectral update details are under-specified.\n\nNone of that undermines the core result. The gains are consistent enough across three datasets that I'd bet on the mechanism, even if the linearization isn't perfectly validated. The paper deserves a serious referee, not a desk reject.\n\nRecommendation: accept conditional on releasing code, adding error bars, and a direct check of the linearization (e.g., true objective before/after, and ideally a comparison to unrolled gradients for a few images). With those, it's a solid paper.","headline":"Optimizing the initial noise with a linearized trajectory and spectral-domain Adam is a simple, effective trick that consistently beats strong baselines on three benchmarks, though the load-bearing linearization is never directly validated.","tokens_in":15528,"tokens_out":2838,"would_cite":true,"duration_ms":29451,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Optimizing the starting noise makes generic models out-inpaint specialized ones.","keywords":["image inpainting","diffusion models","initial seed noise","training-free","spectral optimization","linearization","flow models","inverse problems"],"falsifier":"Take a few masked images, find the optimal seed using the paper's linear method, then run the full denoiser on that seed and measure how well the unmasked region matches the input; also compare against seeds found by true backpropagation through the denoiser. If the linear seed's reconstruction error is much worse than the true-gradient seed's, the linearity assumption is the weak link.","tokens_in":14595,"feed_emoji":"🎨","tokens_out":4857,"duration_ms":43829,"temperature":0.7,"pith_summary":"The paper argues that training-free inpainting with generic text-to-image diffusion models fails not because the model lacks task-specific training, but because the initial noise seed is chosen randomly. It proposes to optimize that seed so that, once denoised, the unmasked parts of the image match the input. To make this feasible, it approximates the entire denoising trajectory as a straight line, avoiding backpropagation through the denoiser, and optimizes in the spectral (Fourier) domain to stabilize convergence. On three standard benchmarks, the method outperforms existing training-free and trained inpainting methods on perceptual metrics (SSIM, LPIPS, FID). If correct, generic models could replace specialized inpainting models with no extra training.","feed_headline":"Optimizing the starting noise makes generic models out-inpaint specialized ones.","feed_subtitle":"A training-free method linearizes the denoising path and tunes the seed in the frequency domain, topping SSIM, LPIPS, and FID.","key_machinery":"The central object is the initial seed noise x_T, optimized against the masked-observation loss L_linear = ||y − A([D_T(x_T) − x_T]_sg + x_T)||². The linearization replaces the entire T-step denoising path with a straight line from x_T to the predicted clean image D_T(x_T), with a stop-gradient on the endpoint, making the loss differentiable with respect to x_T without backpropagating through the denoiser. Spectral-domain optimization, where x_T = F⁻¹(X_T) and the Fourier coefficients X_T are updated, levels the learning pace across spatial frequencies and stabilizes convergence. Masking gradient updates in the masked region preserves the noise's validity as an initial seed.","core_discovery":"The central discovery is that the initial seed noise largely determines the final image structure, and that for inpainting this noise can be directly optimized to match the observed region. The key technical move is to linearize the denoising trajectory—treating the difference between the final clean estimate and the initial noise as constant—so the gradient flows to the noise without unrolling the denoiser. The authors then optimize this noise in the Fourier domain with a standard adaptive optimizer, masking updates inside the masked region to keep the noise on the valid seed manifold. Empirically, this yields state-of-the-art SSIM, LPIPS, and FID on FFHQ, DIV2K, and BrushBench with an off-","pith_inferences":["Inference: If the linear-trajectory assumption holds for modern flow models, this opens a route to solving other inverse problems by optimizing only the seed, without guidance or training; the paper only demonstrates inpainting.","Inference: The spectral-domain stabilization suggests that optimizing other latent-space parameters (e.g., prompts or guidance scales) might also benefit from frequency-domain preconditioning.","Inference: The linearity assumption implies a testable boundary: for very curved trajectories or long sampling schedules, the optimized seed's actual denoised output could drift from the target; measuring that drift would indicate when the method breaks.","Inference: Combining seed optimization with posterior-sampling guidance, rather than simple blending, could yield further gains; the paper uses a simple blending-based inpainter on top."],"forward_implications":["A single off-the-shelf text-to-image model can produce inpainting results that beat specialized trained models on perceptual metrics, without any training or fine-tuning.","The optimized seed can be plugged into any downstream training-free inpainting method, such as blending-based approaches, improving its consistency with the visible image.","The linearization makes optimization cheap enough to run in tens of steps, with runtime comparable to existing inverse-problem solvers.","The formulation is generic and, as the authors state, could extend to other linear inverse problems such as super-resolution and deblurring.","Ablations show that spectral-domain optimization and gradient masking are both critical; removing either degrades results substantially."],"fun_headline_variants":["Spectral noise tuning beats specialized inpainting models—no training needed","Training-free inpainting: optimize the spectral noise, not the network","Spectral seed optimization makes general diffusion models top inpainting benchmarks","Inpaint better by optimizing the initial noise in the Fourier domain","Linearized noise optimization: training-free inpainting that beats fine-tuned models"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The whole method rests on the assumption that a diffusion model's denoising path is nearly a straight line, so the shortcut gradient points toward a noise seed that really does match the visible image.","fun_headline_variants_meta":{"raw":{"variants":["Spectral noise tuning beats specialized inpainting models—no training needed","Training-free inpainting: optimize the spectral noise, not the network","Spectral seed optimization makes general diffusion models top inpainting benchmarks","Inpaint better by optimizing the initial noise in the Fourier domain","Linearized noise optimization: training-free inpainting that beats fine-tuned models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001249,"raw_usage":{"total_tokens":4957,"prompt_tokens":744,"completion_tokens":4213,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":488,"completion_tokens_details":{"reasoning_tokens":4117}},"tokens_in":488,"tokens_out":4213,"duration_ms":26821,"temperature":1.0,"reasoning_tokens":4117,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T20:21:27.038088+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a few masked images, find the optimal seed using the paper's linear method, then run the full denoiser on that seed and measure how well the unmasked region matches the input; also compare against seeds found by true backpropagation through the denoiser. If the linear seed's reconstruction error is much worse than the true-gradient seed's, the linearity assumption is the weak link.","supporting_citations":[],"review_version":1}