{"id":"53ed8191-7e17-47e9-bb95-f7db0dd7b5ce","arxiv_id":"2412.08262","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"The authors prove that SNORE Prox, a stochastic proximal gradient descent using a denoiser, reaches a stationary point of its objective under nonconvex but weakly convex assumptions.","lead":"This paper proves convergence guarantees for SNORE Prox, a stochastic Plug-and-Play image restoration algorithm, under weak convexity and smoothness assumptions. It provides the first theoretical backing for an algorithm that already shows strong practical results, especially for inpainting.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The proof is sound, but 'convergence to a critical point' overstates Proposition 3, which only gives liminf ||∇F|| = 0 without boundedness or subsequence convergence of the iterates.","rationale":"The reader's weakest-assumption identification (Assumption 4) is reasonable but not the point where I would push. The theorem is explicitly conditional on Assumption 4, and the proof uses it in a standard way to control ∇f differences; the counterexample cited in Remark 1 is disclosed by the authors. The more load-bearing issue is the gap between the demonstrated statement and the paper's stated conclusion. Proposition 3 gives liminf ||∇F(x_k)|| = 0 almost surely, which is a standard stochastic-nonconvex result, but the text in Section 3 and the Conclusion claims convergence to a critical point. That requires either boundedness of iterates or convergence of a subsequence of x_k, neither of which is established or implied by the assumptions (F bounded below and smooth does not force boundedness). This is an interpretative overreach, not a flaw in the inequalities. The proposed test with a shelflike smooth objective would settle whether the stronger conclusion can hold; if the iterates can escape, the paper should be revised to state the weaker gradient-subsequence result. I therefore keep the reader's CONDITIONAL verdict: the proofs are acceptable, but the advertised conclusion needs qualification.","tokens_in":13646,"tokens_out":35176,"duration_ms":334376,"concrete_test":"Check whether the assumptions imply boundedness of {x_k}. A decisive construction: take F(x) = ∫_0^x (1+t^2)^{-1} dt (smooth, bounded below, no critical point, |F'| → 0 as |x| → ∞) and realize it as the SNORE Prox objective with a gradient-step denoiser satisfying Assumption 1 (choose hσ by inverse Gaussian deconvolution of F, truncated to be nonnegative). Run SNORE Prox with δ_k = c/k. If ||x_k|| → ∞ while ||∇F(x_k)|| → 0, the phrase 'converges to a critical point' is not justified; Proposition 3's liminf statement still holds but the advertised conclusion does not. If instead all such trajectories are bounded, the overstatement is harmless and the conclusion can be strengthened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The mathematical content of Proposition 3 (Section 3.3) is internally consistent: under Assumptions 1, 2, 4 and 5 it proves a.s. min_{k≤N} ||∇F(x_k)|| → 0 and liminf ||∇F(x_k)|| = 0. The load-bearing issue is the conclusion the paper draws from this. Section 3, the Conclusion, and the abstract describe the result as convergence of the iterates to a critical point x* with ∇F(x*) = 0. That is stronger than what is shown: no boundedness of {x_k} is proved, no convergent subsequence of iterates is exhibited, and the descent inequality does not preclude the iterates drifting to infinity along a 'shelf' where |∇F| decays to zero but no critical point exists. Assumption 4, by contrast, is an explicit hypothesis and is used correctly; the paper even cites a counterexample showing failure without it, so it is not a hidden gap. The gap is terminological and conclusory: the proven statement is a vanishing-gradient-along-a-subsequence result, not iterate convergence to a critical point.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper analyzes the convergence of SNORE Prox, a stochastic proximal gradient descent algorithm for plug-and-play / RED image restoration. The authors reformulate SNORE Prox as a stochastic proximal gradient method on an explicit objective F = f + λgσ, where gσ is defined through a denoiser potential hσ. Under assumptions including L-Lipschitz denoiser, weakly convex differentiable f, M-smoothness of f, and suitable step-size conditions, they prove residual convergence (Lemma 2), a gradient-norm bound (Lemma 3), convergence to a critical point neighborhood with constant step sizes (Proposition 1), and, with non-increasing step sizes satisfying Assumption 5, an almost sure result that the minimum and liminf of ‖∇F(xk)‖ tend to zero (Propositions 2 and 3). Numerical experiments on image inpainting illustrate the practical behavior and restoration performance. The central mathematical derivations appear sound, but the paper's stated conclusion that the iterates converge to a critical point is stronger than what Proposition 3 actually establishes, and the experiments include a case (σy = 0) that the theory explicitly excludes.","tokens_in":13888,"tokens_out":3330,"duration_ms":35783,"significance":"If the claims are read at the level actually proven, the paper makes a useful contribution: it extends stochastic proximal gradient convergence guarantees to weakly convex, M-smooth data-fidelity terms with non-convex denoiser-based regularizers, a regime not covered by existing SPGD results (Table 1). The proofs are detailed and mostly self-contained, with explicit step-size conditions and quantitative rates via Remark 3. The step-size trade-off and the explicit definition of the objective through gσ are strengths: the objective is not fitted to the theorem, and the proof does not assume the desired critical point. However, the advertised conclusion of iterate convergence to a critical point is not established; the proven statement is a vanishing gradient along a subsequence (liminf), which is a weaker form of stationarity. This overstatement, together with the admitted use of σy = 0 experiments outside the assumptions, tempers the significance pending a corrected presentation.","major_comments":[{"comment":"The paper repeatedly claims 'convergence to a critical point x*' (e.g., Section 3 opening, Proposition 3 discussion, and the Conclusion), but Proposition 3 only proves almost surely min_{k=0..N} ‖∇F(xk)‖ → 0 and liminf_{k→∞} ‖∇F(xk)‖ = 0. This is not equivalent to iterate convergence: no boundedness of {xk} is established, no convergent subsequence of iterates is exhibited, and the descent inequality does not preclude a drift to infinity along a 'shelf' on which the gradient norm decays to zero without a critical point. The mathematical content of Proposition 3 is correct, but the corresponding conclusions in the abstract, Section 3, and Section 5 must be reworded to state explicitly that the result is a vanishing-gradient-along-a-subsequence guarantee, not convergence of the iterates to a point with ∇F(x*) = 0.","section":"§3.3, Propositions 2–3, Abstract, Conclusion"},{"comment":"The experiments include a noiseless case σy = 0, where Appendix B states f(x) = i_{A^{-1}(y)} is not differentiable, so Assumption 2(b) fails. Because the convergence theory relies on the differentiability of f (Lemma 4, Lemma 5, and the proximal-optimality condition), the σy = 0 results in Figure 2 and Table 2 lie outside the theorem's scope. The authors acknowledge this in Appendix B, but the main text presents these results and the general statement 'we provide numerical experiments that illustrate the practical convergence of SNORE Prox' without qualification. A caveat must be added in Section 4, and the claims should be limited to the setting where Assumption 2(b) holds, or the theory must be extended to handle the non-smooth indicator term.","section":"§4 and Appendix B (σy = 0 experiments)"},{"comment":"The paper correctly identifies that Assumption 4 (M-smoothness) is load-bearing and cites a counterexample showing the conclusion can fail without it. That is a proper treatment of an explicit hypothesis. However, the manuscript should state more prominently that the convergence guarantee is conditional on an upper-curvature bound on f, not on the weaker weak-convexity condition alone; otherwise a reader may infer that the result covers arbitrary weakly convex f. Adding a sentence in the introduction or Section 3.1 that Assumption 4 is necessary for the gradient-norm guarantee would improve accuracy.","section":"§3, Remark 1 / Assumption 4"}],"minor_comments":[{"comment":"The phrase 'we prove the convergence of SNORE Prox under non convex assumptions' is ambiguous; it should specify that the result concerns the gradient norm of the objective, not necessarily the iterates.","section":"Abstract"},{"comment":"The text says 'the mean value of ∇F(xN) for N = 500 effectively decreases with the stepsize δ', but Table 3 shows ‖∇F(x500)‖ increasing from 0.83 at δ = 0.05 to 1.37 at δ = 2.0. The sentence should be rephrased as 'decreases as the step-size δ decreases,' which matches Proposition 1.","section":"Appendix B, Table 3"},{"comment":"In the display after 'Plugging δk = δ', the constant A2 is written with δ0 inside the parentheses; since δ0 = δ, this is correct but could be simplified to avoid notational redundancy. This is a presentation rather than a correctness issue.","section":"§3.2, Proposition 1 proof"},{"comment":"The upper bound in Lemma 2 has the sum ∑_{k=0}^{N} δ_k^2, but the proof sums from k=0 to N-1. The notation should be harmonized (e.g., by defining N appropriately or using N-1 consistently) to avoid a minor mismatch between the statement and proof.","section":"§2, Lemma 2 / equation (7)"},{"comment":"Reference [6] is cited for the counter-example in Remark 1 but is not otherwise discussed; a one-sentence explanation of the counterexample’s setting would help the reader understand the necessity of Assumption 4 without going to the source.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid theoretical contribution with correct proofs, but the advertised convergence claim overstates Proposition 3, and the experimental section includes a case that the assumptions exclude. These issues are fixable with rewording and caveats; the mathematical core does not require a new derivation. I see no citation-pattern or novelty concerns beyond the normal scope. The paper fits the journal's scope if the claims are made accurate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nMain take: this is a genuinely useful convergence result for SNORE Prox, a stochastic PnP variant that had strong empirical results but no theory. The paper extends the SPGD framework to weakly convex data fidelity and nonconvex regularization, and proves that the gradient of the objective goes to zero along a subsequence almost surely under appropriate step sizes. That is a real extension of the Ghadimi-Lan-Zhang line, and it fills a row that was missing in their Table 1.\n\nWhat's good: the technical work is careful. Lemma 2 cleanly generalizes the residual bound to weakly convex f, and Lemma 3 is a transparent descent estimate that yields the gradient-norm control. The step-size trade-off in Proposition 2 and the rates in Remark 3 are useful and honest. The paper also cites a counterexample showing that Assumption 4 cannot simply be dropped, which is a sign of care.\n\nSoft spots: the main issue is conclusory language. Proposition 3 proves a.s. liminf ||∇F(x_k)|| = 0, i.e., the gradient vanishes along a subsequence. The abstract and conclusion say \"convergence to a critical point,\" which implies iterate convergence. That is not established: no boundedness of iterates is proved, and the descent inequality doesn't rule out drift to infinity along a flat shelf. The authors do say in Section 3.3 that \"zero is an accumulation point of ||∇F||,\" but other statements overstate the result. This is fixable by rewording, but it should be fixed.\n\nMinor: the experiments include the sigma_y=0 case (pure inpainting) where f is an indicator function, so Assumption 2(b) fails. The paper acknowledges this in Appendix B, but doesn't flag it prominently in the main text. The experiments also use constant step sizes, so they illustrate Proposition 1 rather than Proposition 3. That's okay, but it means the main result isn't directly validated numerically.\n\nCitation pattern is fine; self-citations are contextual and not circular. No code is provided, but the proof is the deliverable.\n\nVerdict: worth engaging with. The overstatement is real but easily corrected, and the underlying mathematics looks sound. I'd send this to peer review and expect acceptance after a moderate revision.\n\nRecommend bringing it to the reading group? Maybe, if you care about stochastic PnP convergence. I'd cite it if I worked in that area.\n\nBest,\n[Your name]","headline":"Solid convergence analysis for SNORE Prox, with a fixable overstatement: it proves liminf of the gradient norm goes to zero, not iterate convergence to a critical point.","tokens_in":14383,"tokens_out":3143,"would_cite":true,"duration_ms":31197,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C26","68U10","90C15"],"pacs":[],"model":"deepseek-v4-flash","headline":"SNORE Prox provably converges to critical points in image restoration.","keywords":["image restoration","plug-and-play","stochastic proximal gradient descent","convergence analysis","nonconvex optimization","regularization by denoising","inpainting"],"falsifier":"Run SNORE Prox with decaying step sizes on the quadratic counter-example $F(x)=\\tfrac{a}{2}\\|x\\|^2+\\tfrac{\\lambda}{2}\\|x\\|^2$ with $a \\ge 1/\\delta_0$ and denoiser level $\\sigma$, and measure $\\min_{k\\le N}\\|\\nabla F(x_k)\\|^2$; the counter-example discussed in the paper predicts this quantity remains at least $\\sigma^2/4$ for every $N$, which would disprove the proposition if Assumption 4 were dropped. Directly computing this quantity for the same setup with an M-smooth f (e.g., choosing a below the bound) should show the gradient norm controllable, isolating the role of Assumption 4.","tokens_in":13466,"feed_emoji":"🖼️","tokens_out":14688,"duration_ms":120139,"temperature":0.7,"pith_summary":"Plug-and-play image restoration alternates data-fidelity steps with a denoising step; SNORE Prox is a stochastic variant that performs well on tasks like inpainting, but its convergence was unproven. This paper proves that SNORE Prox, read as a stochastic proximal gradient descent on an explicit objective, converges to critical points under nonconvex regularization and weakly convex data fidelity. With constant step sizes, the iterates land in a neighborhood of a critical point whose radius is controlled by the step size; with a decaying step-size schedule that sums to infinity while its squares sum to a finite value, the gradient of the objective vanishes along a subsequence almost surely. These guarantees turn a heuristic restoration engine into a principled solver of the underlying variational problem.","feed_headline":"SNORE Prox provably converges to critical points","feed_subtitle":"A stochastic denoising restoration method now has proximal-gradient-level convergence guarantees.","key_machinery":"The central machinery is the reformulation of SNORE Prox as a stochastic proximal gradient descent on $F = f + \\lambda g_\\sigma$, where the gradient-step denoiser $D_\\sigma = \\mathrm{Id} - \\nabla h_\\sigma$ turns the denoiser residual into an unbiased stochastic gradient of the explicit regularization $g_\\sigma(x) = \\sigma^{-2}\\,\\mathbb{E}_z\\, h_\\sigma(x+\\sigma z)$. The new analytical step is a descent estimate (Lemma 3) that controls $\\sum_k \\delta_k\\,\\mathbb{E}\\|\\nabla F(x_k)\\|^2$ using the $M$-smoothness of $f$, the Lipschitz smoothness of $g_\\sigma$, the proximal optimality condition, and a generalized residual bound for weakly convex $f$. From that estimate, the constant-step-size neighborhood result and the almost-sure vanishing of $\\min_{k\\le N}\\|\\nabla F(x_k)\\|$ follow by summation and a standard almost-supermartingale convergence theorem.","core_discovery":"The paper establishes that the stochastic proximal gradient descent view of SNORE Prox is mathematically sound. Under Assumptions 1, 2, 4 and 5, the iterates satisfy almost surely that $\\min_{k\\le N}\\|\\nabla F(x_k)\\|\\to 0$ and $\\liminf_{k\\to\\infty}\\|\\nabla F(x_k)\\|=0$, so the algorithm reaches a critical point of $F$ in the sense of vanishing gradient. This is obtained through a descent estimate bounding the weighted sum of expected squared gradient norms by a telescoping objective decrease plus the accumulated squared step sizes. With a constant step size $\\delta$, the average squared gradient norm is asymptotically dominated by a term proportional to $\\delta$, so the algorithm converges to a $\\delta$-neighborhood of a critical point. The analysis relaxes the assumptions of existing stochastic proximal gradient results: the data-fidelity term $f$ may be only weakly convex and $M$-smooth rather than convex, and the regularization $g$ is nonconvex and defined implicitly through a gradient-step denoiser.","pith_inferences":["Because the proof guarantees $\\liminf_k \\|\\nabla F(x_k)\\|=0$ in the decaying-step regime, a practitioner could use the running minimum of the computed gradient norm as a principled stopping criterion for SNORE Prox.","The same descent framework should extend to annealed variants such as Ann-SNORE Prox, where $\\sigma$ and $\\lambda$ change along iterations, since the gradient-step structure is preserved at each noise level; the main adaptation would be tracking the step-size bounds as the constants evolve.","The cited counter-example suggests that for data-fidelity terms with unbounded upper curvature, gradient-norm convergence may genuinely fail; the residual convergence (Lemma 2) may still hold in such cases, giving a weaker but useful guarantee.","The rates in Remark 3 are testable: one could run SNORE Prox on the inpainting benchmark with $\\delta_k \\propto 1/k^\\alpha$ and compare the measured average squared gradient norm against the predicted $O(N^{\\alpha-1})$ scaling."],"forward_implications":["With step sizes $\\delta_k = c/k^\\alpha$ for $\\alpha \\in (1/2,1)$, SNORE Prox converges to a critical point at rate $O(N^{\\alpha-1})$; with $\\delta_k = c/k$, the rate is $O(1/\\log N)$ (Remark 3).","A constant step size $\\delta$ makes the asymptotic average squared gradient norm proportional to $\\delta$, so users can trade accuracy against speed by choosing the step size.","Convergence holds for any finite-Lipschitz gradient-step denoiser, with no need to constrain the Lipschitz constant during training, contrary to some convergent plug-and-play methods.","The residual convergence bound generalizes the mini-batch stochastic approximation analysis of [7] to weakly convex data-fidelity terms, so the proof strategy transfers to other stochastic proximal algorithms in imaging."],"supporting_citations":[{"why":"Supplies the mini-batch SPGD residual convergence result that Lemma 2 generalizes to weakly convex f.","marker":"[7]"},{"why":"Introduces SNORE and SNORE Prox, the algorithm under study, and provides the empirical restoration results motivating the analysis.","marker":"[19]"},{"why":"Defines the gradient-step denoiser $D_\\sigma = \\mathrm{Id} - \\nabla h_\\sigma$ that gives Assumption 1 and the denoiser used in experiments.","marker":"[10]"},{"why":"Provides the counter-example showing that the $M$-smoothness assumption on f is necessary for the gradient-norm conclusion.","marker":"[6]"},{"why":"Gives the almost-supermartingale theorem used to convert residual bounds into almost-sure gradient convergence.","marker":"[20]"},{"why":"Tweedie's formula is the basis for interpreting the MMSE denoiser residual as a gradient, motivating Assumption 1.","marker":"[5]"},{"why":"A convergent plug-and-play result whose denoiser Lipschitz constraint is relaxed by the present analysis.","marker":"[11]"},{"why":"The closest prior SPGD convergence regime (weakly convex and Lipschitz f) that this paper extends.","marker":"[14]"}],"fun_headline_variants":["Proof: SNORE Prox hits critical points","Nonconvex convergence proof for SNORE Prox","SNORE Prox now has convergence guarantee","Stochastic denoising algorithm proven to converge","SNORE Prox provably reaches critical points"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is Assumption 4, that the data-fidelity term f has a finite upper curvature bound (M-smoothness); without it the descent estimate breaks, and the paper itself discusses a counter-example where the gradient norm cannot be driven to zero.","fun_headline_variants_meta":{"raw":{"variants":["Proof: SNORE Prox hits critical points","Nonconvex convergence proof for SNORE Prox","SNORE Prox now has convergence guarantee","Stochastic denoising algorithm proven to converge","SNORE Prox provably reaches critical points"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000172,"raw_usage":{"total_tokens":1249,"prompt_tokens":893,"completion_tokens":356,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":285}},"tokens_in":509,"tokens_out":356,"duration_ms":4351,"temperature":1.0,"reasoning_tokens":285,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:03:26.445028+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SNORE Prox with decaying step sizes on the quadratic counter-example $F(x)=\\tfrac{a}{2}\\|x\\|^2+\\tfrac{\\lambda}{2}\\|x\\|^2$ with $a \\ge 1/\\delta_0$ and denoiser level $\\sigma$, and measure $\\min_{k\\le N}\\|\\nabla F(x_k)\\|^2$; the counter-example discussed in the paper predicts this quantity remains at least $\\sigma^2/4$ for every $N$, which would disprove the proposition if Assumption 4 were dropped. Directly computing this quantity for the same setup with an M-smooth f (e.g., choosing a below the bound) should show the gradient norm controllable, isolating the role of Assumption 4.","supporting_citations":[{"cited_title":"Li and A","cited_arxiv_id":null,"evidence_quote":"The closest prior SPGD convergence regime (weakly convex and Lipschitz f) that this paper extends."},{"cited_title":"Ghadimi, G","cited_arxiv_id":null,"evidence_quote":"Supplies the mini-batch SPGD residual convergence result that Lemma 2 generalizes to weakly convex f."},{"cited_title":"Renaud, J","cited_arxiv_id":null,"evidence_quote":"Introduces SNORE and SNORE Prox, the algorithm under study, and provides the empirical restoration results motivating the analysis."},{"cited_title":"Hurault, A","cited_arxiv_id":null,"evidence_quote":"Defines the gradient-step denoiser $D_\\sigma = \\mathrm{Id} - \\nabla h_\\sigma$ that gives Assumption 1 and the denoiser used in experiments."},{"cited_title":"Robbins and D","cited_arxiv_id":null,"evidence_quote":"Gives the almost-supermartingale theorem used to convert residual bounds into almost-sure gradient convergence."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Tweedie's formula is the basis for interpreting the MMSE denoiser residual as a gradient, motivating Assumption 1."},{"cited_title":"Hurault, A","cited_arxiv_id":null,"evidence_quote":"A convergent plug-and-play result whose denoiser Lipschitz constraint is relaxed by the present analysis."}],"review_version":1}