Pith. sign in

REVIEW 3 major objections 5 minor 35 references

Convergence Analysis of a Proximal Stochastic Denoising Regularization Algorithm

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SNORE Prox provably converges to critical points in image restoration.

desk verdict Solid convergence analysis for SNORE Prox, with a fixable overstatement: it proves liminf of the gradient norm goes to zero, not iterate convergence to a critical point. read the letter →

arxiv 2412.08262 v2 pith:HC3BP4JU submitted 2024-12-11 eess.IV

Marien Renaud , Julien Hermant , Nicolas Papadakis This is my paper · ORCID
classification eess.IV MSC 90C2668U1090C15
keywords imagerestorationplug-and-playstochasticproximalgradientdescentconvergenceanalysisnonconvexoptimizationregularizationbydenoisinginpainting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Plug-and-play image restoration alternates data-fidelity steps with a denoising step; SNORE Prox is a stochastic variant that performs well on tasks like inpainting, but its convergence was unproven. This paper proves that SNORE Prox, read as a stochastic proximal gradient descent on an explicit objective, converges to critical points under nonconvex regularization and weakly convex data fidelity. With constant step sizes, the iterates land in a neighborhood of a critical point whose radius is controlled by the step size; with a decaying step-size schedule that sums to infinity while its squares sum to a finite value, the gradient of the objective vanishes along a subsequence almost surely. These guarantees turn a heuristic restoration engine into a principled solver of the underlying variational problem.

What carries the argument

The central machinery is the reformulation of SNORE Prox as a stochastic proximal gradient descent on $F = f + \lambda g_\sigma$, where the gradient-step denoiser $D_\sigma = \mathrm{Id} - \nabla h_\sigma$ turns the denoiser residual into an unbiased stochastic gradient of the explicit regularization $g_\sigma(x) = \sigma^{-2}\,\mathbb{E}_z\, h_\sigma(x+\sigma z)$. The new analytical step is a descent estimate (Lemma 3) that controls $\sum_k \delta_k\,\mathbb{E}\|\nabla F(x_k)\|^2$ using the $M$-smoothness of $f$, the Lipschitz smoothness of $g_\sigma$, the proximal optimality condition, and a generalized residual bound for weakly convex $f$. From that estimate, the constant-step-size neighborhood result and the almost-sure vanishing of $\min_{k\le N}\|\nabla F(x_k)\|$ follow by summation and a standard almost-supermartingale convergence theorem.

What would settle it

Run SNORE Prox with decaying step sizes on the quadratic counter-example $F(x)=\tfrac{a}{2}\|x\|^2+\tfrac{\lambda}{2}\|x\|^2$ with $a \ge 1/\delta_0$ and denoiser level $\sigma$, and measure $\min_{k\le N}\|\nabla F(x_k)\|^2$; the counter-example discussed in the paper predicts this quantity remains at least $\sigma^2/4$ for every $N$, which would disprove the proposition if Assumption 4 were dropped. Directly computing this quantity for the same setup with an M-smooth f (e.g., choosing a below the bound) should show the gradient norm controllable, isolating the role of Assumption 4.

Watch

Extended reading notes

Core claim

The paper establishes that the stochastic proximal gradient descent view of SNORE Prox is mathematically sound. Under Assumptions 1, 2, 4 and 5, the iterates satisfy almost surely that $\min_{k\le N}\|\nabla F(x_k)\|\to 0$ and $\liminf_{k\to\infty}\|\nabla F(x_k)\|=0$, so the algorithm reaches a critical point of $F$ in the sense of vanishing gradient. This is obtained through a descent estimate bounding the weighted sum of expected squared gradient norms by a telescoping objective decrease plus the accumulated squared step sizes. With a constant step size $\delta$, the average squared gradient norm is asymptotically dominated by a term proportional to $\delta$, so the algorithm converges to a $\delta$-neighborhood of a critical point. The analysis relaxes the assumptions of existing stochastic proximal gradient results: the data-fidelity term $f$ may be only weakly convex and $M$-smooth rather than convex, and the regularization $g$ is nonconvex and defined implicitly through a gradient-step denoiser.

Load-bearing premise

The load-bearing premise is Assumption 4, that the data-fidelity term f has a finite upper curvature bound (M-smoothness); without it the descent estimate breaks, and the paper itself discusses a counter-example where the gradient norm cannot be driven to zero.

Editorial extensions

If this is right

  • With step sizes $\delta_k = c/k^\alpha$ for $\alpha \in (1/2,1)$, SNORE Prox converges to a critical point at rate $O(N^{\alpha-1})$; with $\delta_k = c/k$, the rate is $O(1/\log N)$ (Remark 3).
  • A constant step size $\delta$ makes the asymptotic average squared gradient norm proportional to $\delta$, so users can trade accuracy against speed by choosing the step size.
  • Convergence holds for any finite-Lipschitz gradient-step denoiser, with no need to constrain the Lipschitz constant during training, contrary to some convergent plug-and-play methods.
  • The residual convergence bound generalizes the mini-batch stochastic approximation analysis of [7] to weakly convex data-fidelity terms, so the proof strategy transfers to other stochastic proximal algorithms in imaging.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the proof guarantees $\liminf_k \|\nabla F(x_k)\|=0$ in the decaying-step regime, a practitioner could use the running minimum of the computed gradient norm as a principled stopping criterion for SNORE Prox.
  • The same descent framework should extend to annealed variants such as Ann-SNORE Prox, where $\sigma$ and $\lambda$ change along iterations, since the gradient-step structure is preserved at each noise level; the main adaptation would be tracking the step-size bounds as the constants evolve.
  • The cited counter-example suggests that for data-fidelity terms with unbounded upper curvature, gradient-norm convergence may genuinely fail; the residual convergence (Lemma 2) may still hold in such cases, giving a weaker but useful guarantee.
  • The rates in Remark 3 are testable: one could run SNORE Prox on the inpainting benchmark with $\delta_k \propto 1/k^\alpha$ and compare the measured average squared gradient norm against the predicted $O(N^{\alpha-1})$ scaling.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper analyzes the convergence of SNORE Prox, a stochastic proximal gradient descent algorithm for plug-and-play / RED image restoration. The authors reformulate SNORE Prox as a stochastic proximal gradient method on an explicit objective F = f + λgσ, where gσ is defined through a denoiser potential hσ. Under assumptions including L-Lipschitz denoiser, weakly convex differentiable f, M-smoothness of f, and suitable step-size conditions, they prove residual convergence (Lemma 2), a gradient-norm bound (Lemma 3), convergence to a critical point neighborhood with constant step sizes (Proposition 1), and, with non-increasing step sizes satisfying Assumption 5, an almost sure result that the minimum and liminf of ‖∇F(xk)‖ tend to zero (Propositions 2 and 3). Numerical experiments on image inpainting illustrate the practical behavior and restoration performance. The central mathematical derivations appear sound, but the paper's stated conclusion that the iterates converge to a critical point is stronger than what Proposition 3 actually establishes, and the experiments include a case (σy = 0) that the theory explicitly excludes.

Significance. If the claims are read at the level actually proven, the paper makes a useful contribution: it extends stochastic proximal gradient convergence guarantees to weakly convex, M-smooth data-fidelity terms with non-convex denoiser-based regularizers, a regime not covered by existing SPGD results (Table 1). The proofs are detailed and mostly self-contained, with explicit step-size conditions and quantitative rates via Remark 3. The step-size trade-off and the explicit definition of the objective through gσ are strengths: the objective is not fitted to the theorem, and the proof does not assume the desired critical point. However, the advertised conclusion of iterate convergence to a critical point is not established; the proven statement is a vanishing gradient along a subsequence (liminf), which is a weaker form of stationarity. This overstatement, together with the admitted use of σy = 0 experiments outside the assumptions, tempers the significance pending a corrected presentation.

major comments (3)
  1. [§3.3, Propositions 2–3, Abstract, Conclusion] The paper repeatedly claims 'convergence to a critical point x*' (e.g., Section 3 opening, Proposition 3 discussion, and the Conclusion), but Proposition 3 only proves almost surely min_{k=0..N} ‖∇F(xk)‖ → 0 and liminf_{k→∞} ‖∇F(xk)‖ = 0. This is not equivalent to iterate convergence: no boundedness of {xk} is established, no convergent subsequence of iterates is exhibited, and the descent inequality does not preclude a drift to infinity along a 'shelf' on which the gradient norm decays to zero without a critical point. The mathematical content of Proposition 3 is correct, but the corresponding conclusions in the abstract, Section 3, and Section 5 must be reworded to state explicitly that the result is a vanishing-gradient-along-a-subsequence guarantee, not convergence of the iterates to a point with ∇F(x*) = 0.
  2. [§4 and Appendix B (σy = 0 experiments)] The experiments include a noiseless case σy = 0, where Appendix B states f(x) = i_{A^{-1}(y)} is not differentiable, so Assumption 2(b) fails. Because the convergence theory relies on the differentiability of f (Lemma 4, Lemma 5, and the proximal-optimality condition), the σy = 0 results in Figure 2 and Table 2 lie outside the theorem's scope. The authors acknowledge this in Appendix B, but the main text presents these results and the general statement 'we provide numerical experiments that illustrate the practical convergence of SNORE Prox' without qualification. A caveat must be added in Section 4, and the claims should be limited to the setting where Assumption 2(b) holds, or the theory must be extended to handle the non-smooth indicator term.
  3. [§3, Remark 1 / Assumption 4] The paper correctly identifies that Assumption 4 (M-smoothness) is load-bearing and cites a counterexample showing the conclusion can fail without it. That is a proper treatment of an explicit hypothesis. However, the manuscript should state more prominently that the convergence guarantee is conditional on an upper-curvature bound on f, not on the weaker weak-convexity condition alone; otherwise a reader may infer that the result covers arbitrary weakly convex f. Adding a sentence in the introduction or Section 3.1 that Assumption 4 is necessary for the gradient-norm guarantee would improve accuracy.
minor comments (5)
  1. [Abstract] The phrase 'we prove the convergence of SNORE Prox under non convex assumptions' is ambiguous; it should specify that the result concerns the gradient norm of the objective, not necessarily the iterates.
  2. [Appendix B, Table 3] The text says 'the mean value of ∇F(xN) for N = 500 effectively decreases with the stepsize δ', but Table 3 shows ‖∇F(x500)‖ increasing from 0.83 at δ = 0.05 to 1.37 at δ = 2.0. The sentence should be rephrased as 'decreases as the step-size δ decreases,' which matches Proposition 1.
  3. [§3.2, Proposition 1 proof] In the display after 'Plugging δk = δ', the constant A2 is written with δ0 inside the parentheses; since δ0 = δ, this is correct but could be simplified to avoid notational redundancy. This is a presentation rather than a correctness issue.
  4. [§2, Lemma 2 / equation (7)] The upper bound in Lemma 2 has the sum ∑_{k=0}^{N} δ_k^2, but the proof sums from k=0 to N-1. The notation should be harmonized (e.g., by defining N appropriately or using N-1 consistently) to avoid a minor mismatch between the statement and proof.
  5. [References] Reference [6] is cited for the counter-example in Remark 1 but is not otherwise discussed; a one-sentence explanation of the counterexample’s setting would help the reader understand the necessity of Assumption 4 without going to the source.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the convergence proof is self-contained; self-citations provide context only.

full rationale

The paper's derivation chain does not reduce to its inputs. The objective F = f + λg_σ is explicitly constructed from the denoiser potential in Eq. (3)-(4), and the SNORE Prox update (6) is then shown to be a stochastic proximal gradient step on this F. Lemma 3 derives the key gradient bound from the smoothness of g_σ (Lemma 1) and Assumption 4, using standard descent and Young inequalities, not from the desired conclusion. Lemma 2 is an extended and self-contained proof generalizing [7, Theorem 2], with the proof in Appendix A. Proposition 3 follows from Proposition 2 plus the Robbins–Siegmund theorem [20], an external result; no step-size choice or constant is fitted to the convergence claim. The citations to the authors' prior works [19] and [10] define the algorithm and the gradient-step denoiser used in experiments, but they are not invoked as evidence for the convergence theorems, and the paper explicitly notes that no convergence guarantee for SNORE Prox existed in [19]. The experimental Table 3 compares observed gradient norms to the trend predicted by Proposition 1, which is an illustration rather than a fitted input. The only caveat is terminological: Proposition 3 proves liminf ‖∇F(x_k)‖ = 0 and vanishing of min_{k≤N} ‖∇F(x_k)‖, which the text sometimes summarizes as 'convergence to a critical point'; that is an overstatement about iterates, not a circularity.

Assumptions & free parameters 0 free parameters · 7 assumptions · 0 invented entities

The theorem introduces no fitted constants. It relies on explicit regularity assumptions on the denoiser and fidelity; these are domain assumptions, not fitted parameters. No new physical or algorithmic entities are postulated.

assumptions (7)
  • domain assumption Assumption 1: Dσ = Id - ∇hσ for some potential hσ
    Reformulates the denoiser residual as a gradient of gσ; not true for arbitrary denoisers, but holds for gradient-step denoisers and MMSE denoisers.
  • domain assumption Assumption 2(a): Dσ is L-Lipschitz
    Used to bound the variance of the stochastic gradient ζ_k.
  • domain assumption Assumption 2(b): f is ρ-weakly convex
    Needed for the proximal operator to be single-valued and for the Lipschitz property of Prox in Lemma 5.
  • domain assumption Assumption 2(c): F is lower bounded
    Needed so the telescoping sums in the proofs do not diverge.
  • domain assumption Assumption 4: f is M-smooth
    Crucial for Lemma 3 and all gradient-norm convergence results; without it the main theorem can fail.
  • domain assumption Assumption 5: step sizes non-increasing with sum δ=∞ and sum δ²<∞
    Standard stochastic approximation condition used to get a.s. convergence of min gradient to zero.
  • standard math Robbins-Siegmund theorem
    Used in Proposition 2 to turn the almost supermartingale inequality into a.s. summability of δ_k||∇F||².

how reviews work

0 comments
Cite this review

Pith. "Pith review of Convergence Analysis of a Proximal Stochastic Denoising Regularization Algorithm." pith.science (2026). https://pith.science/paper/HC3BP4JU

@misc{pith2026241208262,
  author       = {Pith},
  title        = {Pith review of: Convergence Analysis of a Proximal Stochastic Denoising Regularization Algorithm},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HC3BP4JU}},
  note         = {Machine review of arXiv:2412.08262}
}
read the original abstract

Plug-and-Play methods for image restoration are iterative algorithms that solve a variational problem to recover a clean image from a degraded observation. These algorithms are known to be flexible to changes of degradation and to perform state-of-the-art restoration. Recently, significant efforts have been made to explore new stochastic algorithms based on the Plug-and-Play or REgularization by Denoising (RED) frameworks, such as SNORE, which is a convergent stochastic gradient descent algorithm. A variant of this algorithm, named SNORE Prox, reaches state-of-the-art performances, especially for inpainting tasks. However, the convergence of SNORE Prox, that can be seen as a stochastic proximal gradient descent, has not been analyzed so far. In this paper, we prove the convergence of SNORE Prox under non convex assumptions.

Figures

Figures reproduced from arXiv: 2412.08262 by the authors.

Figure 1
Figure 1. Inpainting, 50% missing pixels and a noise level σy = 5/255, with various restoration methods with a GS-denoiser trained on natural images. Note that SNORE Prox produces better qualitative results than SNORE or RED. 4 Experiments In this section, we evaluate quantitatively and qualitatively the practical gain of SNORE Prox. We focus on inpainting with random missing pixels. The de￾noiser is the gradient-step DRUNet … view at source ↗
Figure 2
Figure 2. Inpainting result for random missing pixel with probability p = 0.5 on CBSD68 dataset. Best and second￾best results are respectively displayed in bold and underlined. Algorithms are run with constant step-sizes [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 4
Figure 4. Additional inpainting results with 50% missing pixels and a noise level σy = 5/255, with various restoration methods with a GS-denoiser trained on natural images [PITH_FULL_IMAGE:figures/full_fig_p019_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 29 canonical work pages

  1. [1]

    Allen-Zhu

    Z. Allen-Zhu. Katyusha: The first direct acceleration of stochastic gradient meth- ods. Journal of Machine Learning Research , 2018

    work page 2018

  2. [2]

    Y. F. Atchadé, G. Fort, and E. Moulines. On perturbed proximal gradient algo- rithms. JMLR, 2017

    work page 2017

  3. [3]

    X. Chen, Z. Hou, C. A. Metzler, A. Maleki, and S. Jalali. Bagged deep im- age prior for recovering images in the presence of speckle noise.arXiv preprint arXiv:2402.15635, 2024

    work page Pith review arXiv 2024

  4. [4]

    K. Ding, J. Li, and K.-C. Toh. Nonconvex stochastic bregman proximal gradient method with application to deep learning.arXiv preprint arXiv:2306.14522, 2023

    arXiv 2023

  5. [5]

    B. Efron. Tweedie’s formula and selection bias.Journal of the American Statistical Association, 106(496):1602–1614, 2011. 12 Marien Renaud, Julien Hermant, and Nicolas Papadakis

    work page 2011

  6. [6]

    Y. Gao, A. Rodomanov, and S. U. Stich. Non-convex stochastic composite opti- mization with polyak momentum.arXiv preprint arXiv:2403.02967 , 2024

    arXiv 2024

  7. [7]

    Ghadimi, G

    S. Ghadimi, G. Lan, and H. Zhang. Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization.Math. Programming, 2016

    work page 2016

  8. [8]

    Y. Hu, M. Delbracio, P. Milanfar, and U. S. Kamilov. A restoration network as an implicit prior. arXiv preprint arXiv:2310.01391 , 2023

    arXiv 2023

Show all 35 references
  1. [9]

    Y. Hu, A. Peng, W. Gan, P. Milanfar, M. Delbracio, and U. S. Kamilov. Stochastic deep restoration priors for imaging inverse problems. arXiv preprint arXiv:2410.02057, 2024

    2024 arXiv

  2. [10]

    Hurault, A

    S. Hurault, A. Leclaire, and N. Papadakis. Gradient step denoiser for convergent plug-and-play. In ICLR, 2022

    2022

  3. [11]

    Hurault, A

    S. Hurault, A. Leclaire, and N. Papadakis. Proximal denoiser for convergent plug- and-play optimization with nonconvex regularization. InICML, 2022

    2022

  4. [12]

    J Reddi, S

    S. J Reddi, S. Sra, B. Poczos, and A. J. Smola. Proximal stochastic methods for nonsmooth nonconvex finite-sum optimization. InNIPS, volume 29, 2016

    2016

  5. [13]

    Laumont, V

    R. Laumont, V. De Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra. On maximum a posteriori estimation with plug & play priors and stochastic gradient descent. JMIV, 2023

    2023

  6. [14]

    Li and A

    X. Li and A. Milzarek. A unified convergence theorem for stochastic optimization methods. NeurIPS, 2022

    2022

  7. [15]

    S. Mallat. A wavelet tour of signal processing . Elsevier, 1999

    1999

  8. [16]

    Milzarek and J

    A. Milzarek and J. Qiu. Convergence of a normal map-based prox-sgd method under the kl inequality.arXiv preprint arXiv:2305.05828 , 2023

    2023

  9. [17]

    A. Nitanda. Stochastic proximal gradient descent with acceleration techniques. Advances in neural information processing systems , 2014

    2014

  10. [18]

    Pesquet, A

    J.-C. Pesquet, A. Repetti, M. Terris, and Y. Wiaux. Learning maximally monotone operators for image recovery.SIAM SIIMS, 2021

    2021

  11. [19]

    Renaud, J

    M. Renaud, J. Prost, A. Leclaire, and N. Papadakis. Plug-and-play image restora- tion with stochastic denoising regularization.ICML, 2024

    2024

  12. [20]

    Robbins and D

    H. Robbins and D. Siegmund. A convergence theorem for non negative almost supermartingales and some applications. InOptimizing Methods in Statistics . El- sevier, 1971

    1971

  13. [21]

    Romano, M

    Y. Romano, M. Elad, and P. Milanfar. The little engine that could: Regularization by denoising (red).SIAM Journal on Imaging Sciences , 2017

    2017

  14. [22]

    L. I. Rudin, S. Osher, and E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D: nonlinear phenomena , 1992

    1992

  15. [23]

    E. Ryu, J. Liu, S. Wang, X. Chen, Z. Wang, and W. Yin. Plug-and-play methods provably converge with properly trained denoisers. InICML, 2019

    2019

  16. [24]

    Shumaylov, J

    Z. Shumaylov, J. Budd, S. Mukherjee, and C.-B. Schönlieb. Weakly convex regu- larisers for inverse problems: Convergence of critical points and primal-dual opti- misation. ICML, 2024

    2024

  17. [25]

    Y. Sun, J. Liu, and U. Kamilov. Block coordinate regularization by denoising. NeurIPS, 32, 2019

    2019

  18. [26]

    Y. Sun, B. Wohlberg, and U. S. Kamilov. An online plug-and-play algorithm for regularized image reconstruction. IEEE TCI, 5(3):395–408, 2019

    2019

  19. [27]

    Y. Sun, Z. Wu, X. Xu, B. Wohlberg, and U. S. Kamilov. Scalable plug-and-play ADMM with convergence guarantees.IEEE TCI, 2021

    2021

  20. [28]

    Tang and M

    J. Tang and M. Davies. A fast stochastic plug-and-play ADMM for imaging inverse problems. arXiv preprint arXiv:2006.11630 , 2020

    2006 arXiv

  21. [29]

    Ulyanov, A

    D. Ulyanov, A. Vedaldi, and V. Lempitsky. Deep image prior. InCVPR, 2018. Analysis of a Proximal Stochastic Denoising Regularization Algorithm 13

    2018

  22. [30]

    S. V. Venkatakrishnan, C. A. Bouman, and B. Wohlberg. Plug-and-play priors for model based reconstruction. InGlobalSIP, 2013

    2013

  23. [31]

    D. Wei, P. Chen, and F. Li. Learning pseudo-contractive denoisers for inverse problems. In ICML, 2024

    2024

  24. [32]

    Xiao and T

    L. Xiao and T. Zhang. A proximal stochastic gradient method with progressive variance reduction. SIAM Journal on Optimization , 2014

    2014

  25. [33]

    Xu and Y

    Y. Xu and Y. Xu. Momentum-based variance-reduced proximal stochastic gradient method for composite nonconvex stochastic optimization.JOTA, 2023

    2023

  26. [34]

    Zhang, Y

    K. Zhang, Y. Li, W. Zuo, L. Zhang, L. Van Gool, and R. Timofte. Plug-and-play image restoration with deep denoiser prior.IEEE TPAMI, 2021

    2021

  27. [35]

    Y. Zhu, K. Zhang, J. Liang, J. Cao, B. Wen, R. Timofte, and L. Van Gool. De- noising diffusion models for plug-and-play image restoration. InIEEE/CVF, 2023. 14 Marien Renaud, Julien Hermant, and Nicolas Papadakis A Proof of Lemma 2 In this section, we provide the proof of Lemm...

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.