A Plug-and-Play Method with Inpainting Network for Bayesian Uncertainty Quantification in Imaging
Pith reviewed 2026-05-24 09:26 UTC · model grok-4.3
The pith
A convolutional neural network can replace hand-crafted inpainting in Bayesian uncertainty quantification for imaging
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper establishes that the inpainting procedure in the BUQO algorithm can be performed by a convolutional inpainting neural network, yielding a plug-and-play method based on the primal-dual Condat-Vu iterations for solving the hypothesis test on the existence of local artefacts in the MAP estimate, as demonstrated on Fourier undersampling and Radon transform problems from magnetic resonance and computed tomography imaging.
What carries the argument
The convolutional inpainting neural network serving as the data-driven replacement for the mathematical inpainting operator in the minimization problem that formulates the hypothesis test.
If this is right
- The method provides an efficient way to quantify uncertainty without sampling techniques.
- It can be applied to different Fourier undersampling scenarios in MRI and to CT with the Radon operator.
- The plug-and-play nature allows the inpainting network to be swapped or updated independently.
- Simulations show its use in probing local structures in reconstructed medical images.
Where Pith is reading between the lines
- If the network is trained on diverse enough data, the approach could extend to other inverse problems in imaging beyond the tested cases.
- Discrepancies between the network output and traditional inpainting might affect the reliability of the uncertainty quantification in edge cases.
- Future work could explore training the network jointly with the reconstruction process for better consistency.
Load-bearing premise
The trained inpainting network acts as a mathematically equivalent inpainting operator to the hand-crafted versions for the purpose of the hypothesis test.
What would settle it
A direct comparison where the new method and the original BUQO disagree on the presence of an artefact in a controlled simulation with known ground truth would indicate that the neural network does not preserve the required equivalence.
Figures
read the original abstract
We contribute to an uncertainty quantification problem in imaging that evaluates a hypothesis test questioning the existence of local "artefacts" appearing in the maximum a posteriori (MAP) estimate (obtained from standard numerical tools). Such a method, called Bayesian uncertainty quantification by optimization (BUQO), was introduced a few years ago as an efficient and scalable alternative to sampling methods when per-pixel error-bars are not needed. BUQO formulates a hypothesis test for probing the existence of local structures in the MAP estimate as a minimization problem, that can be solved efficiently with standard optimization algorithms. In this context, BUQO requires a "mathematical" definition of the "local artefact". This definition can be interpreted as an inpainting of the structure. However, only simple hand-crafted techniques have been proposed so far due to the complexity of the problem. In this work, we propose a data-driven alternative to BUQO where the inpainting procedure in the algorithm is performed using a convolutional inpainting neural network (NN). This results in a plug-and-play algorithm, based on the primal-dual Condat-Vu iterations,where the inpainting procedure is performed with a NN. The proposed approach is assessed on two image reconstruction problems inspired by medicine. We specifically perform simulations on two Fourier undersampling problems (discrete and non-uniform) encountered in magnetic resonance imaging, as well as a computed tomography problem using the Radon measurement operator.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper extends the BUQO framework for hypothesis testing of local artefacts in MAP estimates by replacing the hand-crafted inpainting operator with a convolutional neural network inpainter. The resulting plug-and-play algorithm embeds the NN inside primal-dual Condat-Vu iterations and is assessed on discrete and non-uniform Fourier undersampling (MRI) and Radon-transform (CT) reconstruction problems.
Significance. If the NN inpainting operator preserves the variational and monotonicity properties required by the original BUQO derivation, the method would allow data-driven handling of complex artefacts without manual design. The plug-and-play structure and medical-imaging simulations are practical strengths, but the significance hinges on whether the test statistic retains its interpretation.
major comments (2)
- [Method / algorithm description] The central substitution of the NN for the mathematical inpainting operator (described in the method section following the BUQO recall) is presented without any derivation or stated conditions showing that the trained network satisfies firm nonexpansiveness, monotonicity, or the same fixed-point characterization as the original hand-crafted operator. This equivalence is load-bearing for both convergence of the Condat-Vu scheme and validity of the hypothesis test.
- [Numerical experiments] The experimental assessment on the two MRI and one CT problems supplies no quantitative comparison of test decisions or convergence behavior against the original BUQO formulation, nor any diagnostic checking that the NN preserves the null-distribution properties of the test statistic. Without such evidence the claim that the procedure remains a valid uncertainty-quantification tool is unsupported.
minor comments (2)
- Notation distinguishing the NN-based inpainting operator from the original mathematical operator should be introduced explicitly to avoid ambiguity in the algorithm statement.
- The abstract states that the approach is assessed on the cited problems but provides no numerical results, error metrics, or figures; the main text should ensure all quantitative claims are accompanied by the corresponding tables or plots.
Simulated Author's Rebuttal
We thank the referee for the detailed review and valuable comments on our manuscript. We address each major comment below and outline the revisions we plan to make.
read point-by-point responses
-
Referee: [Method / algorithm description] The central substitution of the NN for the mathematical inpainting operator (described in the method section following the BUQO recall) is presented without any derivation or stated conditions showing that the trained network satisfies firm nonexpansiveness, monotonicity, or the same fixed-point characterization as the original hand-crafted operator. This equivalence is load-bearing for both convergence of the Condat-Vu scheme and validity of the hypothesis test.
Authors: We acknowledge that the manuscript provides no derivation establishing that the trained inpainting network satisfies firm nonexpansiveness, monotonicity, or the fixed-point properties of the original hand-crafted operator. The presentation treats the network as a plug-and-play module whose output is inserted into the Condat-Vu scheme. In the revised manuscript we will add an explicit discussion paragraph after the algorithm description that states the assumptions inherited from BUQO, notes that these properties are not verified for the trained network, and clarifies that convergence and test validity are supported only by the observed numerical behavior rather than by the original theoretical guarantees. revision: partial
-
Referee: [Numerical experiments] The experimental assessment on the two MRI and one CT problems supplies no quantitative comparison of test decisions or convergence behavior against the original BUQO formulation, nor any diagnostic checking that the NN preserves the null-distribution properties of the test statistic. Without such evidence the claim that the procedure remains a valid uncertainty-quantification tool is unsupported.
Authors: The reported experiments demonstrate feasibility on discrete and non-uniform Fourier undersampling (MRI) and Radon (CT) problems but contain no side-by-side quantitative comparison of test decisions or iteration counts with the original hand-crafted BUQO, nor any Monte-Carlo verification that the test statistic retains its null distribution under the NN inpainter. In the revision we will add (i) a table comparing test outcomes and convergence metrics on identical instances and (ii) empirical null-distribution diagnostics obtained by simulating under the null hypothesis, thereby providing direct evidence for the retained validity of the uncertainty-quantification procedure. revision: yes
Circularity Check
No significant circularity; extension of prior BUQO framework with external NN component
full rationale
The paper extends the existing BUQO hypothesis-testing minimization (solved via Condat-Vu) by substituting its hand-crafted inpainting operator with a trained convolutional NN. No equations, derivations, or self-citations are shown that reduce the test statistic, convergence properties, or validity claim to a fitted quantity defined by the same data, a self-referential definition, or a load-bearing self-citation chain. The central premise remains an independent modeling choice whose correctness rests on the (unproven here) equivalence assumption rather than on any circular reduction to inputs.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption A convolutional inpainting network can be trained to approximate the mathematical definition of local artefact inpainting used in BUQO
Reference graph
Works this paper leans on
-
[1]
Mallat, A Wavelet Tour of Signal Processing (Third Edition)
S. Mallat, A Wavelet Tour of Signal Processing (Third Edition). Boston: Academic Press, 2009
work page 2009
-
[2]
Daubechies, Ten Lectures on Wavelets
I. Daubechies, Ten Lectures on Wavelets . Society for Industrial and Applied Mathematics, 1992
work page 1992
-
[3]
A fast iterative shrinkage-thresholding algorithm for linear inverse problems,
A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM J. Imaging Sci. , vol. 2, pp. 183–202, 2009
work page 2009
-
[4]
Proximal splitting methods in signal processing,
P. L. Combettes and J.-C. Pesquet, “Proximal splitting methods in signal processing,” in Fixed-Point Algorithms for Inverse Problems in Science and Engineering, 2009
work page 2009
-
[5]
S. P. Boyd, N. Parikh, E. K. wah Chu, B. Peleato, and J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Found. Trends Mach. Learn. , vol. 3, pp. 1–122, 2011
work page 2011
-
[6]
A first-order primal-dual algorithm for convex problems with applications to imaging,
A. Chambolle and T. Pock, “A first-order primal-dual algorithm for convex problems with applications to imaging,” J. Math. Imaging Vision, vol. 40, no. 1, pp. 120–145, 2011
work page 2011
-
[7]
L. Condat, “A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms,” J. Optim. Theory Appl. , vol. 158, no. 2, pp. 460–479, 2013
work page 2013
-
[8]
A splitting algorithm for dual monotone inclusions involving cocoercive operators,
B. C. Vu, “A splitting algorithm for dual monotone inclusions involving cocoercive operators,” Adv. Comput. Math., vol. 38, no. 3, pp. 667–681, 2013
work page 2013
-
[9]
iPiasco: inertial proximal algorithm for strongly convex optimization,
P. Ochs, T. Brox, and T. Pock, “iPiasco: inertial proximal algorithm for strongly convex optimization,” J. Math. Imaging Vision , vol. 53, no. 2, pp. 171–181, 2015. 10
work page 2015
-
[10]
E. Chouzenoux, J.-C. Pesquet, and A. Repetti, “Variable metric forward- backward algorithm for minimizing the sum of a differentiable function and a convex function,” J. Optim. Theory Appl. , vol. 162, no. 1, pp. 107–132, 2014
work page 2014
-
[11]
A class of randomized primal-dual algorithms for distributed optimization,
J.-C. Pesquet and A. Repetti, “A class of randomized primal-dual algorithms for distributed optimization,” J. Nonlinear Convex Anal. , vol. 16, no. 12, pp. 2453–2490, 2015
work page 2015
-
[12]
H. H. Bauschke and P. L. Combettes, Convex analysis and monotone operator theory in Hilbert spaces , ser. CMS Books in Mathemat- ics/Ouvrages de Mathématiques de la SMC. Springer, New York, 2011, with a foreword by Hédy Attouch
work page 2011
-
[13]
Learning deep cnn denoiser prior for image restoration,
K. Zhang, W. Zuo, S. Gu, and L. Zhang, “Learning deep cnn denoiser prior for image restoration,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2017, pp. 2808–2817
work page 2017
-
[14]
Deep plug-and-play super-resolution for arbitrary blur kernels,
K. Zhang, W. Zuo, and L. Zhang, “Deep plug-and-play super-resolution for arbitrary blur kernels,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2019
work page 2019
-
[15]
Gradient step denoiser for convergent plug-and-play,
S. Hurault, A. Leclaire, and N. Papadakis, “Gradient step denoiser for convergent plug-and-play,” 2021, Preprint 2021, arXiv:2110.03220
-
[16]
Adversarial regularizers in inverse problems,
S. Lunz, O. Öktem, and C.-B. Schönlieb, “Adversarial regularizers in inverse problems,” in Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Bianchi, and R. Garnett, Eds., vol. 31. Curran Associates, Inc., 2018
work page 2018
-
[17]
Plug-and-play methods for magnetic resonance imaging: Using denoisers for image recovery,
R. Ahmad, C. Bouman, G. Buzzard, S. Chan, S. Liu, E. Reehorst, and P. Schniter, “Plug-and-play methods for magnetic resonance imaging: Using denoisers for image recovery,” IEEE Signal Process. Mag. , vol. 37, pp. 105–116, 01 2020
work page 2020
-
[18]
Plug-and- play methods provably converge with properly trained denoisers,
E. Ryu, J. Liu, S. Wang, X. Chen, Z. Wang, and W. Yin, “Plug-and- play methods provably converge with properly trained denoisers,” in Proceedings of the 36th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, K. Chaudhuri and R. Salakhutdinov, Eds., vol. 97. PMLR, 09–15 Jun 2019, pp. 5546– 5557
work page 2019
-
[19]
Building firmly nonexpansive convolutional neural networks,
M. Terris, A. Repetti, J.-C. Pesquet, and Y . Wiaux, “Building firmly nonexpansive convolutional neural networks,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2020), 2020, pp. 8658–8662
work page 2020
-
[20]
Enhanced convergent PNP algorithms for image restoration,
——, “Enhanced convergent PNP algorithms for image restoration,” in IEEE International Conference on Image Processing (ICIP 2021) , Aug. 2021
work page 2021
-
[21]
Convolutional proximal neural networks and plug-and-play algorithms,
J. Hertrich, S. Neumayer, and G. Steidl, “Convolutional proximal neural networks and plug-and-play algorithms,” Linear Algebra Appl., vol. 631, pp. 203–234, 2021
work page 2021
-
[22]
Learning maximally monotone operators for image recovery,
J.-C. Pesquet, A. Repetti, M. Terris, and Y . Wiaux, “Learning maximally monotone operators for image recovery,” SIAM J. Imaging Sci. , vol. 14, no. 3, pp. 1206–1237, 2021
work page 2021
-
[23]
Dual forward- backward unfolded network for flexible plug-and-play,
A. Repetti, M. Terris, Y . Wiaux, and J.-C. Pesquet, “Dual forward- backward unfolded network for flexible plug-and-play,” in 30th Euro- pean Signal Processing Conference 2022 , Oct. 2022, pp. 957–961
work page 2022
-
[24]
C. P. Robert, The Bayesian choice, 2nd ed., ser. Springer Texts in Statis- tics. Springer, New York, 2007, from decision-theoretic foundations to computational implementation
work page 2007
-
[25]
C. P. Robert and G. Casella, Monte Carlo statistical methods , 2nd ed., ser. Springer Texts in Statistics. Springer-Verlag, New York, 2004
work page 2004
-
[26]
Proximal Markov chain Monte Carlo algorithms,
M. Pereyra, “Proximal Markov chain Monte Carlo algorithms,” Stat. Comput., vol. 26, no. 4, pp. 745–760, 2016
work page 2016
-
[27]
Accelerating proximal markov chain Monte Carlo by using an explicit stabilized method,
M. Pereyra, L. V . Mieles, and K. C. Zygalakis, “Accelerating proximal markov chain Monte Carlo by using an explicit stabilized method,”SIAM J. Imaging Sci. , vol. 13, no. 2, pp. 905–935, 2020
work page 2020
-
[28]
V . De Bortoli, A. Durmus, M. Pereyra, and A. F. Vidal, “Efficient stochastic optimisation by unadjusted Langevin Monte Carlo. Applica- tion to maximum marginal likelihood and empirical Bayesian estima- tion,” Stat. Comput., vol. 31, no. 3, pp. Paper No. 29, 18, 2021
work page 2021
-
[29]
Asymptotically exact data augmentation: models, properties, and algorithms,
M. V ono, N. Dobigeon, and P. Chainais, “Asymptotically exact data augmentation: models, properties, and algorithms,” J. Comput. Graph. Statist., vol. 30, no. 2, pp. 335–348, 2021
work page 2021
-
[30]
A distributed Gibbs sam- pler with hypergraph structure for high-dimensional inverse problems,
P.-A. Thouvenin, A. Repetti, and P. Chainais, “A distributed Gibbs sam- pler with hypergraph structure for high-dimensional inverse problems,” Université de Lille and Heriot-Watt University, Tech. Rep., Oct. 2022, arXiv.2210.02341
-
[31]
Uncertainty quantification in imaging: When convex optimization meets Bayesian analysis,
A. Repetti, M. Pereyra, and Y . Wiaux, “Uncertainty quantification in imaging: When convex optimization meets Bayesian analysis,” in 26th European Signal Processing Conference (EUSIPCO 2018) , Dec. 2018, pp. 2668–2672
work page 2018
-
[32]
Maximum-a-posteriori estimation with Bayesian confi- dence regions,
M. Pereyra, “Maximum-a-posteriori estimation with Bayesian confi- dence regions,” SIAM J. Imaging Sci,, vol. 10, no. 1, pp. 285–302, 2017
work page 2017
-
[33]
Scalable Bayesian uncertainty quantification in imaging inverse problems via convex optimization,
A. Repetti, M. Pereyra, and Y . Wiaux, “Scalable Bayesian uncertainty quantification in imaging inverse problems via convex optimization,” SIAM J. Imaging Sci. , vol. 12, no. 1, pp. 87–118, 2019
work page 2019
-
[34]
Uncertainty quantification for radio interferometric imaging – I. Proximal MCMC methods,
X. Cai, M. Pereyra, and J. D. McEwen, “Uncertainty quantification for radio interferometric imaging – I. Proximal MCMC methods,” Mon. Not. R. Astron. Soc. , vol. 480, no. 3, pp. 4154–4169, 07 2018
work page 2018
-
[35]
Uncertainty quantification for radio interferometric imaging: II. MAP estimation,
——, “Uncertainty quantification for radio interferometric imaging: II. MAP estimation,” Mon. Not. R. Astron. Soc. , vol. 480, no. 3, pp. 4170– 4182, 08 2018
work page 2018
-
[36]
Uncertainty quantification in CT pulmonary angiography,
A. M. Rambojun, H. Komber, J. Rossdale, J. Suntharalingam, J. C. L. Rodrigues, M. J. Ehrhardt, and A. Repetti, “Uncertainty quantification in CT pulmonary angiography,” 2023, Preprint 2023, arXiv:2301.02467
-
[37]
N. Komodakis and J.-C. Pesquet, “Playing with duality: An overview of recent primal-dual approaches for solving large-scale optimization problems,” IEEE Signal Process. Mag. , vol. 32, pp. 31–54, 2014
work page 2014
-
[38]
Orthonormal bases of compactly supported wavelets,
I. Daubechies, “Orthonormal bases of compactly supported wavelets,” Comm. Pure Appl. Math. , vol. 41, no. 7, pp. 909–996, 1988
work page 1988
-
[39]
Biorthogonal bases of compactly supported wavelets,
A. Cohen, I. Daubechies, and J.-C. Feauveau, “Biorthogonal bases of compactly supported wavelets,” Comm. Pure Appl. Math., vol. 45, no. 5, pp. 485–560, 1992
work page 1992
-
[40]
Daubechies, Ten lectures on wavelets , ser
I. Daubechies, Ten lectures on wavelets , ser. CBMS-NSF Regional Conference Series in Applied Mathematics. SIAM, Philadelphia, PA, 1992, vol. 61
work page 1992
-
[41]
P. L. Combettes and J.-C. Pesquet, “Primal-dual splitting algorithm for solving inclusions with mixtures of composite, Lipschitzian, and parallel-sum type monotone operators,” Set-Valued Var. Anal., vol. 20, no. 2, pp. 307–330, 2012
work page 2012
-
[42]
Image inpainting for irregular holes using partial convolutions,
G. Liu, F. A. Reda, K. J. Shih, T.-C. Wang, A. Tao, and B. Catanzaro, “Image inpainting for irregular holes using partial convolutions,” in Computer Vision – ECCV 2018 . Springer International Publishing, 2018, pp. 89–105
work page 2018
-
[43]
Free-form image inpainting with gated convolution,
J. Yu, Z. Lin, J. Yang, X. Shen, X. Lu, and T. Huang, “Free-form image inpainting with gated convolution,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV) , 2019, pp. 4470–4479
work page 2019
-
[44]
Shift-net: Image inpainting via deep feature rearrangement,
Z. Yan, X. Li, M. Li, W. Zuo, and S. Shan, “Shift-net: Image inpainting via deep feature rearrangement,” in Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Pro- ceedings, Part XIV, ser. Lecture Notes in Computer Science, V . Ferrari, M. Hebert, C. Sminchisescu, and Y . Weiss, Eds., vol. 11218. Springer, 2018,...
work page 2018
-
[45]
Palette: Image-to-image diffusion models,
C. Saharia, W. Chan, H. Chang, C. A. Lee, J. Ho, T. Salimans et al. , “Palette: Image-to-image diffusion models,” 2022
work page 2022
-
[46]
Zero-shot image restoration using denoising diffusion null-space model,
J. Z. Yinhuai Wang, Jiwen Yu, “Zero-shot image restoration using denoising diffusion null-space model,” in International Conference on Learning Representations (ICLR) , 2023
work page 2023
-
[47]
H. Attouch, Jérôme, and B. F. Svaiter, “Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward- backward splitting, and regularized gauss-seidel methods,” Math. Pro- gram., vol. 137, no. 1-2, pp. 91–129, Aug. 2011
work page 2011
-
[48]
Recovery of nonuni- formdirac pulses from noisy linear measurements,
L. Condat, A. Hirabayashi, and Y . Hironaga, “Recovery of nonuni- formdirac pulses from noisy linear measurements,” in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing , 2013, pp. 6014–6018
work page 2013
-
[49]
Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising,
K. Zhang, W. Zuo, Y . Chen, D. Meng, and L. Zhang, “Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising,” Trans. Img. Proc., vol. 26, no. 7, p. 3142–3155, jul 2017
work page 2017
-
[50]
Very deep convolutional networks for large-scale image recognition,
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in International Conference on Learning Representations, 2015
work page 2015
-
[51]
U. Baid, S. Ghodasara, S. Mohan, M. Bilello, E. Calabrese, E. Colak et al. , “The RSNA-ASNR-MICCAI BraTS 2021 benchmark on brain tumor segmentation and radiogenomic classification,” 2021, Preprint 2021, arXiv:2107.02314
work page internal anchor Pith review Pith/arXiv arXiv 2021
-
[52]
S. Bakas, H. Akbari, A. Sotiras, M. Bilello, M. Rozycki, J. S. Kirby et al., “Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features,” Sci. Data , vol. 4, no. 1, Sep. 2017
work page 2017
-
[53]
The multimodal brain tumor image segmentation benchmark (BRATS),
B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer, K. Farahani, J. Kirby et al. , “The multimodal brain tumor image segmentation benchmark (BRATS),” IEEE Trans. Med. Imag. , vol. 34, no. 10, pp. 1993–2024, Oct. 2015
work page 1993
-
[54]
Uncertainty quantification in astro-imaging by optimisation,
A. Repetti, M. Pereyra, and Y . Wiaux, “Uncertainty quantification in astro-imaging by optimisation,” in Proceedings of the International BASP Frontiers Workshop 2019, Feb. 2019, international BASP Frontiers workshop 2019
work page 2019
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.