Comments on the article "A Bayesian conjugate gradient method"

T. J. Sullivan

arxiv: 1906.10240 · v1 · pith:O6YUAPK7new · submitted 2019-06-24 · 🧮 math.NA · cs.NA· math.PR· math.ST· stat.TH

Comments on the article "A Bayesian conjugate gradient method"

T. J. Sullivan This is my paper

Pith reviewed 2026-05-25 16:45 UTC · model grok-4.3

classification 🧮 math.NA cs.NAmath.PRmath.STstat.TH

keywords Bayesian methodsconjugate gradientlinear systemsuncertainty quantificationGaussian estimatesiterative methods

0 comments

The pith

A Bayesian conjugate gradient method produces a sequence of Gaussian estimates for the solution of linear equations.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The article under discussion proposes an approximately Bayesian iterative procedure based on the conjugate gradient method for solving systems of linear equations. This procedure generates a sequence of Gaussian estimates for the exact solution. The purpose of the probabilistic enrichment is that the covariance structure provides a posterior measure of uncertainty or confidence in the solution mean. The note offers comments on the proposal, poses questions, and suggests directions for further research.

Core claim

An approximately Bayesian iterative procedure based on the conjugate gradient method can be constructed to give a sequence of Gaussian estimates for the exact solution of a linear system, with the covariance structure intended to provide a posterior measure of uncertainty in the solution mean.

What carries the argument

Approximately Bayesian iterative procedure based on the conjugate gradient method that produces a sequence of Gaussian estimates.

If this is right

The covariance structure provides a posterior measure of uncertainty or confidence in the solution mean.
The procedure remains iterative and produces estimates at each step.
Further research directions can be explored based on the questions raised about the method.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The questions posed may concern the precise relationship between the Gaussian estimates and classical CG convergence properties.
Suggested research could test the method on ill-conditioned systems to check uncertainty calibration.

Load-bearing premise

That the conjugate gradient method can be made approximately Bayesian in an iterative procedure that gives a sequence of Gaussian estimates for the exact solution.

What would settle it

A concrete linear system where the sequence of Gaussian estimates fails to capture the true uncertainty around the exact solution.

read the original abstract

The recent article "A Bayesian conjugate gradient method" by Cockayne, Oates, Ipsen, and Girolami proposes an approximately Bayesian iterative procedure for the solution of a system of linear equations, based on the conjugate gradient method, that gives a sequence of Gaussian/normal estimates for the exact solution. The purpose of the probabilistic enrichment is that the covariance structure is intended to provide a posterior measure of uncertainty or confidence in the solution mean. This note gives some comments on the article, poses some questions, and suggests directions for further research.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

Sullivan's note flags some open questions in the Bayesian CG construction but stays at the level of commentary without new results or fixes.

read the letter

Sullivan's note on the Cockayne et al. Bayesian conjugate gradient paper mainly points out that the iterative Gaussian approximations are meant to deliver a posterior covariance for uncertainty in the solution mean, then asks how well that holds and suggests a few directions for tightening the setup. That is the core of what it does: it separates the algorithmic steps from the probabilistic claims and lists places where the approximation might need more justification, such as the choice of prior or the accumulation of errors across iterations. The note is clear on those distinctions and gives credit to the original intent of providing confidence measures beyond the point estimate. It does not, however, derive any corrected procedure, run any checks on the questions it raises, or compare against alternatives. As a result the piece remains a list of observations rather than a resolution or extension. The questions themselves look reasonable on the surface but rest on the reader's familiarity with the original paper; without that, some of the concerns read as generic rather than load-bearing. This is the kind of short note that specialists in probabilistic numerics might want to see in a comments section so the conversation around the method can continue, but it will not move the broader field. A reader already working on Bayesian linear solvers could pick up one or two useful caveats; anyone else will find little to cite or build on. The work shows honest engagement with the literature and is coherent on its own terms, so it is worth sending to referees if the journal accepts commentary pieces, even though the contribution is modest.

Referee Report

0 major / 0 minor

Summary. This manuscript is a comment on Cockayne et al.'s 'A Bayesian conjugate gradient method,' which proposes an approximately Bayesian iterative procedure for solving linear systems based on the conjugate gradient method. The procedure produces a sequence of Gaussian estimates for the exact solution, with the covariance structure intended to supply a posterior measure of uncertainty in the solution mean. The note offers comments on the construction, poses questions, and suggests directions for further research; it advances no new theorem, algorithm, or quantitative result of its own.

Significance. The comment identifies the iterative Gaussian approximation as the key modeling choice in the original work and frames open questions around it. If those questions prove fruitful, the note could help sharpen the foundations of probabilistic numerics for linear solvers. Its value is therefore indirect and depends on whether the points raised are taken up by subsequent research; the manuscript itself contains no load-bearing derivation or empirical claim whose validity can be assessed independently.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their positive assessment of the manuscript and for recommending acceptance.

Circularity Check

0 steps flagged

No significant circularity

full rationale

This is a short comment note whose purpose is to pose questions about the Cockayne et al. construction and suggest further research. It contains no derivation chain, no predictions, no fitted parameters, and no load-bearing claims that could reduce to inputs by construction. The reader's assessment that the manuscript advances no positive central claim is correct; therefore no circularity analysis applies.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The paper is a commentary and introduces no new free parameters, axioms, or invented entities.

pith-pipeline@v0.9.0 · 5612 in / 881 out tokens · 32455 ms · 2026-05-25T16:45:20.934455+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

10 extracted references · 10 canonical work pages · 1 internal anchor

[1]

V. Chen, M. M. Dunlop, O. Papaspiliopoulos, and A. M. Stuart. Dimension-robust MCMC in Bayesian inverse problems, 2018. arXiv:1803.03344

work page internal anchor Pith review Pith/arXiv arXiv 2018
[2]

Cockayne, C

J. Cockayne, C. J. Oates, T. J. Sullivan, and M. Girolami. Probabilistic numerical methods for PDE -constrained Bayesian inverse problems. In G. Verdoolaege, editor, Proceedings of the 36 th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering , volume 1853 of AIP Conference Proceedings, pages 060001--1--0600...

work page doi:10.1063/1.4985359 2017
[3]

S. L. Cotter, G. O. Roberts, A. M. Stuart, and D. White. MCMC methods for functions: modifying old algorithms to make them faster. Statist. Sci., 28 0 (3): 0 424--446, 2013. doi:10.1214/13-STS421

work page doi:10.1214/13-sts421 2013
[4]

Z. Fortuna. Superlinear convergence of conjugate gradient method in Hilbert space. In Theory of nonlinear operators (Proc. Fourth Internat. Summer School, Acad. Sci., Berlin, 1975), Abh. Akad. Wiss. DDR Abt. Math.-Natur.-Tech., 1977, 1, pages 313--318. Akademie-Verlag, Berlin, 1977

work page 1975
[5]

Z. Fortuna. Some convergence properties of the conjugate gradient method in Hilbert space. SIAM J. Numer. Anal., 16 0 (3): 0 380--384, 1979. doi:10.1137/0716031

work page doi:10.1137/0716031 1979
[6]

M\' a lek and Z

J. M\' a lek and Z. Strako s . Preconditioning and the conjugate gradient method in the context of solving PDE s , volume 1 of SIAM Spotlights. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2015. Chapter 5. doi:10.1137/1.9781611973846.ch5

work page doi:10.1137/1.9781611973846.ch5 2015
[7]

C. J. Oates and T. J. Sullivan. A modern retrospective on probabilistic numerics. Stat. Comput., 2019. To appear. arXiv:1901.04457

work page arXiv 2019
[8]

H. Owhadi. Bayesian numerical homogenization. Multiscale Model. Simul., 13 0 (3): 0 812--828, 2015. doi:10.1137/140974596

work page doi:10.1137/140974596 2015
[9]

H. Owhadi. Multigrid with rough coefficients and multiresolution operator decomposition from hierarchical information games. SIAM Rev., 59 0 (1): 0 99--149, 2017. doi:10.1137/15M1013894

work page doi:10.1137/15m1013894 2017
[10]

Schober, D

M. Schober, D. K. Duvenaud, and P. Hennig. Probabilistic ODE solvers with Runge -- Kutta means. In Advances in Neural Information Processing Systems 27, 2014. https://papers.nips.cc/paper/5451-probabilistic-ode-solvers-with-runge-kutta-means

work page 2014

[1] [1]

V. Chen, M. M. Dunlop, O. Papaspiliopoulos, and A. M. Stuart. Dimension-robust MCMC in Bayesian inverse problems, 2018. arXiv:1803.03344

work page internal anchor Pith review Pith/arXiv arXiv 2018

[2] [2]

Cockayne, C

J. Cockayne, C. J. Oates, T. J. Sullivan, and M. Girolami. Probabilistic numerical methods for PDE -constrained Bayesian inverse problems. In G. Verdoolaege, editor, Proceedings of the 36 th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering , volume 1853 of AIP Conference Proceedings, pages 060001--1--0600...

work page doi:10.1063/1.4985359 2017

[3] [3]

S. L. Cotter, G. O. Roberts, A. M. Stuart, and D. White. MCMC methods for functions: modifying old algorithms to make them faster. Statist. Sci., 28 0 (3): 0 424--446, 2013. doi:10.1214/13-STS421

work page doi:10.1214/13-sts421 2013

[4] [4]

Z. Fortuna. Superlinear convergence of conjugate gradient method in Hilbert space. In Theory of nonlinear operators (Proc. Fourth Internat. Summer School, Acad. Sci., Berlin, 1975), Abh. Akad. Wiss. DDR Abt. Math.-Natur.-Tech., 1977, 1, pages 313--318. Akademie-Verlag, Berlin, 1977

work page 1975

[5] [5]

Z. Fortuna. Some convergence properties of the conjugate gradient method in Hilbert space. SIAM J. Numer. Anal., 16 0 (3): 0 380--384, 1979. doi:10.1137/0716031

work page doi:10.1137/0716031 1979

[6] [6]

M\' a lek and Z

J. M\' a lek and Z. Strako s . Preconditioning and the conjugate gradient method in the context of solving PDE s , volume 1 of SIAM Spotlights. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2015. Chapter 5. doi:10.1137/1.9781611973846.ch5

work page doi:10.1137/1.9781611973846.ch5 2015

[7] [7]

C. J. Oates and T. J. Sullivan. A modern retrospective on probabilistic numerics. Stat. Comput., 2019. To appear. arXiv:1901.04457

work page arXiv 2019

[8] [8]

H. Owhadi. Bayesian numerical homogenization. Multiscale Model. Simul., 13 0 (3): 0 812--828, 2015. doi:10.1137/140974596

work page doi:10.1137/140974596 2015

[9] [9]

H. Owhadi. Multigrid with rough coefficients and multiresolution operator decomposition from hierarchical information games. SIAM Rev., 59 0 (1): 0 99--149, 2017. doi:10.1137/15M1013894

work page doi:10.1137/15m1013894 2017

[10] [10]

Schober, D

M. Schober, D. K. Duvenaud, and P. Hennig. Probabilistic ODE solvers with Runge -- Kutta means. In Advances in Neural Information Processing Systems 27, 2014. https://papers.nips.cc/paper/5451-probabilistic-ode-solvers-with-runge-kutta-means

work page 2014