Pith. sign in

REVIEW 3 major objections 5 minor 8 references

On the construction of a gradient method of quadratic optimization, optimal from the point of view of minimizing the distance to the exact solution

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper constructs the $m$-moment minimal error method for quadratic optimization in Hilbert space and proves that for $m=\infty$ no first-order Krylov-subspace method can achieve a smaller distance to the exact solution after the same…

desk verdict The finite-m sliding-window minimal error method is a real and potentially useful construction, but the headline optimality claim only holds in exact arithmetic and Theorem 4's rate proof is invalid as written. read the letter →

arxiv 2506.04866 v1 pith:TR2GHOIP submitted 2025-06-05 math.OC

N. V. Pletnev This is my paper
classification math.OC MSC 90C2065K0565N21
keywords m-momentminimalerrormethodquadraticoptimizationinHilbertspaceill-posedinverseproblemsKrylovsubspacemethodsdistance-to-solutionminimizationHelmholtzequationCauchyproblemretrospectiveheatthermoacoustics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper builds an $m$-moment minimal error method for minimizing a quadratic functional in Hilbert space when the target value is known and the gradient is computable through well-posed solves. At each step the method selects the point in the affine hull of the current point, the negative gradient, and the last $m$ steps that is closest to the exact solution. The paper proves that for $m=\infty$ this is globally optimal among all first-order methods that operate in Krylov subspaces: no such method can achieve a smaller distance to the solution after the same number of steps. It also proves convergence in infinite dimension for compact operators, superlinear functional decay, and shows that uniform convergence over all starting points is impossible. Numerical experiments on three ill-posed inverse problems — a Cauchy problem for the Helmholtz equation, a retrospective heat equation, and thermoacoustics — show the method is competitive and often superior to existing accelerated schemes.

What carries the argument

The load-bearing mechanism is the orthogonality of successive steps. Lemma 1 shows that the step vector $h_k=q_{k+1}-q_k$ produced by the $m$-moment method is orthogonal to each of the previous $m$ step vectors, $\langle h_k,h_{k-i}\rangle=0$ for $1\le i\le m$. This makes the linear system for the coefficients diagonal and gives the search direction $s_k$ as the projection of the negative gradient $-\nabla J(q_k)$ onto the orthogonal complement of the span of the last $m$ steps, with step length $\alpha_k = 2J(q_k)/\|s_k\|^2$. For $m=\infty$, the iterates satisfy $q_n\in q_0+\mathcal{K}_n$ and the method picks the point in that affine Krylov subspace closest to $q^*$; Theorem 1 then follows by induction on the dimension of the Krylov subspace.

What would settle it

Run the thermoacoustics experiment of Section 5.3 while recording the dot products $\langle h_k,h_{k-i}\rangle$ for $i=1,\dots,5$; if these become significantly nonzero well before the $m=\infty$ error stops decreasing, the exact-arithmetic premise is falsified. Alternatively, rerun the same experiment in high-precision arithmetic; if $m=\infty$ then outperforms $m=5$, the roundoff explanation is supported, while failure to outperform would contradict the optimality theorem in the discretized setting.

Watch

Extended reading notes

Core claim

The paper's central claim is that among all first-order methods whose iterates lie in Krylov subspaces — subspaces spanned by the initial gradient and its repeated images under the operator $A^*A_0$ — the $\infty$-moment minimal error method achieves the smallest possible distance to the exact solution after any fixed number of steps. The method chooses the next point as the metric projection of the true solution onto the affine subspace spanned by the current point, the negative gradient, and all previous steps; for finite $m$ it uses only the last $m$ steps. Because the exact solution is unknown, the key is that the inner products that define the projection are computable from the residual via $\langle q_k-q^*, \nabla J(q_k)\rangle = 2J(q_k)$ and the geometric orthogonality delivered by Lemma 1. The paper proves this optimality as Theorem 1, and also proves convergence in argument for compact operators (Theorem 3), superlinear functional convergence $J(q_k)=o(1/k)$ (Theorem 4), and the absence of a uniform convergence rate in infinite dimension (Theorem 6).

Load-bearing premise

The entire theory rests on the assumption that exact arithmetic keeps the computed steps exactly orthogonal to the previous $m$ steps; once floating-point roundoff violates this orthogonality, the $m=\infty$ method's theoretical superiority is no longer guaranteed, and the paper's own thermoacoustics experiment shows it can become the worst variant.

Editorial extensions

If this is right

  • For any quadratic problem where gradients are cheap relative to inner products, the $m=\infty$ method is the theoretical best-in-class; conjugate-gradient variants cannot beat its distance error after the same iteration count.
  • For compact operators, the method converges in argument even in infinite-dimensional Hilbert space, where finite-step finite-dimensional arguments fail.
  • The functional value decays superlinearly, $J(q_k)=o(1/k)$, and for strongly convex quadratics the distance decreases by a factor $1-(\mu/L)\sin^2\varphi_k$ per step.
  • No uniform convergence estimate over all starting points exists in infinite dimension; for any prescribed $\varepsilon$ and step count $N$ there are initial points of unit distance whose error after $N$ steps exceeds $\varepsilon$.
  • The method extends to composite functionals that are sums of quadratic terms with a common minimizer, which is exactly the structure of the thermoacoustics inverse problem.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The $m$ parameter is a practical regularization lever: small $m$ keeps reorthogonalization local, while $m=\infty$ maximizes theoretical optimality but accumulates roundoff; adding explicit reorthogonalization to the $m=\infty$ update might recover its theoretical edge in floating point.
  • The non-uniform convergence theorem suggests that no fixed-iteration budget can be guaranteed for ill-posed inverse problems; stopping rules based on residual stagnation may be more reliable than iteration counts.
  • Because the method only needs the gradient and the ability to solve the forward problem, it can be applied to large-scale inverse problems where Hessians are unavailable; a natural test is whether the same optimality holds for non-quadratic least-squares after linearization.
  • The proof of optimality depends on exact orthogonalization; in finite precision, periodic reorthogonalization (as in classical Lanczos implementations) is a testable remedy that the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper constructs an m-moment minimum-error method for minimizing a quadratic functional in Hilbert space. At each step the method chooses the point in an affine subspace spanned by the negative gradient and the m most recent steps that is closest to the unknown exact solution q*, using the fact that the relevant inner products with q* are computable from the residual. The paper proves that the steps are mutually orthogonal (Lemma 1), that for m=∞ the affine subspaces coincide with the Krylov subspaces and hence the method is optimal among Krylov-subspace methods (Theorem 1), that simple gradient descent and the ∞-moment method converge in norm under a compactness assumption (Theorems 2 and 3), that the functional error satisfies J(q_k)=o(1/k) (Theorem 4), and that no uniform convergence rate in the argument can exist (Theorem 6). Numerical experiments on a Helmholtz initial-boundary value problem, a retrospective heat equation problem, and a thermoacoustic inverse problem compare several m-moment variants with conjugate gradient, heavy-ball, Polyak step, and similar-triangle methods.

Significance. The main theoretical contribution, Theorem 1, is a clean and, in exact arithmetic, correct observation: the m=∞ minimum-error method is the orthogonal projection onto the affine Krylov subspace, so no Krylov-subspace method can achieve a smaller distance to the exact solution after the same number of steps. The formulas for the coefficients are explicit and computable, and the paper gives a self-contained convergence and non-uniform-convergence analysis in infinite-dimensional Hilbert space. The experimental section covers three genuinely ill-posed problems and is honest about the failure of the theoretically optimal m=∞ method in floating-point arithmetic. The paper would benefit from acknowledging that the m=∞ method is a classical minimum-error Krylov method, but the presentation is original in its optimization-oriented language and in the explicit treatment of the composite functional case.

major comments (3)
  1. [Section 4, Theorem 4] This comment is complete.
  2. [Section 5.3, Table 4] This comment is complete.
  3. [Section 4, Theorem 1, final paragraph] This comment is complete.
minor comments (5)
  1. [Section 2] This comment is complete.
  2. [Section 5, Tables 2 and 3] This comment is complete.
  3. [Section 5.3, equations (36) and (37)] This comment is complete.
  4. [Section 4, Theorem 4] This comment is complete.
  5. [General] This comment is complete.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the optimality claim follows from a proven Krylov-spanning argument, not from the definition alone.

full rationale

The derivation chain is self-contained. The m-moment method is explicitly constructed as the minimizer of ||q-q*|| over an affine subspace (eq. 15), and the central optimality theorem (Theorem 1) does not assume that this subspace is the full Krylov subspace; it proves the coincidence by induction: ∇J(q_n) ∈ K_{n+1}\K_n and h_n ∈ K_{n+1}\K_n are established, and then the search manifold is shown to equal q0+K_{n+1}. Since any Krylov method's iterate lies in q0+K_{n+1}, the optimality conclusion is a consequence of the proven set equality rather than a restatement of the construction. Theorems 2 and 3 provide independent convergence grounding through spectral analysis and comparison with gradient descent, and Theorem 6 is an independent lower-bound construction. The numerical experiments, including the explicitly reported failure of m=∞ under roundoff in the thermoacoustics test (Table 4), do not rely on circular reasoning. Self-citations [3] and [4] are used for problem-specific operator/gradient formulas and spectral asymptotics in the experimental sections; the optimality and convergence proofs stand on their own and do not import the paper's conclusions from prior author work. No step reduces by construction to its own inputs.

Assumptions & free parameters 2 free parameters · 8 assumptions · 0 invented entities

The method introduces no new physical entities or fitted physical constants. The load-bearing assumptions are: exact arithmetic for the orthogonality lemma, uniqueness of the solution, computability of the gradient through adjoint well-posed problems, compactness of A0 for the convergence theorems, and strong convexity for the linear-rate estimates. The memory parameter m and the discretization steps are hand-chosen inputs that affect the numerical demonstrations but are not fitted to data.

free parameters (2)
  • m (memory length) = 1, 2, 5, ∞ in the experiments
    The method searches over the last m step directions; the theory does not prescribe m, and the experiments select different values per problem (m=∞ for heat, m=5 for thermoacoustics).
  • grid steps h and tau = h=0.01 or 0.04, tau=0.002
    Discretization parameters for the numerical experiments; chosen by hand and not part of the theoretical claims, but they determine the reported errors.
assumptions (8)
  • standard math Spectral theorem and Hilbert-Schmidt decomposition for compact self-adjoint operators
    Invoked in the proof of Theorem 2 to decompose the error into eigencomponents.
  • standard math Cauchy-Schwarz inequality and nonvanishing of Vandermonde determinants
    Used in Lemma 1 and Theorem 6 to prove strict positivity of determinants and linear independence.
  • domain assumption The operator equation Aq=f has a unique solution q*
    Stated as a condition in Theorem 2 and used throughout; without uniqueness the optimization target is not well-defined.
  • domain assumption Aq - Aq' = A0(q - q') with a bounded linear operator A0, and the gradient ∇J(q) = A*(Aq - f) is computable via well-posed adjoint problems
    Central structural assumption of Section 1 that makes the minimal error step computable from the residual.
  • domain assumption A0 is compact for the ill-posed examples
    Required by Theorem 2 and Theorem 6 for convergence and for the existence of the eigenbasis.
  • domain assumption Exact arithmetic (no roundoff) in the proofs of orthogonality and optimality
    Lemma 1 and Theorem 1 assume exact orthogonality of the step vectors; numerical results show degradation for m=∞.
  • domain assumption Strong convexity with parameter mu for the rate estimates in Theorem 5
    The factor (1 - mu/L) requires a positive lower bound on the spectrum of the Hessian.
  • domain assumption For Theorem 6, the eigenvalues lambda_n of B are positive, tend to zero, and are simple (or the initial vector is restricted to one eigencomponent per eigenspace)
    The proof explicitly simplifies the degenerate case in this way to construct the worst-case initial approximation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On the construction of a gradient method of quadratic optimization, optimal from the point of view of minimizing the distance to the exact solution." pith.science (2026). https://pith.science/paper/TR2GHOIP

@misc{pith2026250604866,
  author       = {Pith},
  title        = {Pith review of: On the construction of a gradient method of quadratic optimization, optimal from the point of view of minimizing the distance to the exact solution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TR2GHOIP}},
  note         = {Machine review of arXiv:2506.04866}
}
abstract

Problems of quadratic optimization in Hilbert space often arise when solving ill-posed problems for differential equations. In this case, the target value of the functional is known. In addition, the structure of the functional allows calculating the gradient by solving well-posed problems, which allows applying first-order methods. This article is devoted to the construction of the $m$-moment minimum error method -- an effective method that minimizes the distance to the exact solution. The convergence and optimality of the constructed method are proved, as well as the impossibility of uniform convergence of methods operating in Krylov subspaces. Numerical experiments are carried out demonstrating the efficiency of applying the $m$-moment minimum error method to solving various ill-posed problems: the initial-boundary value problem for the Helmholtz equation, the retrospective Cauchy problem for the heat equation, and the inverse problem of thermoacoustics.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

8 extracted references · 8 canonical work pages

  1. [1]

    Оптимизационный метод решения обратной задачи термоакустики // Сибирские электронные математические известия, 2011, Т

    Кабанихин С.\,И., Шишленин М.\,А., Криворотько О.\,И. Оптимизационный метод решения обратной задачи термоакустики // Сибирские электронные математические известия, 2011, Т. 8, с. 263-292

    work page 2011

  2. [2]

    Элементы теории функций и функционального анализа // М.: ФИЗМАТЛИТ, 2004

    Колмогоров А.\,Н., Фомин С.\,В. Элементы теории функций и функционального анализа // М.: ФИЗМАТЛИТ, 2004. ISBN 5-9221-0266-4

    work page 2004

  3. [3]

    Применение градиентных методов оптимизации для решения задачи Коши для уравнения Гельмгольца // Компьютерные исследования и моделирование, 2022, т

    Плетнев Н.\,В., Двуреченский П.\,Е., Гасников А.\,В. Применение градиентных методов оптимизации для решения задачи Коши для уравнения Гельмгольца // Компьютерные исследования и моделирование, 2022, т. 14, № 2, с. 417-444

    work page 2022

  4. [4]

    О модификации метода покомпонентного спуска для решения некоторых обратных задач математической физики // Компьютерные исследования и моделирование, 2023, т

    Плетнев Н.\,В., Матюхин В.\,В. О модификации метода покомпонентного спуска для решения некоторых обратных задач математической физики // Компьютерные исследования и моделирование, 2023, т. 15, № 2, с. 301-316

    work page 2023

  5. [5]

    Минимизация негладких функционалов // Журнал вычислительной математики и математической физики, 1969, т

    Поляк Б.\,Т. Минимизация негладких функционалов // Журнал вычислительной математики и математической физики, 1969, т. 9, № 3, с. 509-521

    work page 1969

  6. [6]

    Polyak Minorant Method for Convex Optimization

    Devanathan N., Boyd S. Polyak Minorant Method for Convex Optimization // e-print, 2024. --- URL: https://arxiv.org/abs/2310.07922

    work page Pith review arXiv 2024

  7. [7]

    Quadratic minimization: from conjugate gradient to an adaptive Heavy-ball method with Polyak step-sizes

    Goujaud B., Taylor A., Dieuleveut A. Quadratic minimization: from conjugate gradient to an adaptive Heavy-ball method with Polyak step-sizes // e-print, 2022. --- URL: \ ://arxiv.org/abs/2210.06367

    work page Pith review arXiv 2022

  8. [8]

    Inverse and ill-posed problems: theory and applications // Berlin: Walter de Gruyter GmbH & Co, 2012

    Kabanikhin S.\,I. Inverse and ill-posed problems: theory and applications // Berlin: Walter de Gruyter GmbH & Co, 2012. -- 459 p. ISBN 978-3-11-022400-9

    work page 2012

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.