Iterative methods for linear systems of equations: A brief historical journey

Yousef Saad

Iterative methods for linear systems of equations: A brief historical journey

T0 review · 0 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Iterative methods for linear systems have been redirected again and again by application demands and computing hardware, from 1823 relaxation to today's machine-learning-scale problems.

desk verdict A solid, readable historical survey from a leading expert; the math is right and the narrative holds, though the 'first' claims need hedging and Eq. (3.1) has a typo. read the letter →

arxiv 1908.01083 v1 pith:HVLZHLYM submitted 2019-08-02 math.HO

Yousef Saad This is my paper · ORCID

classification math.HO MSC 65F1001A5501A60

keywords iterativemethodshistoryofnumericalanalysisGauss-Seidelmethodsuccessiveover-relaxationKrylovsubspaceconjugategradientpreconditioningasynchronousiterations

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that iterative methods for linear systems have never developed in a straight line: every major turn—from Gauss's indirect elimination in 1823, through relaxation and SOR, to Krylov subspace methods, sparse direct solvers, and asynchronous iterations—was a response to new application demands and to the computing hardware of the moment. The survey is deliberately selective and reads the record as a series of ways of thinking rather than a list of algorithms. A sympathetic reader should care because the pattern gives a basis for predicting the next turn: the paper locates that turn in data mining and machine learning, where randomness and stochasticity are central. The historical claims are grounded in primary sources such as an 1823 letter and early twentieth-century papers, together with the standard monographs that codified each era.

What carries the argument

The recurring mathematical object carrying the story is the residual polynomial. In Richardson's scheme the residual after $k$ steps is $r_k = p_k(A) r_0$ with $p_k(0) = 1$, so making the iteration fast is the problem of choosing a polynomial that is small on the spectrum of $A$. That single identity connects the fixed-point splittings behind relaxation ($A = D - E - F$, giving Jacobi and Gauss-Seidel), Chebyshev acceleration, the three-term recurrence of conjugate gradients, and preconditioning as a way of reshaping the spectrum. It explains both why SOR needed an optimal over-relaxation parameter and why CG removed the need for eigenvalue estimates.

What would settle it

A search of pre-1823 geodesy, astronomy, or actuarial manuscripts for a residual-updating indirect solution scheme—or archival evidence that Richardson knew the older relaxation work—would directly test the two signature historical claims.

Watch

Extended reading notes

Core claim

The paper's central claim is that the history of iterative methods is driven from the outside in: applications and machines set the agenda, and the mathematics follows. It argues that Gauss's 1823 indirect elimination—updating one coordinate at a time by zeroing the largest residual component—is the first known iterative method, that Jacobi's 1845 relaxation paper introduced what is effectively the first preconditioner through rotations, and that each later family (Liebmann/Gauss-Seidel, SOR, Richardson polynomial iteration, conjugate gradients, nonsymmetric Krylov accelerators) answered a specific practical pressure. The conjugate gradient method, in this telling, was the single most important advance of the 1950s, but it became dominant only after incomplete factorization preconditioning made it reliable in the 1970s. The survey ends by arguing that the next pressure is already visible in machine-learning-scale linear algebra, where the global-optimality assumptions of CG and GMRES fit poorly with randomness.

Load-bearing premise

The origin and priority claims depend on the historical record being complete: if an earlier iterative method or an undocumented line of influence exists, the story would shift.

Editorial extensions

If this is right

Because cyclic relaxation is easy to mechanize, Gauss-Seidel-type iterations became the natural first iterative solvers on digital computers, vindicating the mechanization that Southwell dismissed.
SOR with a well-chosen over-relaxation parameter could handle systems of order 20,000 in 1960 and 108,000 in 3-D versions—sizes direct methods could not touch at the time.
Conjugate gradients became the default for large symmetric positive definite systems only after incomplete Cholesky preconditioning (ICCG) appeared in 1977, not when the method was invented in the early 1950s.
For nonsymmetric problems, application demand produced a sequence of transpose-free accelerators (CGS, BiCGSTAB, QMR), and research has since shifted from accelerators to preconditioners.
Sparse direct methods remain competitive for 2-D problems but face intrinsic $O(N^{4/3})$ fill and $O(N^2)$ time in 3-D, which is why iterative methods stay essential for large 3-D discretizations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the paper's application-driven pattern holds, machine-learning workloads will not just add new problems but will push for solver reformulations built around randomness and low-rank structure, as the paper hints at the end.
The same pattern suggests that learned selection of orderings, preconditioners, or solver families—rather than a single new algorithm—is the likely near-term next direction, extending the historical role of heuristics in sparse direct methods.
The 1970s dismissal of asynchronous iterations as utopian, followed by their return on massively parallel machines, implies that communication cost, not flops, may be the binding constraint that determines the next solver family.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Take: this is a fine, readable survey by someone who knows the territory. It will be useful for people entering numerical linear algebra and for anyone teaching the history of iterative methods. What is genuinely new is modest: the author's personal recollections of the French asynchronous iteration school and the explicit 2-D vs 3-D complexity comparison for sparse direct solvers. The rest is a coherent synthesis of standard accounts, and it is accurate in its mathematics.

The paper does several things well. Gauss's 1823 letter to Gerling is presented concretely with the actual system and iteration table, which is a nice touch. The discussion of Southwell, Young, and the SOR era is well grounded. The treatment of Richardson's contribution correctly notes that he missed Chebyshev acceleration despite identifying the right problem. And the author is explicit at the start that the survey is not exhaustive; that disclaimer is honest and should be taken at face value.

Soft spots are minor. The most concrete defect is Eq. (3.1): with the residual convention r = Ax + b, the update should be x_{j+1} = x_j - (1/alpha_j) r_j, not A r_j. That is a typo, and it does not affect the historical argument. The priority claims – Gauss as 'first known' iterative method, Jacobi rotations as 'first known' preconditioning, Richardson unaware of Gauss and Seidel – are asserted with more confidence than a careful historian might want. They are traceable to the cited secondary literature, but a survey of this kind should either say 'appears to be' or footnote the uncertainty more consistently. The author does use 'appears' for Gauss, but the Jacobi claim is stated flatly.

The reader's cautious 'UNVERDICTED' verdict is reasonable in the sense that there is no new falsifiable claim here, but it is the wrong frame for a survey. The paper is not trying to prove a theorem. On its own terms, the central historical thesis – that the field repeatedly shifted direction in response to application and hardware pressure – holds up and is well supported. The citation pattern is fine; self-citation to Saad & van der Vorst (2000) is appropriate for a compressed version of the 20th century.

Who is this for? Colleagues who want a short historical orientation before teaching, students looking for context, and researchers who want a compact reminder of how the pieces fit. It is not a research contribution, but it deserves to be refereed and published as a survey. I would bring it to reading group and would cite it for the 2-D vs 3-D complexity table and the historical timeline.

Recommendation: send to peer review as a survey; the typo should be fixed and the 'first' claims lightly hedged, but no deeper problem exists.

Referee Report

0 major / 5 minor

Summary. This manuscript is a historical survey of iterative methods for solving linear systems, beginning with Gauss's 1823 letter to Gerling and moving through relaxation methods, SOR, Krylov subspace methods, sparse direct solvers, asynchronous iterations, and contemporary machine-learning-related developments. The central thesis is that iterative methods have repeatedly changed direction in response to application demands and hardware constraints, and the paper argues that the next frontier lies in machine-learning-driven linear algebra. The paper is explicitly not exhaustive and aims to 'underline the way of thinking at a specific time' rather than to provide a complete bibliography.

Significance. If accepted as an interpretation, the survey provides a valuable synthesis by a leading researcher in numerical linear algebra, with useful primary-source documentation including Forsythe's translation of Gauss's letter, Southwell and Young anecdotes, and Varga's quantitative examples. Its strengths include a broad and well-cited reference list, careful hedging of priority claims with phrases such as 'first known' and 'seems to be,' and explicit admission of the survey's limits. The paper presents no new mathematics, so its contribution is historiographical rather than technical; for a journal or venue in the history of mathematics this is appropriate. The historical narrative is coherent and does not contain internal inconsistencies that undermine the central claim.

minor comments (5)

[3, Eq. (3.1)] The displayed Richardson update is inconsistent with the residual convention r = Ax + b stated in the same section: substituting x_{j+1} = x_j - (1/alpha_j) A r_j into r_{j+1} = A x_{j+1} + b gives r_{j+1} = (I - A^2/alpha_j) r_j, not the factor (I - A/alpha_j) required by Eq. (3.2). The intended update is presumably x_{j+1} = x_j - (1/alpha_j) r_j. This is a transcription error rather than a flaw in the historical argument, but it should be corrected.
[2, Eq. (2.4)] The sentence 'write Ax = b as (D - E)x = F + b' appears to have a typo: it should be '(D - E)x = F x + b'. As printed, the right-hand side has incompatible dimensions.
[1] The priority claims that Gauss's 1823 letter is 'the first known reference' to an iterative method and that Jacobi rotations are 'the first known form of preconditioning' are appropriately hedged, but they rest on secondary literature rather than an independent archival audit. A sentence making this attribution explicit would strengthen the scholarly apparatus.
[Global] The manuscript would benefit from a careful proofreading pass. Examples of typos and minor errors include 'exaustive' in the abstract, 'there there were' in Section 1, 'phsyics' in Section 4, 'targetted' in Section 5, 'Bulleting' in Section 6, 'Hesteness' in Section 10, and 'disapperaring' and 'it they are' in Section 12.
[8, Table 8.1] The column header of Table 8.1 ('Off-diagonal Ordering Factor Nonz Time') is garbled; the table should be reformatted so that the columns for nonzeros and time are clearly labeled.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the historical narrative rests on cited sources and standard mathematics; the few self-citations are background references and are not load-bearing.

full rationale

This paper is a historical survey, not a derivation, so the equation-level circularity patterns do not apply. The central thesis—that iterative methods repeatedly changed direction in response to application demands and hardware developments (Sec. 12)—is supported by dated examples and citations to primary and secondary sources such as Forsythe [19, 20], Young [79], Varga [75], and George and Liu [29], rather than by fitting equations to target conclusions. The only self-citations are Saad's own textbook [59], used for the standard Chebyshev residual-polynomial result, and the earlier joint survey [60], used for background on twentieth-century accelerators; neither is invoked to justify a contested claim or to preclude alternatives. The Richardson residual-polynomial equations in Sec. 3 follow from the stated iteration by standard manipulation; the typo in Eq. (3.1), where an extra factor A appears in the update relative to the residual relation in Eq. (3.2), is an algebraic slip and not a circular step. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the author's prior work, and no known result is repackaged as a novel organizing principle. The survey is self-contained as an interpretive narrative, and the appropriate circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters or invented entities appear because the paper is a survey, not a model. The load-bearing assumptions are historical trust and standard mathematical background.

assumptions (3)

domain assumption The cited historical sources (Forsythe 1951, Bodewig 1959, Varga 1962, Young 1971, Golub and O'Leary 1989) accurately reflect the primary record.
The survey constructs its narrative from these secondary accounts and does not re-examine the original documents, so the reliability of the narrative is inherited from them (Sections 1-3).
standard math Standard convergence theory for Jacobi, Gauss-Seidel, and conjugate gradient methods is correct.
The fixed-point forms and residual polynomial arguments in Sections 2, 3, and 10 are standard textbook results that the paper relies on for its historical explanations.
domain assumption The author's personal recollections of the French school of chaotic iterations and of recent computer science curricula are representative.
Sections 7 and 12 blend autobiographical recollection with scholarly history; the reader must trust the author's first-hand account for these parts of the narrative.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Iterative methods for linear systems of equations: A brief historical journey." pith.science (2026). https://pith.science/paper/HVLZHLYM

@misc{pith2026190801083,
  author       = {Pith},
  title        = {Pith review of: Iterative methods for linear systems of equations: A brief historical journey},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HVLZHLYM}},
  note         = {Machine review of arXiv:1908.01083}
}

read the original abstract

This paper presents a brief historical survey of iterative methods for solving linear systems of equations. The journey begins with Gauss who developed the first known method that can be termed iterative. The early 20th century saw good progress of these methods which were initially used to solve least-squares systems, and then linear systems arising from the discretization of partial different equations. Then iterative methods received a big impetus in the 1950s - partly because of the development of computers. The survey does not attempt to be exhaustive. Rather, the aim is to underline the way of thinking at a specific time and to highlight the major ideas that steered the field.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Spacetime Wavelet Method for Linear Boundary-Value Problems in Sylvester Matrix Equation Form
math.NA 2025-09 conditional novelty 5.0 of 10

A spacetime wavelet method for linear PDEs produces a Sylvester matrix equation, solved efficiently with Global GMRES and a wavelet-based recursive initial guess.
Lower bounds for trace estimation via Block Krylov and other methods
math.ST 2025-06 conditional novelty 4.0 of 10

Any matrix-vector query algorithm that estimates tr(W^{-p}) for a d×d Wishart matrix within a constant factor needs Ω(d) queries when p>1/2.

Reference graph

Works this paper leans on

80 extracted references · 80 canonical work pages · cited by 2 Pith papers

[1]

D. N. D. G. Allen and R. V. Southwell , Relaxation methods applied to engineering problems xiv. plastic straining in two-dimensional stress-systems , Philosophical Transactions of the Royal Society of London. Series A, Mathematical and Physica l Sciences, 242 (1950)

work page 1950
[2]

H. Anzt, S. Tomov, J. Dongarra, and V. Heuveline , A block-asynchronous relaxation method for graphics processing units , Journal of Parallel and Distributed Computing, 73 13 (2013), pp. 1613 – 1626. Heterogeneity in Parallel and Distr ibuted Computing

work page 2013
[3]

Axelsson , A survey of preconditionied iterative methods for linear sy stems of algebraic equations, BIT, 25 (1985), pp

O. Axelsson , A survey of preconditionied iterative methods for linear sy stems of algebraic equations, BIT, 25 (1985), pp. 166–187

work page 1985
[4]

Bahi and J.-C

M. Bahi and J.-C. Miellou , Contractive mappings with maximum norms: comparison of constants of contraction and application to asynchronous i terations, Parallel Comput., 19 (1993), pp. 511–523

work page 1993
[5]

G. M. Baudet , Asynchronous iterative methods for multiprocessors , J. Assoc. Comput. Mach., 25 (1978), pp. 226–244

work page 1978
[6]

Benzi , Gianfranco Cimmino’s contributions to numerical mathemat ics, Tech

M. Benzi , Gianfranco Cimmino’s contributions to numerical mathemat ics, Tech. Rep. Tec- noprint, 2005, Atti del Seminario di Analisi Matematica, Di partimento di Matematica dell’Universita‘ di Bologna, Bologna, 2005

work page 2005
[7]

D. P. Bertsekas and D. El Baz , Distributed asynchronous relaxation methods for convex network ﬂow problems , SIAM J. Control Optim., 25 (1987), pp. 74–85

work page 1987
[8]

Bodewig , Matrix Calculus, 2nd Edition , North-Holland, Amsterdam, 1959

E. Bodewig , Matrix Calculus, 2nd Edition , North-Holland, Amsterdam, 1959

work page 1959

Show all 80 references

[9]

T. F. Chan, E. Gallopoulos, V. Simoncini, T. Szeto, and C. H. Ton g, A quasi-minimal residual variant of the Bi-CGSTAB algorithm for nonsymmetr ic systems , SIAM J. Sci. Comput., 15 (1994), pp. 338–347

1994
[10]

D. J. Chow E., Anzt H. , Asynchronous iterative algorithm for computing incomplet e factor- izations on gpus , in High Performance Computing. ISC High Performance, L. T. Kunkel J., ed., Lecture Notes in Computer Science, vol 9137, Springer, 2015

2015
[11]

E. Chu, A. George, J. W.-H. Liu, and E. G.-Y. Ng , Users guide for SPARSPAK–A: Water- loo sparse linear equations package, Tech. Rep. CS-84-36, University of W aterloo, W aterloo, IA, 1984

1984
[12]

Cimmino , Calcolo approssimato per le soluzioni dei sistemi di equazi oni lineari , Ricerca Scientiﬁca, II, 9 (1938), pp

G. Cimmino , Calcolo approssimato per le soluzioni dei sistemi di equazi oni lineari , Ricerca Scientiﬁca, II, 9 (1938), pp. 326–333

1938
[13]

D. N. De G. Allen and R. V. Southwell , Relaxation methods applied to determine the motion, in two dimensions, of a viscous ﬂuid past a ﬁxed cylin der, The Quarterly Journal of Mechanics and Applied Mathematics, 8 (1955), pp. 129–145

1955
[14]

I. S. Duff , A survey of sparse matrix research , in Proceedings of the IEEE, 65, New York, 1977, Prentice Hall, pp. 500–535

1977
[15]

El Baz , M -functions and parallel asynchronous algorithms , SIAM J

D. El Baz , M -functions and parallel asynchronous algorithms , SIAM J. Numer. Anal., 27 (1990), pp. 136–140

1990
[16]

M. N. El Tarazi , Some convergence results for asynchronous algorithms , Numer. Math., 39 (1982), pp. 325–340

1982
[17]

Emad, S.-A

N. Emad, S.-A. Shahzadeh-F azeli, and J. Dongarra , An asynchronous algorithm on the netsolve global computing system , Future Generation Computer Systems, 22 (2006), pp. 279 – 290

2006
[18]

Engeli, T

M. Engeli, T. Ginsburg, H. Rutishauser, and E. Stiefel , Reﬁned Iterative Methods for Computation of the Solution and the Eigenvalues of Self-Adj oint Boundary Value Prob- lems, Birkh¨ auser, Basel/Stuttgart, 1959

1959
[19]

G. E. Forsythe , Gauss to gerling on relaxation , Math. Tables and Other Aids to Comp., 5 (1951), pp. 255–258

1951
[20]

G. E. Forsythe , Solving linear algebraic equations can be interesting , Bull. Amer. Math. Soc., 59 (1953), pp. 299–329

1953
[21]

Fox , A SHORT ACCOUNT OF RELAXATION METHODS , The Quarterly Journal of Mechanics and Applied Mathematics, 1 (1948), pp

L. Fox , A SHORT ACCOUNT OF RELAXATION METHODS , The Quarterly Journal of Mechanics and Applied Mathematics, 1 (1948), pp. 253–280

1948
[22]

Fox and R

L. Fox and R. V. Southwell , Relaxation methods applied to engineering problems, vii a: Biharmonic analysis as applied to the ﬂexure and extension o f ﬂat elastic plates , Philo- sophical Transactions of the Royal Society of London. Serie s A, Mathematical and Physical Science...

1945
[23]

Frankel , Convergence rates of iterative treatments of partial diﬀer ential equations, MTAC, (1950), pp

S. Frankel , Convergence rates of iterative treatments of partial diﬀer ential equations, MTAC, (1950), pp. 65–75

1950
[24]

R. W. Freund , A Transpose-Free Quasi-Minimal Residual algorithm for non -Hermitian linear systems, SIAM J. Sci. Comput., 14 (1993), pp. 470–482

1993
[25]

R. W. Freund and N. M. Nachtigal , QMR: a quasi-minimal residual method for non- Hermitian linear systems , Numer. Math., 60 (1991), pp. 315–339

1991
[26]

Frommer and D

A. Frommer and D. B. Szyld , On asynchronous iterations , Journal of Computational and Applied Mathematics, 123 (2000), pp. 201 – 216. Numerical An alysis 2000. Vol. III: Linear Algebra

2000
[27]

Frommer and D

A. Frommer and D. B. Szyld , On asynchronous iterations , J. Comput. Appl. Math., 123 (2000), pp. 201–216. Numerical analysis 2000, Vol. III. Lin ear algebra

2000
[28]

George and J

A. George and J. W. Liu , The evolution of the minimum degree ordering algorithm , SIAM 14 Review, 31 (1989), pp. 1–19

1989
[29]

J. A. George and J. W. Liu , Computer Solution of Large Sparse Positive Deﬁnite Systems , Prentice-Hall, Englewood Cliﬀs, NJ, 1981

1981
[30]

Giraud and P

L. Giraud and P. Spiteri , R´ esolution parall` ele de probl` emes aux limites non lin´ eaires, RAIRO Mod´ el. Math. Anal. Num´ er., 25 (1991), pp. 579–606

1991
[31]

G. H. Golub and D. P. O’Leary , Some history of the conjugate gradient and Lanczos algo- rithms: 1948-1976 , SIAM review, 31 (1989), pp. 50–102

1989
[32]

G. H. Golub and R. S. V arga , Chebyshev semi iterative methods successive overrelaxati on iterative methods and second order Richardson iterative me thods, Numer. Math., 3 (1961), pp. 147–168

1961
[33]

J. R. Green and R. V. Southwell , Relaxation methods applied to engineering problems ix. high-speed ﬂow of compressible ﬂuid through a two-dimen sional nozzle , Philosophical Transactions of the Royal Society of London. Series A, Mathe matical and Physical Sciences, 239 (1944)...

1944
[34]

A. S. Householder , Theory of Matrices in Numerical Analysis , Blaisdell Pub. Co., Johnson, CO, 1964

1964
[35]

C. G. J. Jacobi , Ueber eine neue Auﬂ¨ osungsart der bei der Methode der kleins ten Quadrate vorkommende linearen Gleichungen , Astronomische Nachrichten, (1845), pp. 297–306
[36]

, Ueber ein leichtes Verfahren, die in der Theorie der S¨ acula rst¨ orungen vorkommenden Gleichungen numerisch aufzul¨ osen, Journal f¨ ur die reine und angewandte Mathematik, (1846), pp. 51–94
[37]

Kaczmarz, Angen¨ aherte auﬂ¨ osung von systemen linearer gleichungen, Bulletin international de l’Acad´ emie polonaise des Sciences et Lettres, III, clas s A (1937), pp

S. Kaczmarz, Angen¨ aherte auﬂ¨ osung von systemen linearer gleichungen, Bulletin international de l’Acad´ emie polonaise des Sciences et Lettres, III, clas s A (1937), pp. 355–357

1937
[38]

D. E. Knuth , George forsythe and the development of computer science , Commun. ACM, 15 (1972), pp. 721–726

1972
[39]

Lanczos , Solution of systems of linear equations by minimized iterat ions, J

C. Lanczos , Solution of systems of linear equations by minimized iterat ions, J. of Res. of the Nat. Bur. of Stand., 49 (1952), pp. 33–53

1952
[40]

Liebmann , Die angen¨ aherte harmonischer Funktionen und konformer Ab bildungen (nach Ideen von Boltzmann und Jacobi) , S

H. Liebmann , Die angen¨ aherte harmonischer Funktionen und konformer Ab bildungen (nach Ideen von Boltzmann und Jacobi) , S. B. Math. Nat. Kl. Bayerischen Akad. Wiss. M¨ unchen, (1918), pp. 385–416

1918
[41]

Magoul `es, D

F. Magoul `es, D. B. Szyld, and C. Venet , Asynchronous optimized schwarz methods with and without overlap , Numerische Mathematik, 137 (2017), pp. 199–227

2017
[42]

Mehmke and P

R. Mehmke and P. Nekrasov , Solutions of linear systems of equations by means of success ive approximations, Mat. Sbornik, 16 (1892), pp. 437–459. German translation a vailable in Math-Net.Ru
[43]

J. A. Meijerink and H. A. van der Vorst , An iterative solution method for linear systems of which the coeﬃcient matrix is a symmetric M-matrix , Math. of Comput., 31 (1977), pp. 148–162

1977
[44]

Miellou , It´ erations chaotiques ` a retards, C

J.-C. Miellou , It´ erations chaotiques ` a retards, C. R. Acad. Sci. Paris S´ er. A, 278 (1974), pp. 957–960

1974
[45]

J. C. Miellou , Algorithmes de relaxation chaotiques a retard , RAIRO, R-1 (1975), pp. 55–82

1975
[46]

Miellou , It´ erations chaotiques ` a retards; ´ etudes de la convergence dans le cas d’espaces partiellement ordonn´ es, C

J.-C. Miellou , It´ erations chaotiques ` a retards; ´ etudes de la convergence dans le cas d’espaces partiellement ordonn´ es, C. R. Acad. Sci. Paris S´ er. A-B, 280 (1975), pp. A233–A236

1975
[47]

J. C. Miellou, D. E. Baz, and P. Spiteri , A new class of asynchronous iterative algorithms with order intervals , Math. Comp., 67 (1998), pp. 237–255

1998
[48]

P. A. Nekrasov , Determining unknowns using the least squares method when th e number of unknowns is large , Mat. Sb., 12 (1884), pp. 189–204. (in Russian)
[49]

N ´ekrassov, D´ etermination des inconnues par la m´ ethode de moindres ca rr´ es dans le cas o` u le nombre d’inconnues est consid´ erable, Mat

P. N ´ekrassov, D´ etermination des inconnues par la m´ ethode de moindres ca rr´ es dans le cas o` u le nombre d’inconnues est consid´ erable, Mat. Sb., 12 (1885), pp. 189–204. In Russian. Available from http://mi.mathnet.ru/msb7148
[50]

Ostrowski, Determinanten mit ¨ uberwiegender haupt diagonale und die absolute konvergenz von linearen iterationsprozessen , Comm

A. Ostrowski, Determinanten mit ¨ uberwiegender haupt diagonale und die absolute konvergenz von linearen iterationsprozessen , Comm. Math. Helv., 30 (1955), pp. 175–210

1955
[51]

Parter, The use of linear graphs in gauss elimination , SIAM Review, 3 (1961)

S. Parter, The use of linear graphs in gauss elimination , SIAM Review, 3 (1961)

1961
[52]

S. S. Petrova and A. D. Solov’ev , The origin of the method of steepest descent , Historia Mathematica, 24 (1997), pp. 361 – 375

1997
[53]

J. K. Reid , On the method of conjugate gradients for the solution of larg e sparse systems of linear equations, in Large Sparse Sets of Linear Equations, J. K. Reid, ed., Ac ademic Press, 1971, pp. 231–254

1971
[54]

L. F. Richardson , The approximate arithmetical solution by ﬁnite diﬀerences of physical problems involving diﬀerential equations with an applicat ion to the stresses to a masonry dam, Philos. Trans. Roy. Soc. London ser. A, 210 (1910), pp. 307– 357

1910
[55]

Robert, M

F. Robert, M. Charnay, and F. Musy , It´ erations chaotiques s´ erie-parall` ele pour des 15 ´ equations non-lin´ eaires de point ﬁxe, Apl. Mat., 20 (1975), pp. 1–38

1975
[56]

D. Rose, R. Tarjan, and G. Lueker , Algorithmic aspects of vertex elimination on graphs , SIAM Journal on Computing, 5 (1976), pp. 266–283

1976
[57]

D. J. Rose and R. E. Tarjan , Algorithmic aspects of vertex elimination on directed grap hs, SIAM J. Appl. Math., 34 (1978), pp. 176–197

1978
[58]

Rosenfeld , A case study on programming for parallel processors

J. Rosenfeld , A case study on programming for parallel processors. , Tech. Rep. Research Report RC 64, IBM, W atson Research Center, Yorktown Heights , New-York, 1967

1967
[59]

Saad , Iterative Methods for Sparse Linear Systems, 2nd edition , SIAM, Philadelpha, PA, 2003

Y. Saad , Iterative Methods for Sparse Linear Systems, 2nd edition , SIAM, Philadelpha, PA, 2003

2003
[60]

Saad and H

Y. Saad and H. A. van der Vorst , Iterative solution of linear systems in the 20th century , J. of Comput. Appl. Math., 123 (2000), pp. 1–33

2000
[61]

Schaeffer , Briefwechsel zwischen Carl Friedrich Gauss und Christian L udwig Gerling , Otto Eisner, Berlin, 1927

C. Schaeffer , Briefwechsel zwischen Carl Friedrich Gauss und Christian L udwig Gerling , Otto Eisner, Berlin, 1927. see also the review of this book by David Eugene Smith

1927
[62]

Schechter , Relaxation methods for linear equations , Comm

S. Schechter , Relaxation methods for linear equations , Comm. Pure and appl. Math., 12 (1959), pp. 313–335

1959
[63]

L. Seidel , ¨ uber ein verfahren, die gleichungen, auf welche die method e der kleinsten quadrate f¨ uhrt, sowie lineare gleichungen uiberhaupt, durch succe ssive ann¨ aherung aufzul¨ osen, Akad. Wiss., Munich, mat.-nat. Abt. Abhandlungen, 11 (1874 ), pp. 81–108
[64]

J. W. Sheldon , On the numerical solution of elliptic diﬀerence equations , Mathematical Tables and Other Aids to Computation, 9 (1955), pp. 101–112

1955
[65]

Shortley, Use of tschebyscheﬀ polynomial operators in the numerical s olution of boundary value problems , J

G. Shortley, Use of tschebyscheﬀ polynomial operators in the numerical s olution of boundary value problems , J. Appl. Phys., 24 (1953), pp. 392–296

1953
[66]

G. L. G. Sleijpen and D. R. Fokkema , BICGSTAB(ℓ) for linear equations involving un- symmetric matrices with complex spectrum , Electr. Trans. on Numer. Analysis, 1 (1993), pp. 11–32

1993
[67]

D. E. Smith , Book review: Briefwechsel zwischen carl friedrich gauss un d christian ludwig gerling, by clemens schaeﬀer, eisner - berlin (1927) , Bull. of the Amer. Math. Soc., 34 (1928), pp. 665–666

1927
[68]

Sonneveld , CGS: a fast Lanczos-type solver for nonsymmetric linear sys tems, SIAM J

P. Sonneveld , CGS: a fast Lanczos-type solver for nonsymmetric linear sys tems, SIAM J. Sci. Stat. Comput., 10 (1989), pp. 36–52

1989
[69]

R. V. Southwell , Stress calclation in frameworks by the method of systematic relaxation of constraints, Proc. Roy. Soc. London, Ser. A, 151 (1935), pp. 56–95

1935
[70]

Press, 1940, 1943

, Relaxation methods in Engineering Science, A Treatise on Ap proximate Computation , Oxford Univ. Press, 1940, 1943

1940
[71]

2 , Oxford Univ

, Relaxation methods in theoretical physics, vol. 2 , Oxford Univ. Press, Oxford, 1946

1946
[72]

P. L. Tchebycheff , Theorie des mecanismes connus sous le nom de parallelogramm es (mem- oires presentes a l’academie imperiale des sciences de st-p etersbourg par divers savants vii, 1854, pp. 539-568) , in Oeuvres de P. L. Tchebycheﬀ, A. Markoﬀ and N. Sonin, eds., Academie i...
[73]

H. A. van der Vorst , Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG f or the solution of non-symmetric linear systems , SIAM J. Sci. Stat. Comput., 13 (1992), pp. 631–644

1992
[74]

R. S. V arga, A comparison of the successive overrelaxation method and se mi-iterative methods using Chebyshev polynomials , SIAM, 5 (1957)

1957
[75]

R. S. V arga, Matrix Iterative Analysis , Prentice Hall, Englewood Cliﬀs, NJ, 1962

1962
[76]

von Mises and H

R. von Mises and H. Pollaczek-Geiringer , Praktische verfahren der gleichungsauﬂ osung , Zeitschrift f ur Angewandte Mathematik und Mechanik, 9 (192 9), p. 58–77
[77]

D. M. Young , Iterative Methods for Solving Partial Diﬀerential Equatio ns of Elliptic Type , PhD thesis, Harvadd University, 1950

1950
[78]

, Iterative Solution of Large Linear Systems , Academic Press, New York, 1971

1971
[79]

D. M. Young , An historical review of iterative methods , in Proceedings of the ACM Conference on History of Scientiﬁc and Numeric Computation, HSNC ’87, N ew York, NY, USA, 1987, ACM, pp. 117–123

1987
[80]

Zurm ul , Matrizen, Springer, Berlin, 1950

R. Zurm ul , Matrizen, Springer, Berlin, 1950. 16

1950

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

D. N. D. G. Allen and R. V. Southwell , Relaxation methods applied to engineering problems xiv. plastic straining in two-dimensional stress-systems , Philosophical Transactions of the Royal Society of London. Series A, Mathematical and Physica l Sciences, 242 (1950)

work page 1950

[2] [2]

H. Anzt, S. Tomov, J. Dongarra, and V. Heuveline , A block-asynchronous relaxation method for graphics processing units , Journal of Parallel and Distributed Computing, 73 13 (2013), pp. 1613 – 1626. Heterogeneity in Parallel and Distr ibuted Computing

work page 2013

[3] [3]

Axelsson , A survey of preconditionied iterative methods for linear sy stems of algebraic equations, BIT, 25 (1985), pp

O. Axelsson , A survey of preconditionied iterative methods for linear sy stems of algebraic equations, BIT, 25 (1985), pp. 166–187

work page 1985

[4] [4]

Bahi and J.-C

M. Bahi and J.-C. Miellou , Contractive mappings with maximum norms: comparison of constants of contraction and application to asynchronous i terations, Parallel Comput., 19 (1993), pp. 511–523

work page 1993

[5] [5]

G. M. Baudet , Asynchronous iterative methods for multiprocessors , J. Assoc. Comput. Mach., 25 (1978), pp. 226–244

work page 1978

[6] [6]

Benzi , Gianfranco Cimmino’s contributions to numerical mathemat ics, Tech

M. Benzi , Gianfranco Cimmino’s contributions to numerical mathemat ics, Tech. Rep. Tec- noprint, 2005, Atti del Seminario di Analisi Matematica, Di partimento di Matematica dell’Universita‘ di Bologna, Bologna, 2005

work page 2005

[7] [7]

D. P. Bertsekas and D. El Baz , Distributed asynchronous relaxation methods for convex network ﬂow problems , SIAM J. Control Optim., 25 (1987), pp. 74–85

work page 1987

[8] [8]

Bodewig , Matrix Calculus, 2nd Edition , North-Holland, Amsterdam, 1959

E. Bodewig , Matrix Calculus, 2nd Edition , North-Holland, Amsterdam, 1959

work page 1959