{"id":"58f62ae7-646e-4cef-9976-777aa83df177","arxiv_id":"1908.01083","paper_version":1,"verdict":"UNVERDICTED","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"A concise historical survey of iterative linear solvers, from Gauss's relaxation through SOR and Krylov methods to asynchronous iterations and machine learning applications.","lead":"This paper traces the history of iterative methods for solving linear equations, from Gauss's 1823 letter to modern Krylov and asynchronous parallel methods. A generalist reader can see how computer hardware and application needs repeatedly reshaped numerical linear algebra.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No load-bearing objection identified: the historical thesis is coherent and hedged; only a minor typo in Eq. (3.1) and unverified \"first\" claims, neither of which changes the narrative.","rationale":"The reader correctly notes that the paper is a historical survey rather than a falsifiable research claim, and hence UNVERDICTED is a reasonable disposition. My stress-test looked for a load-bearing weakness in the central causal claim: that iterative methods were shaped by applications and hardware. The paper supports this with a consistent, well-cited timeline (geodesy/least squares, PDEs, engineering, digital computers, parallel machines), and the author explicitly disclaims exhaustiveness. I could not find a point at which the thesis would collapse if one historical priority claim were revised. The only concrete mathematical issue I identified is Eq. (3.1), which conflicts with Eq. (3.2) under the paper's own residual definition; this is a typo in the Richardson formula but does not affect the historical narrative. The reader's weakest assumption (source reliability for \"first\" claims) is reasonable but not load-bearing: those superlatives are side remarks, and the main argument does not depend on them. Therefore I leave the verdict unchanged. agreement_with_reader is partial rather than agree because the reader's stated weakest assumption is not the same as the soft spot I found; I agree that primary-source verification is absent, but I do not think it threatens the central claim.","tokens_in":16483,"tokens_out":8817,"duration_ms":89789,"concrete_test":"Compare Eq. (3.1) with Richardson's original [54] or a standard treatment such as Saad [59, Section 12.3]: if the published update is x_{j+1} = x_j - r_j/alpha_j, then Eq. (3.2) follows and Eq. (3.1) has a typo. This is the one check that would settle the only concrete concern; the broader historical narrative is already supported by the cited documents.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is an interpretive historical narrative, not a falsifiable theorem: that iterative methods repeatedly changed direction in response to application and hardware pressures. The author explicitly disclaims exhaustiveness in the abstract, and the specific priority claims (Gauss 1823 as first iterative method; Jacobi rotations as first preconditioning; Richardson unaware of Gauss/Seidel) are attributable to cited translations and standard secondary literature ([19], [35], [36], [34], [79]), even though they are not independently audited against primary archives. I find no internal inconsistency that threatens the causal thesis. The one concrete defect is in Section 3, Eq. (3.1): with the paper's own residual convention r = Ax + b, Richardson's update cannot be x_{j+1} = x_j - (1/alpha_j) A r_j, since that would give r_{j+1} = (I - A^2/alpha_j) r_j rather than the (I - A/alpha_j) r_j required by Eq. (3.2). The correct update is x_{j+1} = x_j - (1/alpha_j) r_j. This is a typographical or transcription error, not a flaw in the historical argument: Richardson still introduced polynomial iteration, and the subsequent discussion of Chebyshev acceleration is unaffected. Likewise, the \"first known\" claims are peripheral to the main narrative; even if one were overstated, the survey's thesis would survive. For these reasons I do not regard the reader's source-reliability concern as load-bearing, though it is worth a spot-check.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript is a historical survey of iterative methods for solving linear systems, beginning with Gauss's 1823 letter to Gerling and moving through relaxation methods, SOR, Krylov subspace methods, sparse direct solvers, asynchronous iterations, and contemporary machine-learning-related developments. The central thesis is that iterative methods have repeatedly changed direction in response to application demands and hardware constraints, and the paper argues that the next frontier lies in machine-learning-driven linear algebra. The paper is explicitly not exhaustive and aims to 'underline the way of thinking at a specific time' rather than to provide a complete bibliography.","tokens_in":16757,"tokens_out":4937,"duration_ms":46416,"significance":"If accepted as an interpretation, the survey provides a valuable synthesis by a leading researcher in numerical linear algebra, with useful primary-source documentation including Forsythe's translation of Gauss's letter, Southwell and Young anecdotes, and Varga's quantitative examples. Its strengths include a broad and well-cited reference list, careful hedging of priority claims with phrases such as 'first known' and 'seems to be,' and explicit admission of the survey's limits. The paper presents no new mathematics, so its contribution is historiographical rather than technical; for a journal or venue in the history of mathematics this is appropriate. The historical narrative is coherent and does not contain internal inconsistencies that undermine the central claim.","major_comments":[],"minor_comments":[{"comment":"The displayed Richardson update is inconsistent with the residual convention r = Ax + b stated in the same section: substituting x_{j+1} = x_j - (1/alpha_j) A r_j into r_{j+1} = A x_{j+1} + b gives r_{j+1} = (I - A^2/alpha_j) r_j, not the factor (I - A/alpha_j) required by Eq. (3.2). The intended update is presumably x_{j+1} = x_j - (1/alpha_j) r_j. This is a transcription error rather than a flaw in the historical argument, but it should be corrected.","section":"3, Eq. (3.1)"},{"comment":"The sentence 'write Ax = b as (D - E)x = F + b' appears to have a typo: it should be '(D - E)x = F x + b'. As printed, the right-hand side has incompatible dimensions.","section":"2, Eq. (2.4)"},{"comment":"The priority claims that Gauss's 1823 letter is 'the first known reference' to an iterative method and that Jacobi rotations are 'the first known form of preconditioning' are appropriately hedged, but they rest on secondary literature rather than an independent archival audit. A sentence making this attribution explicit would strengthen the scholarly apparatus.","section":"1"},{"comment":"The manuscript would benefit from a careful proofreading pass. Examples of typos and minor errors include 'exaustive' in the abstract, 'there there were' in Section 1, 'phsyics' in Section 4, 'targetted' in Section 5, 'Bulleting' in Section 6, 'Hesteness' in Section 10, and 'disapperaring' and 'it they are' in Section 12.","section":"Global"},{"comment":"The column header of Table 8.1 ('Off-diagonal Ordering Factor Nonz Time') is garbled; the table should be reformatted so that the columns for nonzeros and time are clearly labeled.","section":"8, Table 8.1"}],"recommendation":"minor_revision","confidential_remarks":"The paper is largely a synthesis of the author's own earlier survey [60] and textbook [59]. This is acceptable self-reference in a historical survey, but the editor may wish to confirm that the overlap is appropriately disclosed in the front matter."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Take: this is a fine, readable survey by someone who knows the territory. It will be useful for people entering numerical linear algebra and for anyone teaching the history of iterative methods. What is genuinely new is modest: the author's personal recollections of the French asynchronous iteration school and the explicit 2-D vs 3-D complexity comparison for sparse direct solvers. The rest is a coherent synthesis of standard accounts, and it is accurate in its mathematics.\n\nThe paper does several things well. Gauss's 1823 letter to Gerling is presented concretely with the actual system and iteration table, which is a nice touch. The discussion of Southwell, Young, and the SOR era is well grounded. The treatment of Richardson's contribution correctly notes that he missed Chebyshev acceleration despite identifying the right problem. And the author is explicit at the start that the survey is not exhaustive; that disclaimer is honest and should be taken at face value.\n\nSoft spots are minor. The most concrete defect is Eq. (3.1): with the residual convention r = Ax + b, the update should be x_{j+1} = x_j - (1/alpha_j) r_j, not A r_j. That is a typo, and it does not affect the historical argument. The priority claims – Gauss as 'first known' iterative method, Jacobi rotations as 'first known' preconditioning, Richardson unaware of Gauss and Seidel – are asserted with more confidence than a careful historian might want. They are traceable to the cited secondary literature, but a survey of this kind should either say 'appears to be' or footnote the uncertainty more consistently. The author does use 'appears' for Gauss, but the Jacobi claim is stated flatly.\n\nThe reader's cautious 'UNVERDICTED' verdict is reasonable in the sense that there is no new falsifiable claim here, but it is the wrong frame for a survey. The paper is not trying to prove a theorem. On its own terms, the central historical thesis – that the field repeatedly shifted direction in response to application and hardware pressure – holds up and is well supported. The citation pattern is fine; self-citation to Saad & van der Vorst (2000) is appropriate for a compressed version of the 20th century.\n\nWho is this for? Colleagues who want a short historical orientation before teaching, students looking for context, and researchers who want a compact reminder of how the pieces fit. It is not a research contribution, but it deserves to be refereed and published as a survey. I would bring it to reading group and would cite it for the 2-D vs 3-D complexity table and the historical timeline.\n\nRecommendation: send to peer review as a survey; the typo should be fixed and the 'first' claims lightly hedged, but no deeper problem exists.","headline":"A solid, readable historical survey from a leading expert; the math is right and the narrative holds, though the 'first' claims need hedging and Eq. (3.1) has a typo.","tokens_in":17257,"tokens_out":2967,"would_cite":true,"duration_ms":25843,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["65F10","01A55","01A60"],"pacs":[],"model":"deepseek-v4-flash","headline":"Iterative methods for linear systems have been redirected again and again by application demands and computing hardware, from 1823 relaxation to today's machine-learning-scale problems.","keywords":["iterative methods","history of numerical analysis","Gauss-Seidel method","successive over-relaxation","Krylov subspace methods","conjugate gradient","preconditioning","asynchronous iterations"],"falsifier":"A search of pre-1823 geodesy, astronomy, or actuarial manuscripts for a residual-updating indirect solution scheme—or archival evidence that Richardson knew the older relaxation work—would directly test the two signature historical claims.","tokens_in":16265,"feed_emoji":"📐","tokens_out":8764,"duration_ms":85169,"temperature":0.7,"pith_summary":"The paper sets out to show that iterative methods for linear systems have never developed in a straight line: every major turn—from Gauss's indirect elimination in 1823, through relaxation and SOR, to Krylov subspace methods, sparse direct solvers, and asynchronous iterations—was a response to new application demands and to the computing hardware of the moment. The survey is deliberately selective and reads the record as a series of ways of thinking rather than a list of algorithms. A sympathetic reader should care because the pattern gives a basis for predicting the next turn: the paper locates that turn in data mining and machine learning, where randomness and stochasticity are central. The historical claims are grounded in primary sources such as an 1823 letter and early twentieth-century papers, together with the standard monographs that codified each era.","feed_headline":"Applications, not math alone, wrote linear-solver history","feed_subtitle":"From 1823 relaxation to today's solver families and machine learning, a survey shows what pushed each turn.","key_machinery":"The recurring mathematical object carrying the story is the residual polynomial. In Richardson's scheme the residual after $k$ steps is $r_k = p_k(A) r_0$ with $p_k(0) = 1$, so making the iteration fast is the problem of choosing a polynomial that is small on the spectrum of $A$. That single identity connects the fixed-point splittings behind relaxation ($A = D - E - F$, giving Jacobi and Gauss-Seidel), Chebyshev acceleration, the three-term recurrence of conjugate gradients, and preconditioning as a way of reshaping the spectrum. It explains both why SOR needed an optimal over-relaxation parameter and why CG removed the need for eigenvalue estimates.","core_discovery":"The paper's central claim is that the history of iterative methods is driven from the outside in: applications and machines set the agenda, and the mathematics follows. It argues that Gauss's 1823 indirect elimination—updating one coordinate at a time by zeroing the largest residual component—is the first known iterative method, that Jacobi's 1845 relaxation paper introduced what is effectively the first preconditioner through rotations, and that each later family (Liebmann/Gauss-Seidel, SOR, Richardson polynomial iteration, conjugate gradients, nonsymmetric Krylov accelerators) answered a specific practical pressure. The conjugate gradient method, in this telling, was the single most important advance of the 1950s, but it became dominant only after incomplete factorization preconditioning made it reliable in the 1970s. The survey ends by arguing that the next pressure is already visible in machine-learning-scale linear algebra, where the global-optimality assumptions of CG and GMRES fit poorly with randomness.","pith_inferences":["If the paper's application-driven pattern holds, machine-learning workloads will not just add new problems but will push for solver reformulations built around randomness and low-rank structure, as the paper hints at the end.","The same pattern suggests that learned selection of orderings, preconditioners, or solver families—rather than a single new algorithm—is the likely near-term next direction, extending the historical role of heuristics in sparse direct methods.","The 1970s dismissal of asynchronous iterations as utopian, followed by their return on massively parallel machines, implies that communication cost, not flops, may be the binding constraint that determines the next solver family."],"forward_implications":["Because cyclic relaxation is easy to mechanize, Gauss-Seidel-type iterations became the natural first iterative solvers on digital computers, vindicating the mechanization that Southwell dismissed.","SOR with a well-chosen over-relaxation parameter could handle systems of order 20,000 in 1960 and 108,000 in 3-D versions—sizes direct methods could not touch at the time.","Conjugate gradients became the default for large symmetric positive definite systems only after incomplete Cholesky preconditioning (ICCG) appeared in 1977, not when the method was invented in the early 1950s.","For nonsymmetric problems, application demand produced a sequence of transpose-free accelerators (CGS, BiCGSTAB, QMR), and research has since shifted from accelerators to preconditioners.","Sparse direct methods remain competitive for 2-D problems but face intrinsic $O(N^{4/3})$ fill and $O(N^2)$ time in 3-D, which is why iterative methods stay essential for large 3-D discretizations."],"supporting_citations":[{"why":"Supplies the primary text for the origin claim that iterative solution begins with an 1823 letter on indirect elimination.","marker":"[19]"},{"why":"Supplies the relaxation variant and the rotation trick the paper calls the first preconditioning.","marker":"[35]"},{"why":"Introduces the polynomial-residual viewpoint and the missed Chebyshev acceleration.","marker":"[54]"},{"why":"Introduces the extrapolated Liebmann/SOR scheme and an optimal parameter for finite-difference Laplacians.","marker":"[23]"},{"why":"Generalizes SOR theory beyond the Laplacian case and defines the SOR era.","marker":"[77]"},{"why":"Codifies the SOR and semi-iterative theory and documents the large 1960 engineering successes.","marker":"[75]"},{"why":"Sets the post-war research agenda and introduces conjugate gradients to a broad audience.","marker":"[20]"},{"why":"Makes CG practical for symmetric positive definite systems through incomplete Cholesky preconditioning.","marker":"[43]"},{"why":"Provides the sparse direct method synthesis and the 2-D versus 3-D complexity comparison.","marker":"[29]"},{"why":"Frames the twentieth-century history and contextualizes nonsymmetric Krylov accelerators.","marker":"[60]"}],"fun_headline_variants":["Gauss's 1823 trick: the first iterative solver, by need","Applications steered linear solvers, not pure math","From Gauss to GMRES: real problems drove each iteration","Conjugate gradients won only after 1970s preconditioning","Machine learning now pressures linear solvers, survey says"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The origin and priority claims depend on the historical record being complete: if an earlier iterative method or an undocumented line of influence exists, the story would shift.","fun_headline_variants_meta":{"raw":{"variants":["Gauss's 1823 trick: the first iterative solver, by need","Applications steered linear solvers, not pure math","From Gauss to GMRES: real problems drove each iteration","Conjugate gradients won only after 1970s preconditioning","Machine learning now pressures linear solvers, survey says"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000213,"raw_usage":{"total_tokens":1368,"prompt_tokens":839,"completion_tokens":529,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":455,"completion_tokens_details":{"reasoning_tokens":444}},"tokens_in":455,"tokens_out":529,"duration_ms":5407,"temperature":1.0,"reasoning_tokens":444,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T15:23:25.491372+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A search of pre-1823 geodesy, astronomy, or actuarial manuscripts for a residual-updating indirect solution scheme—or archival evidence that Richardson knew the older relaxation work—would directly test the two signature historical claims.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the primary text for the origin claim that iterative solution begins with an 1823 letter on indirect elimination."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the relaxation variant and the rotation trick the paper calls the first preconditioning."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the polynomial-residual viewpoint and the missed Chebyshev acceleration."},{"cited_title":"Frankel , Convergence rates of iterative treatments of partial differ ential equations, MTAC, (1950), pp","cited_arxiv_id":null,"evidence_quote":"Introduces the extrapolated Liebmann/SOR scheme and an optimal parameter for finite-difference Laplacians."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Generalizes SOR theory beyond the Laplacian case and defines the SOR era."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Codifies the SOR and semi-iterative theory and documents the large 1960 engineering successes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Sets the post-war research agenda and introduces conjugate gradients to a broad audience."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Makes CG practical for symmetric positive definite systems through incomplete Cholesky preconditioning."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the sparse direct method synthesis and the 2-D versus 3-D complexity comparison."},{"cited_title":"Saad and H","cited_arxiv_id":null,"evidence_quote":"Frames the twentieth-century history and contextualizes nonsymmetric Krylov accelerators."}],"review_version":1}