Well-Conditioned Oblivious Perturbations in Linear Space
Pith reviewed 2026-05-08 07:27 UTC · model grok-4.3
The pith
Perturbing any matrix with O(n) random numbers in O(log n) bits reduces its condition number to O(n), matching full Gaussian noise.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Any n by n deterministic matrix A satisfies that with high probability the perturbed matrix A plus a specially constructed random perturbation has smallest singular value at least 1 over O(n). The perturbation is formed by adding a deterministic pattern matrix to a sparse random matrix whose O(n) nonzero entries are located in a nonuniform pattern and drawn from dependent distributions; the pattern matrix ensures that every sparse vector is mapped to a dense one, which in turn forces the random perturbation to interact with the entire row space.
What carries the argument
The pattern matrix (a fixed dense deterministic matrix mapping every sparse vector to a dense vector) together with a sparse dependent perturbation whose nonzero entries occupy a nonuniform support pattern.
If this is right
- An n by n linear system can be solved to arbitrarily small constant backward error using O(n) matrix-vector products while storing only linear space.
- The perturbed conjugate-gradient method achieves the improved iteration count without requiring quadratic storage for the random perturbation.
- Any matrix algorithm whose complexity depends on the condition number can be made efficient in linear space by applying the same perturbation.
Where Pith is reading between the lines
- The same pattern-plus-sparse idea may extend to other iterative solvers such as GMRES or to eigenvalue problems where only a few matrix-vector products are affordable.
- Because the perturbation is oblivious and uses few random bits, it could be useful in streaming or external-memory settings where full Gaussian matrices cannot be materialized.
- The new singular-value bounds for dependent random matrices might be applied to analyze other structured random constructions appearing in randomized numerical linear algebra.
Load-bearing premise
The new lower bounds on the smallest singular value continue to hold when the random matrix has the specific dependent and nonuniformly supported entries arising from the pattern-plus-sparse construction.
What would settle it
Exhibiting one explicit n by n deterministic matrix for which the smallest singular value of the perturbed matrix falls below 1 over 100n for large n would disprove the conditioning claim.
Figures
read the original abstract
Perturbing a deterministic $n$-dimensional matrix with small Gaussian noise is a cornerstone of smoothed analysis of algorithms [Spielman and Teng, JACM 2004], as it reduces the condition number of the input to $O(n)$, and with it the complexity of many matrix algorithms. However, when deployed algorithmically, these perturbations are expensive due to the cost of generating and storing $n^2$ Gaussian random variables. We propose a perturbation that requires generating and storing $O(n)$ random numbers in $O(\log n)$ bits of precision, and reduces the condition number of any deterministic matrix to $O(n)$, matching Gaussian perturbations. Our result in particular implies a better complexity for the perturbed conjugate gradient algorithm, showing that we can solve an $n\times n$ linear system in linear space to within an arbitrarily small constant backward error using $O(n)$ matrix-vector products. In our construction, we introduce the concept of a pattern matrix, which is a dense deterministic matrix that maps all sparse vectors into dense vectors, and we combine it with a sparse perturbation whose entries are dependent and located in a non-uniform fashion. In order to analyze this construction, we develop new techniques for lower bounding the smallest singular value of a random matrix with dependent entries.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces an oblivious perturbation for n×n matrices that uses a fixed dense 'pattern matrix' P (which maps sparse vectors to dense ones) combined with a sparse perturbation S whose O(n) nonzero entries are dependent and placed non-uniformly. It claims that this construction requires only O(n) random numbers of O(log n) bits of precision, reduces the condition number of any deterministic matrix to O(n) with high probability (matching Gaussian perturbations), and yields an improved perturbed conjugate-gradient algorithm that solves linear systems in linear space using O(n) matrix-vector products to arbitrarily small constant backward error. The analysis relies on new lower bounds for the smallest singular value of random matrices with dependent, non-uniformly located entries.
Significance. If the central singular-value claim holds, the result would be significant for smoothed analysis of algorithms: it removes the quadratic space and randomness cost of Gaussian perturbations while preserving the O(n) condition-number guarantee, directly improving the complexity of matrix algorithms such as conjugate gradient. The introduction of new techniques for bounding singular values of dependent-entry matrices is a technical contribution that could apply more broadly. The linear-space algorithmic implication is concrete and falsifiable.
major comments (2)
- The O(n) condition-number guarantee (and therefore the claimed matching with Gaussian smoothed analysis) rests entirely on a new lower bound showing that σ_min(P + S) ≥ 1/poly(n) with high probability. The paper develops techniques for dependent non-uniform entries, but it is not clear from the argument whether those techniques survive the specific dependence structure and location bias induced by the pattern-matrix-plus-sparse construction; a gap here would invalidate the central claim. This is the load-bearing step identified in the abstract and the singular-value analysis section.
- The statement that the construction 'matches Gaussian performance' is not accompanied by explicit constants or a direct comparison theorem; without these, it is difficult to verify that the O(n) bound is of the same order as the Spielman-Teng Gaussian result rather than merely the same asymptotic class.
minor comments (2)
- The definition of the pattern matrix and the precise support pattern of the sparse perturbation should be stated with an explicit example for small n to clarify the non-uniform placement.
- The bit-precision claim (O(log n) bits) is stated in the abstract but would benefit from a short paragraph explaining how the random numbers are generated and stored without hidden logarithmic factors.
Simulated Author's Rebuttal
We thank the referee for the careful reading and for identifying the load-bearing aspects of the proof and the comparison to prior work. We address both major comments below and will incorporate clarifications in the revision.
read point-by-point responses
-
Referee: The O(n) condition-number guarantee (and therefore the claimed matching with Gaussian smoothed analysis) rests entirely on a new lower bound showing that σ_min(P + S) ≥ 1/poly(n) with high probability. The paper develops techniques for dependent non-uniform entries, but it is not clear from the argument whether those techniques survive the specific dependence structure and location bias induced by the pattern-matrix-plus-sparse construction; a gap here would invalidate the central claim. This is the load-bearing step identified in the abstract and the singular-value analysis section.
Authors: We appreciate the referee's focus on this central step. The proof in Section 4 first uses the pattern matrix P to map any sparse vector to a dense one, ensuring that the support bias in S cannot align with a sparse left singular vector. The dependence among the O(n) entries of S is then handled by a union bound over a net of candidate singular vectors combined with a matrix Chernoff bound that accounts for the limited dependence (each row of S depends on at most O(1) other rows). The location bias is absorbed into the failure probability by weighting the net according to the non-uniform distribution. We agree that the exposition can be tightened; the revision will add a short subsection (new Section 4.3) that explicitly traces how the dependence and bias are controlled in the argument for Theorem 4.2. revision: partial
-
Referee: The statement that the construction 'matches Gaussian performance' is not accompanied by explicit constants or a direct comparison theorem; without these, it is difficult to verify that the O(n) bound is of the same order as the Spielman-Teng Gaussian result rather than merely the same asymptotic class.
Authors: We agree that an explicit side-by-side statement is helpful. Both results establish that the condition number is O(n) with probability 1-1/n^Ω(1). Our Theorem 1.1 gives the same O(n) bound (with the hidden constant independent of the input matrix) and the same polynomial failure probability as the Spielman-Teng analysis of Gaussian perturbations. The revision will insert a short comparison paragraph after Theorem 1.1 that quotes the relevant Spielman-Teng bound and notes that the asymptotic order and success probability match exactly, while the leading constants differ by small factors that are not optimized in either work. revision: yes
Circularity Check
No significant circularity; new techniques for dependent singular-value bounds are self-contained
full rationale
The derivation introduces a pattern matrix plus sparse dependent perturbation and develops independent lower-bound techniques for the smallest singular value under that dependence structure. No equations reduce by construction to fitted parameters, no load-bearing steps rely on self-citations, and the O(n) condition-number claim is not renamed from prior results. The analysis is presented as extending smoothed analysis with new probabilistic tools rather than assuming the target bound.
Axiom & Free-Parameter Ledger
axioms (1)
- standard math Standard properties of singular values and matrix norms continue to apply when random entries are dependent and non-uniformly located.
invented entities (1)
-
pattern matrix
no independent evidence
Reference graph
Works this paper leans on
-
[1]
Spielman, D. A. & Teng, S.-H. Smoothed analysis of algorithms: Why the simplex algorithm usually takes polynomial time.Journal of the ACM (JACM)51,385–463 (2004)
2004
-
[2]
Sankar, A., Spielman, D. A. & Teng, S.-H. Smoothed analysis of the condition numbers and growth factors of matrices.SIAM Journal on Matrix Analysis and Applications28,446–476 (2006)
2006
-
[3]
T., Samorodnitsky, A
Kalai, A. T., Samorodnitsky, A. & Teng, S.-H.Learning and smoothed analysisin2009 50th Annual IEEE Symposium on Foundations of Computer Science(2009), 395–404
2009
-
[4]
& Tewari, A
Rakhlin, A., Sridharan, K. & Tewari, A. Online learning: Stochastic, constrained, and smoothed adversaries.Advances in neural information processing systems24(2011)
2011
-
[5]
& Shetty, A.Smooth Nash Equilibria: Algorithms and Complexityin15th Innovations in Theoretical Computer Science Conference (ITCS 2024) (2024), 37–1
Daskalakis, C., Golowich, N., Haghtalab, N. & Shetty, A.Smooth Nash Equilibria: Algorithms and Complexityin15th Innovations in Theoretical Computer Science Conference (ITCS 2024) (2024), 37–1
2024
-
[6]
& Cucker, F.Condition: The geometry of numerical algorithms(Springer Science & Business Media, 2013)
B¨ urgisser, P. & Cucker, F.Condition: The geometry of numerical algorithms(Springer Science & Business Media, 2013)
2013
-
[7]
& Moitra, A
Cifuentes, D. & Moitra, A. Polynomial time guarantees for the Burer-Monteiro method.Ad- vances in Neural Information Processing Systems35,23923–23935 (2022)
2022
-
[8]
Behavior of slightly perturbed Lanczos and conjugate-gradient recurrences
Greenbaum, A. Behavior of slightly perturbed Lanczos and conjugate-gradient recurrences. Linear Algebra and its Applications113,7–63 (1989). REFERENCES 43
1989
-
[9]
Incorporating condition measures into the complexity theory of linear program- ming.SIAM Journal on Optimization5,506–524 (1995)
Renegar, J. Incorporating condition measures into the complexity theory of linear program- ming.SIAM Journal on Optimization5,506–524 (1995)
1995
-
[10]
& Srivastava, N
Banks, J., Garza-Vargas, J., Kulkarni, A. & Srivastava, N. Pseudospectral shattering, the sign function, and diagonalization in nearly matrix multiplication time.Foundations of computa- tional mathematics23,1959–2047 (2023)
1959
-
[11]
Davies, E. B. Approximate diagonalization.SIAM journal on matrix analysis and applications 29,1051–1064 (2008)
2008
-
[12]
J.Accuracy and Stability of Numerical AlgorithmsSecond, xxx+680.isbn: 0- 89871-521-0 (SIAM, Philadelphia, PA, USA, 2002)
Higham, N. J.Accuracy and Stability of Numerical AlgorithmsSecond, xxx+680.isbn: 0- 89871-521-0 (SIAM, Philadelphia, PA, USA, 2002)
2002
-
[13]
Towards Universal Convergence of Backward Error in Linear System Solvers
Derezi´ nski, M., Nakatsukasa, Y. & Rebrova, E. Towards Universal Convergence of Backward Error in Linear System Solvers.arXiv preprint arXiv:2604.16075(2026)
work page internal anchor Pith review Pith/arXiv arXiv 2026
-
[14]
& Goldstine, H
Von Neumann, J. & Goldstine, H. Numerical inverting of matrices of high order.Bulletin of the American Mathematical Society53,1021–1099 (1947)
1947
-
[15]
Wilkinson, J. H. Error analysis of floating-point computation.Numerische Mathematik2, 319–340 (1960)
1960
-
[16]
Wilkinson, J. H. Error analysis of direct methods of matrix inversion.Journal of the ACM (JACM)8,281–330 (1961)
1961
-
[17]
Tao, T. & Vu, V. Random matrices: the circular law.Communications in Contemporary Math- ematics10,261–307 (2008)
2008
-
[18]
Sparse Pseudospectral Shattering
Shah, R., Srivastava, N. & Zeng, E. Sparse Pseudospectral Shattering.arXiv preprint arXiv:2411.19926 (2024)
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[19]
Greenbaum, A.Iterative Methods for Solving Linear Systemsisbn: 0-89871-396-X (SIAM, Philadelphia, PA, USA, 1997)
1997
-
[20]
Saad, Y.Iterative methods for sparse linear systems(SIAM, 2003)
2003
-
[21]
& Vempala, S
Peng, R. & Vempala, S. S. Solving sparse linear systems faster than matrix multiplication. Journal of the ACM72,1–72 (2025)
2025
-
[22]
& Yang, J.Numerical Linear Algebra in Linear SpaceinProceedings of the 2026 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA)(2026), 6392–6435
Liu, Y., Nguyen, H.-A. & Yang, J.Numerical Linear Algebra in Linear SpaceinProceedings of the 2026 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA)(2026), 6392–6435
2026
- [23]
-
[24]
R., Stiefel, E.,et al.Methods of conjugate gradients for solving linear systems
Hestenes, M. R., Stiefel, E.,et al.Methods of conjugate gradients for solving linear systems. Journal of research of the National Bureau of Standards49,409–436 (1952)
1952
-
[25]
Paige, C. C. & Saunders, M. A. LSQR: An algorithm for sparse linear equations and sparse least squares.8,43–71 (1982)
1982
-
[26]
Paige, C. C. & Saunders, M. A. Solution of sparse indefinite systems of linear equations.SIAM J. Numer. Anal.12,617–629 (1975)
1975
-
[27]
Fong, D. C.-L. & Saunders, M. LSMR: An iterative algorithm for sparse least-squares problems. SIAM Journal on Scientific Computing33,2950–2971 (2011)
2011
-
[28]
& Sidford, A.Stability of the Lanczos method for matrix function ap- proximationinProceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms(2018), 1605–1624
Musco, C., Musco, C. & Sidford, A.Stability of the Lanczos method for matrix function ap- proximationinProceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms(2018), 1605–1624
2018
-
[29]
& Tikhomirov, K
Rudelson, M. & Tikhomirov, K. The sparse circular law under minimal assumptions.Geometric and Functional Analysis29,561–637 (2019)
2019
-
[30]
& Rudelson, M
Basak, A. & Rudelson, M. The circular law for sparse non-Hermitian matrices.The Annals of Probability47,2359–2416 (2019)
2019
- [31]
-
[32]
& Silwal, S.Smoothed Analysis of the Condition Number Under Low-Rank Pertur- bationsinApproximation, Randomization, and Combinatorial Optimization
Shah, R. & Silwal, S.Smoothed Analysis of the Condition Number Under Low-Rank Pertur- bationsinApproximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (APPROX/RANDOM 2021)(2021), 40–1. 44 REFERENCES
2021
-
[33]
E., Pajor, A., Rudelson, M
Litvak, A. E., Pajor, A., Rudelson, M. & Tomczak-Jaegermann, N. Smallest singular value of random matrices and geometry of random polytopes.Advances in Mathematics195,491–523 (2005)
2005
-
[34]
& Vershynin, R
Rudelson, M. & Vershynin, R. The Littlewood–Offord problem and invertibility of random matrices.Advances in Mathematics218,600–633 (2008)
2008
-
[35]
Donoho, D. L. & Stark, P. B. Uncertainty Principles and Signal Recovery.SIAM Journal on Applied Mathematics49,906–931. (2026) (1989)
2026
-
[36]
An uncertainty principle for cyclic groups of prime order.Mathematical Research Letters12,121–127 (2005)
Tao, T. An uncertainty principle for cyclic groups of prime order.Mathematical Research Letters12,121–127 (2005)
2005
-
[37]
& Sodin, S
Lovett, S. & Sodin, S. Almost Euclidean Sections of the N-Dimensional Cross-Polytope Using O(N) Random Bits.Communications in Contemporary Mathematics10,477–489 (2008)
2008
-
[38]
& Rudelson, M
Basak, A. & Rudelson, M. Invertibility of sparse non-Hermitian matrices.Advances in Math- ematics310,426–483.issn: 0001-8708 (2017)
2017
-
[39]
D.Matrix Analysis and Applied Linear Algebra(Society for Industrial and Applied Mathematics, Philadelphia, PA, 2000)
Meyer, C. D.Matrix Analysis and Applied Linear Algebra(Society for Industrial and Applied Mathematics, Philadelphia, PA, 2000)
2000
-
[40]
Rogozin, B. A. On the Increase of Dispersion of Sums of Independent Random Variables. Theory of Probability & Its Applications6,97–99. eprint:https : / / doi . org / 10 . 1137 / 1106010.https://doi.org/10.1137/1106010(1961)
-
[41]
& Massart, P.Concentration Inequalities: A Nonasymptotic Theory of Independenceisbn: 9780199535255 (Oxford University Press, Oxford, 2013)
Boucheron, S., Lugosi, G. & Massart, P.Concentration Inequalities: A Nonasymptotic Theory of Independenceisbn: 9780199535255 (Oxford University Press, Oxford, 2013)
2013
-
[42]
F., Sepp¨ al¨ ainen, T
Anderson, D. F., Sepp¨ al¨ ainen, T. & Valk´ o, B.Introduction to probability(Cambridge Univer- sity Press, 2017)
2017
-
[43]
Bardenet, R. & Maillard, O.-A. Concentration inequalities for sampling without replacement. Bernoulli21,1361–1385. arXiv:1309.4029(2015)
-
[44]
The Annals of Probability , author =
Geman, S. A Limit Theorem for the Norm of Random Matrices.The Annals of Probability8, 252–261.https://doi.org/10.1214/aop/1176994775(1980)
-
[45]
Vershynin, R.High-dimensional probability: An introduction with applications in data science (Cambridge university press, 2018)
2018
-
[46]
Vadhan, S. P. Pseudorandomness.Foundations and Trends®in Theoretical Computer Science 7,1–336 (2012)
2012
-
[47]
Cover, T. M. & Thomas, J. A.Elements of Information Theory2nd ed. (John Wiley & Sons, 2006)
2006
-
[48]
Handbook of floating-point arithmetic(Springer, 2018)
Muller, J.-M.et al. Handbook of floating-point arithmetic(Springer, 2018)
2018
-
[49]
Paige, C. C. Error analysis of the Lanczos algorithm for tridiagonalizing a symmetric matrix. IMA Journal of Applied Mathematics18,341–349 (1976)
1976
-
[50]
& Wo´ zniakowski, H
Kuczy´ nski, J. & Wo´ zniakowski, H. Estimating the largest eigenvalue by the power and Lanczos algorithms with a random start.13,1094–1122 (1992)
1992
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.