Basis pursuit by inconsistent alternating projections
Pith reviewed 2026-05-18 21:40 UTC · model grok-4.3
The pith
Basis pursuit is solved by alternating projections on deliberately inconsistent subproblems that converge linearly.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
By enforcing inconsistency in alternating projections subproblems for the ℓ1-minimization form of basis pursuit, the ℓ1-radii converge linearly to the optimal value, and when the solution is unique, all generated sequences converge linearly to it at a rate governed by a natural error bound between the feasible set and the optimal ℓ1-ball.
What carries the argument
The inconsistent alternating projections scheme, with subproblems deliberately constructed to have disjoint feasible sets.
If this is right
- The ℓ1-radii converge linearly to the optimal value.
- When the solution is unique, the generated sequences converge linearly to the solution.
- The convergence rate is controlled by the natural error bound between the feasible set and the optimal ℓ1-ball.
- The method is numerically competitive against state-of-the-art open-source solvers on synthetic and real-world instances.
Where Pith is reading between the lines
- The deliberate inconsistency technique might accelerate projection methods on other convex problems that are usually solved via linear programming reformulations.
- Implementation on large-scale compressed sensing instances could show whether the linear rate translates into practical speed gains over interior-point or first-order LP solvers.
- The error-bound analysis may extend to related sparse recovery formulations with additional convex constraints.
Load-bearing premise
Subproblems can be constructed with disjoint feasible sets by design, and a natural error bound exists between the feasible set and the optimal ℓ1-ball.
What would settle it
A numerical run on an instance with unique solution in which the observed convergence rate of the radii or iterates deviates from the linear rate predicted by the error bound.
Figures
read the original abstract
Basis pursuit is the problem of finding a vector with smallest $\ell_1$-norm among the solutions of a given linear system of equations. It is a well-known convex relaxation of the sparse affine feasibility problem, where sparse solutions to underdetermined systems are sought. Since basis pursuit admits a linear programming reformulation, standard LP solvers are directly applicable. We instead address the basis pursuit directly in its $\ell_1$-minimization form, without LP reformulation, via a scheme that uses alternating projections in its subproblems. These subproblems are designed to be inconsistent in the sense that they relate to two non-intersecting sets. Recently in [R. Behling, Y. Bello-Cruz and L.-R. Santos, SIAM J. Optim., 31 (2021), pp. 2863-2892], inconsistency coming from infeasibility has been shown to accelerate convergence of alternating projections. We deliberately enforce this inconsistency by constructing subproblems whose feasible sets are disjoint by design. We prove that the resulting $\ell_1$-radii converge linearly to the optimal value, and that when the solution is unique, all generated sequences converge linearly to it at a rate governed by a natural error bound between the feasible set and the optimal $\ell_1$-ball. The proposed method is numerically competitive against state-of-the-art open-source solvers on synthetic and real-world instances.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes solving basis pursuit (minimum ℓ1-norm solution to Ax=b) via alternating projections applied to a sequence of deliberately inconsistent subproblems whose feasible sets are constructed to be disjoint. It claims to prove that the generated ℓ1-radii converge linearly to the optimal value, and that when the solution is unique all iterates converge linearly to it at a rate controlled by a natural error bound between each feasible set and the optimal ℓ1-ball. Numerical tests indicate competitiveness with state-of-the-art open-source LP solvers on synthetic and real instances.
Significance. If the linear-convergence claims can be placed on a fully rigorous footing with an explicit, instance-independent positive lower bound on the governing error-bound constant, the work would supply a projection-based alternative to LP reformulations of basis pursuit and would illustrate a concrete use of inconsistency-driven acceleration for a canonical sparse-recovery problem.
major comments (2)
- [§4 (Convergence Analysis)] §4 (Convergence Analysis), main theorem on linear rates: the linear convergence of both radii and (unique-solution) iterates is asserted to follow from a 'natural error bound' between the constructed feasible set and the optimal ℓ1-ball, yet no explicit positive lower bound on this constant is derived in terms of A, b or the current radius. Without such a bound the claimed uniform linear rate is not guaranteed and the central convergence result remains conditional on an unverified assumption.
- [§3 (Subproblem construction)] §3 (Subproblem construction): the manuscript states that the subproblems are designed so their feasible sets are disjoint 'by construction,' but does not supply a self-contained verification that this design preserves equivalence to the original basis-pursuit problem and does not inadvertently make the error-bound constant vanish for feasible (A,b) pairs.
minor comments (2)
- [§2] The phrase 'natural error bound' is used repeatedly before it is formally defined; a precise definition (with reference to the relevant distance or Hoffman-type constant) should appear in §2.
- [Numerical Experiments] Numerical section: the choice of the inconsistency parameter and the stopping tolerance for the inner alternating-projection loops should be stated explicitly to permit exact reproduction of the reported timings and iteration counts.
Simulated Author's Rebuttal
We thank the referee for the careful reading and constructive feedback on our manuscript. The comments help clarify the presentation of the convergence results and the subproblem design. We address each major comment below and indicate the revisions we will make.
read point-by-point responses
-
Referee: [§4 (Convergence Analysis)] §4 (Convergence Analysis), main theorem on linear rates: the linear convergence of both radii and (unique-solution) iterates is asserted to follow from a 'natural error bound' between the constructed feasible set and the optimal ℓ1-ball, yet no explicit positive lower bound on this constant is derived in terms of A, b or the current radius. Without such a bound the claimed uniform linear rate is not guaranteed and the central convergence result remains conditional on an unverified assumption.
Authors: The manuscript does not assert a uniform linear rate that is independent of the instance. Theorem 4.1 establishes linear convergence of the radii to the optimal value and, under uniqueness, linear convergence of the iterates, with the rate governed by the natural error-bound constant between each constructed feasible set and the optimal ℓ1-ball. This constant is strictly positive for any fixed feasible pair (A, b) possessing a unique solution, because the deliberate inconsistency in the subproblem construction ensures a positive separation that is preserved by the equivalence to basis pursuit. We will revise the statement of the main theorem and add a short remark after the proof to make explicit that positivity follows from uniqueness and the construction; we will also note that the resulting rate is necessarily instance-dependent. Deriving a closed-form, instance-independent lower bound on the constant is not feasible in general and is not claimed in the paper. revision: partial
-
Referee: [§3 (Subproblem construction)] §3 (Subproblem construction): the manuscript states that the subproblems are designed so their feasible sets are disjoint 'by construction,' but does not supply a self-contained verification that this design preserves equivalence to the original basis-pursuit problem and does not inadvertently make the error-bound constant vanish for feasible (A,b) pairs.
Authors: We agree that an explicit verification strengthens the exposition. In the revised manuscript we will insert a short proposition in §3 that (i) confirms the sequence of inconsistent subproblems is equivalent to the original basis-pursuit problem in the sense that the optimal value is recovered in the limit and (ii) shows that the enforced disjointness does not force the error-bound constant to zero for any feasible (A, b). The argument relies on the fact that the radii are monotonically nonincreasing and bounded below by the optimal ℓ1-norm, together with the uniqueness assumption guaranteeing a positive distance between the constructed sets and the optimal ball. revision: yes
- Supplying an explicit instance-independent positive lower bound on the governing error-bound constant (the referee's request for a uniform rate guarantee across all instances).
Circularity Check
Linear convergence rate governed by self-cited natural error bound
specific steps
-
self citation load bearing
[Abstract]
"Recently in [R. Behling, Y. Bello-Cruz and L.-R. Santos, SIAM J. Optim., 31 (2021), pp. 2863-2892], inconsistency coming from infeasibility has been shown to accelerate convergence of alternating projections. We deliberately enforce this inconsistency by constructing subproblems whose feasible sets are disjoint by design. We prove that the resulting ℓ1-radii converge linearly to the optimal value, and that when the solution is unique, all generated sequences converge linearly to it at a rate governed by a natural error bound between the feasible set and the optimal ℓ1-ball."
The linear rate for both radii and sequences is explicitly stated to be governed by the natural error bound, whose acceleration property and applicability are justified solely by citation to the authors' overlapping prior work rather than by an independent derivation or explicit lower bound on the constant within this manuscript.
full rationale
The paper's central convergence result for ℓ1-radii and iterates invokes a natural error bound between the feasible set and optimal ℓ1-ball, with the inconsistency-acceleration property imported directly from the authors' own 2021 SIAM J. Optim. paper. This creates moderate dependence on prior self-work for the rate analysis, even though the specific subproblem construction for basis pursuit is new and the manuscript supplies its own proof structure around the bound. No self-definitional reductions, fitted predictions, or ansatz smuggling appear in the given text; the derivation remains partially independent.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption A natural error bound exists between the feasible set and the optimal ℓ1-ball that controls the linear convergence rate.
- domain assumption Subproblems can be constructed whose feasible sets are disjoint by design.
Reference graph
Works this paper leans on
-
[1]
Dykstra’s Alternating Projection Algorithm for Two Sets
“Dykstra’s Alternating Projection Algorithm for Two Sets”.Journal of Approximation Theory, 79, 3, (Dec. 1994), 418–443. doi: 10.1006/jath.1994.1136. Heinz H Bauschke and Jonathan M Borwein
-
[2]
On Projection Algorithms for Solving Convex Feasibility Problems
“On Projection Algorithms for Solving Convex Feasibility Problems”.SIAM Review, 38, 3, 367–426. doi: 10.1137/S0036144593251710. Heinz H Bauschke and Jonathan M Borwein
-
[3]
On the Convergence of von Neumann’s Alternating Projection Algorithm for Two Sets
“On the Convergence of von Neumann’s Alternating Projection Algorithm for Two Sets”.Set-Valued Analysis, 1, 2, 185–212. doi: 10.1007/BF01027691. Heinz H. Bauschke and Patrick L. Combettes
-
[4]
Convex Analysis and Monotone Operator Theory in Hilbert Spaces . (2nd ed.). CMS Books in Mathematics. Springer International Publishing, Cham, Switzerland. isbn: 978-3-319-48310-8. doi: 10.1007/978-3-319-48311-5. Roger Behling, Yunier Bello-Cruz, Alfredo Noel Iusem, Ademir Alves Ribeiro, and Luiz-Rafael Santos. Oct
-
[5]
Fejér* Monotonicity in Optimization Algorithms. (Oct. 2024). arXiv: 2410.08331. doi: 10.48550/arXiv.2410.08331. preprint. Roger Behling, Yunier Bello-Cruz, and Luiz-Rafael Santos
work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2410.08331 2024
-
[6]
Infeasibility and Error Bound Imply Finite Convergence of Alternating Projections
“Infeasibility and Error Bound Imply Finite Convergence of Alternating Projections”. SIAM Journal on Optimization , 31, 4, 2863–2892. arXiv: 2008.03354. doi: 10.1137/20M1358669. Yunier Bello-Cruz, Guoyin Li, and Tran Thai An Nghia. July
-
[7]
Quadratic Growth Conditions and Uniqueness of Optimal Solution to Lasso
“Quadratic Growth Conditions and Uniqueness of Optimal Solution to Lasso”.Journal of Optimization Theory and Applications , 194, 1, (July 2022), 167–190. doi: 10.1007/s10957-022-02013-2. Jeff Bezanson, Alan Edelman, Stefan Karpinski, and Viral B Shah. Feb
-
[8]
“Julia: A Fresh Approach to Numerical Computing”.SIAM Review, 59, 1, (Feb. 2017), 65–98. doi: 10.1137/141000671. Alfred M. Bruckstein, David L. Donoho, and Michael Elad. Feb
-
[9]
From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images
“From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images”. SIAM Review, 51, 1, (Feb. 2009), 34–81. doi: 10.1137/060657704. Emmanuel J. Candès. May
-
[10]
The Restricted Isometry Property and Its Implications for Compressed Sensing
“The Restricted Isometry Property and Its Implications for Compressed Sensing”.Comptes Rendus Mathematique, 346, 9, (May 2008), 589–592. doi: 10.1016/j.crma.2008.03.014. Emmanuel J. Candès, Justin Romberg, and Terence Tao. Feb
-
[11]
“Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information”. IEEE Transactions on Information Theory , 52, 2, (Feb. 2006), 489–509. doi: 10.1109/TIT.2005.862083. Emmanuel J. Candès and Terence Tao. Dec
-
[12]
Decoding by Linear Programming
“Decoding by Linear Programming”. IEEE Transactions on Information Theory , 51, 12, (Dec. 2005), 4203–4215. doi: 10.1109/TIT.2005.858979. Scott Shaobing Chen, David L. Donoho, and Michael A. Saunders. Jan
-
[13]
Atomic Decomposition by Basis Pursuit
“Atomic Decomposition by Basis Pursuit”.SIAM Review, 43, 1, (Jan. 2001), 129–159. doi: 10.1137/S003614450037906X. 16 Roger Behling, Yunier Bello-Cruz, Luiz-Rafael Santos, and Paulo J. S. Silva Ward Cheney and Allen A. Goldstein
-
[14]
Proximity Maps for Convex Sets
“Proximity Maps for Convex Sets”.Proceedings of the American Mathematical Society , 10, 3, 448–450. JSTOR: 2032864. doi: 10.2307/2032864. P. L. Combettes and H. J. Trussell. Dec
-
[15]
Method of Successive Projections for Finding a Common Point of Sets in Metric Spaces
“Method of Successive Projections for Finding a Common Point of Sets in Metric Spaces”. Journal of Optimization Theory and Applications, 67, 3, (Dec. 1990), 487–507. doi: 10.1007/BF00939646. Patrick L. Combettes. Jan
-
[16]
by Dan Butnariu, Yair Censor, and Simeon Reich
Ed. by Dan Butnariu, Yair Censor, and Simeon Reich. Elsevier, (Jan. 2001), 115–152. doi: 10.1016/S1570-579X(01)80010-0. Roberto Cominetti, Walter F. Mascarenhas, and Paulo J. S. Silva. June
-
[17]
A Newton’s Method for the Continuous Quadratic Knapsack Problem
“A Newton’s Method for the Continuous Quadratic Knapsack Problem”. Mathematical Programming Computation, 6, 2, (June 2014), 151–169. doi: 10.1007/s12532-014-0066-y. Laurent Condat. July
-
[18]
Fast Projection onto the Simplex and the $\ell_{1}$-Ball
“Fast Projection onto the Simplex and the $\ell_{1}$-Ball”. Mathematical Programming, 158, 1, (July 2016), 575–585. doi: 10.1007/s10107-015-0946-6. Laurent Demanet and Xiangxiong Zhang. Jan
-
[19]
Eventual Linear Convergence of the Douglas-Rachford Iteration for Basis Pursuit
“Eventual Linear Convergence of the Douglas-Rachford Iteration for Basis Pursuit”.Mathematics of Computation, 85, 297, (Jan. 2016), 209–238. doi: 10.1090/mcom/2965. Monica Dessole, Marco Dell’Orto, and Fabio Marcuzzi
-
[20]
The Lawson-Hanson Algorithm with Deviation Maximization: Finite Convergence and Sparse Recovery
“The Lawson-Hanson Algorithm with Deviation Maximization: Finite Convergence and Sparse Recovery”. Numerical Linear Algebra with Applications , 30, 5, e2490. doi: 10.1002/nla.2490. Elisabeth D Dolan and Jorge J Moré
-
[21]
Yarin Gal, Riashat Islam, and Zoubin Ghahramani
“Benchmarking Optimization Software with Performance Profiles”.Mathematical Programming, 91, 2, 201–213. doi: 10.1007/s101070100263. D.L. Donoho. Apr
-
[22]
“Compressed Sensing”.IEEE Transactions on Information Theory , 52, 4, (Apr. 2006), 1289–1306. doi: 10.1109/TIT.2006.871582. Jerome H. Friedman, Trevor Hastie, and Rob Tibshirani. Feb. 2,
-
[23]
Regularization Paths for Generalized Linear Models via Coordinate Descent
“Regularization Paths for Generalized Linear Models via Coordinate Descent”. Journal of Statistical Software , 33, (Feb. 2, 2010), 1–22. Jean Charles Gilbert. Jan
work page 2010
-
[24]
On the Solution Uniqueness Characterization in the L1 Norm and Polyhedral Gauge Recovery
“On the Solution Uniqueness Characterization in the L1 Norm and Polyhedral Gauge Recovery”.Journal of Optimization Theory and Applications, 172, 1, (Jan. 2017), 70–101. doi: 10.1007/s10957-016-1004-0. Gene H. Golub and Charles F. Van Loan
-
[25]
Alternating Projections and Douglas-Rachford for Sparse Affine Feasibility
“Alternating Projections and Douglas-Rachford for Sparse Affine Feasibility”.IEEE Transactions on Signal Processing , 62, 18, (Sept. 2014), 4868–4881. doi: 10.1109/TSP.2014.2339801. Alan J. Hoffman
-
[26]
On Approximate Solutions of Systems of Linear Inequalities
“On Approximate Solutions of Systems of Linear Inequalities”. Journal of Research of the National Bureau of Standards , 49, 4, 263–265. doi: 10.6028/jres.049.027. Q. Huangfu and J. A. J. Hall. Mar
-
[27]
Parallelizing the Dual Revised Simplex Method
“Parallelizing the Dual Revised Simplex Method”.Mathematical Programming Computation, 10, 1, (Mar. 2018), 119–142. doi: 10.1007/s12532-017-0130-5. [SW], JuliaStats/Lasso.Jl June 6,
-
[28]
Accelerating Block Coordinate Descent Methods with Identification Strategies
“Accelerating Block Coordinate Descent Methods with Identification Strategies”. Computational Optimization and Applications, 72, 3, (Apr. 2019), 609–640. doi: 10.1007/s10589-018-00056-8. Dirk A. Lorenz. Mar
-
[29]
Constructing Test Instances for Basis Pursuit Denoising
“Constructing Test Instances for Basis Pursuit Denoising”.IEEE Transactions on Signal Processing , 61, 5, (Mar. 2013), 1210–1214. doi: 10.1109/TSP.2012.2236322. Dirk A. Lorenz, Marc E. Pfetsch, and Andreas M. Tillmann. Mar
-
[30]
An Infeasible-Point Subgradient Method Using Adaptive Approximate Projections
“An Infeasible-Point Subgradient Method Using Adaptive Approximate Projections”. Computational Optimization and Applications, 57, 2, (Mar. 2014), 271–306. doi: 10.1007/s10589-013-9602-3. Dirk A. Lorenz, Marc E. Pfetsch, and Andreas M. Tillmann. Feb
-
[31]
Solving Basis Pursuit: Heuristic Optimality Check and Solver Comparison
“Solving Basis Pursuit: Heuristic Optimality Check and Solver Comparison”.ACM Transactions on Mathematical Software, 41, 2, (Feb. 2015), 8:1–8:29. doi: 10.1145/2689662. Alexis Montoison and Dominique Orban. Sept
-
[32]
Krylov.Jl: A Julia Basket of Hand-Picked Krylov Methods
“Krylov.Jl: A Julia Basket of Hand-Picked Krylov Methods”.Journal of Open Source Software , 8, 89, (Sept. 2023),
work page 2023
-
[33]
Seyedahmad Mousavi and Jinglai Shen
doi: 10.21105/joss.05187. Seyedahmad Mousavi and Jinglai Shen
-
[34]
Lorenzo Stella, Niccolò Antonelo, and Matias Fält
doi: 10.1051/cocv/2018061. Lorenzo Stella, Niccolò Antonelo, and Matias Fält. June
- [35]
-
[36]
Regression Shrinkage and Selection via the Lasso
“Regression Shrinkage and Selection via the Lasso”.Journal of the Royal Statistical Society. Series B (Methodological) , 58, 1, 267–288. Retrieved Feb. 28, 2025 from JSTOR: 2346178. Andreas M. Tillmann and Marc E. Pfetsch. Feb
work page 2025
-
[37]
“The Computational Complexity of the Restricted Isometry Property, the Nullspace Property, and Related Concepts in Compressed Sensing”. IEEE Transactions on Information Theory , 60, 2, (Feb. 2014), 1248–1259. doi: 10.1109/TIT.2013.2290112. J.A. Tropp. Oct
-
[38]
Greed Is Good: Algorithmic Results for Sparse Approximation
“Greed Is Good: Algorithmic Results for Sparse Approximation”. IEEE Transactions on Information Theory , 50, 10, (Oct. 2004), 2231–2242. doi: 10.1109/TIT.2004.834793. Hui Zhang, Wotao Yin, and Lizhi Cheng. Jan
-
[39]
Necessary and Sufficient Conditions of Solution Uniqueness in 1-Norm Minimization
“Necessary and Sufficient Conditions of Solution Uniqueness in 1-Norm Minimization”.Journal of Optimization Theory and Applications, 164, 1, (Jan. 2015), 109–122. doi: 10.1007/s10957-014-0581-z
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.