REVIEW 3 major objections 6 minor 31 references
Computing Estimators of Dantzig Selector type via Column and Constraint Generation
T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read This paper shows that the Dantzig selector, basis pursuit, and fused Dantzig selector—three sparse-signal recovery problems normally too large for general LP solvers—can be solved at scales of $n\approx 10^3$ and $p\approx 10^6$ in…
desk verdict Useful large-scale Dantzig selector / basis pursuit solver, but the fused Dantzig section has a wrong inverse matrix that invalidates those results as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a reduced master LP coupled with pricing and separation. For the $\ell_1$-DS problem, the full LP is replaced by $\mathrm{DS}(I,J)$, which optimizes only $\beta_j$ for $j\in J$ while enforcing only the constraints indexed by $I$. After each solve, dual information gives reduced costs $|X_{*,j}^T\alpha|>1$, which identify columns to add, and primal residuals $|X_{*,i}^T r|>\lambda$, which identify violated constraints to add. The Lasso solution supplies the initial $I$ and $J$, and simplex warm starts carry basis information between subproblems. The same separation-and-pricing loop solves basis pursuit by column generation alone, and the fused Dantzig selector after a transformation that keeps the constraint matrix banded and makes each reduced-cost or constraint check cost $O(1)$.
What would settle it
Construct an $\ell_1$-DS instance with $n\approx 10^3$, $p\approx 10^6$ in which the true solution has about $p/2$ nonzero entries of comparable size, run DantzigLP with the Lasso warm start, and record total simplex iterations and runtime: if it still finishes in minutes the claim is stronger than the paper argues, while if it takes as long as or longer than solving the full LP, the paper's efficiency claim is limited to sparse solutions.
Extended reading notes
Core claim
The central claim is that column generation and constraint generation, run jointly inside a simplex LP solver and warmed up by a Lasso solution, produce accurate solutions to $\ell_1$-Dantzig selector problems at scales that commercial interior-point or first-order methods cannot reach: about $p\approx 10^6$ variables with $n\approx 10^3$ for the Dantzig selector, $p\approx 10^5$ for basis pursuit, and $n=p\approx 10^6$ for the fused Dantzig selector, all in minutes. The method relies on the optimal signal being sparse and on the active constraint set being small. The paper formulates a reduced LP over selected variable indices and selected constraint indices, checks dual reduced costs to add columns and primal violations to add constraints, and iterates until exact optimality conditions hold. It reports that the Lasso initialization is essential: random initialization, or column generation alone without constraint generation, loses most of the speed advantage.
Load-bearing premise
The whole speedup rests on the assumption that the Dantzig selector solution is sparse and that the Lasso solution already identifies most of its nonzero entries and active constraints; when that fails, column and constraint generation must add many pieces and the method slows toward solving the full LP.
Editorial extensions
If this is right
- The $\ell_1$-Dantzig selector becomes solvable in minutes at $n\approx 10^3$, $p\approx 10^6$, with memory requirements far below methods that form $X^TX$.
- Basis pursuit with $p\approx 10^5$ is solvable in minutes, while full-LP and ADMM baselines either exceed memory limits or fail to converge at that scale.
- The fused Dantzig selector in the signal-estimation case $X=I$ scales to $n=p\approx 10^6$, and the regression case with general $X$ improves on solving the full LP.
- Computing a whole path of solutions over a grid of $\lambda$ values is efficient because each solved subproblem warm-starts the next.
- The Lasso initialization, not column generation by itself, is the key to the speed: removing it or replacing it with random initialization degrades performance substantially.
Reading between the lines
- The same pricing-and-separation loop should extend to other LP-based sparse estimators, such as group or ordered Dantzig variants, whenever a cheap surrogate provides a reasonable initial support; this is a direct extrapolation of the paper's mechanism, not a claim it tests.
- The paper's own runtime table shows DantzigLP degrading sharply as $\lambda$ shrinks and the solution densifies; read literally, the practical scope of the method is bounded by solution sparsity, not simply by $n$ and $p$.
- Because the framework returns both primal and dual information at every iteration, a user could stop early with a certificate of suboptimality, trading exactness for large time savings on massive instances; the paper mentions the certificate but does not develop a stopping rule that uses it.
- The reported wall-clock times depend on a warm-start-friendly simplex engine, so the advantage is not inherent to column generation alone and may not transfer to interior-point or first-order subsolvers.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper develops DantzigLP, a column- and constraint-generation framework for solving large-scale linear-programming formulations of three sparse-estimation problems: the ℓ1-Dantzig selector, basis pursuit, and the fused Dantzig selector. The method solves reduced LPs over a subset of columns and constraints, checks optimality through dual violations and reduced costs, and warm-starts from solutions of related Lasso-type problems. The authors report experiments on synthetic and two real datasets showing large speedups over ADMM implementations, the parametric simplex method (PSM), and direct LP solves with Gurobi, and they provide a Julia implementation.
Significance. If the results hold, the paper addresses a real gap: LP-based Dantzig-type estimators are statistically attractive but are rarely used at scale because general-purpose LP solvers struggle with p in the millions. The idea of using classical column and constraint generation with a simplex solver and a Lasso initialization is clean and well motivated, and the pseudocode is precise enough to reimplement. The ℓ1-Dantzig selector and basis pursuit experiments support the central efficiency claim for sparse regimes, and the code release is a concrete strength. However, the fused Dantzig selector section contains a mathematical error in the claimed inverse matrix H, so the fused results in Tables 6 and 7 are not supported as written; in addition, the introduction's "few minutes" scalability claim is inconsistent with the 3,702-second Boston1M result in Table 4.
major comments (3)
- [§2.4, Eq. (14)] The matrix H defined by H_{i,j}=1 if j=1, H_{i,j}=i-j if i>j, and 0 otherwise is not the inverse of D=[e_1^T; D^(0)]. For p=4, the fourth column of H is zero and (DH)_{2,4}=0, so DH is not the identity; the correct inverse is the lower-triangular matrix of ones. Because this H is used in Eqs. (15)-(17), the derivation of the fused Dantzig selector LP is not valid: the constraints g_A=0 and g_B in [-lambda, lambda] in Eq. (17) do not represent H_A^T r=0 and ||H_B^T r||_inf <= lambda from Eq. (16). The regression reduction in §2.4.2 inherits the same issue, and Tables 6 and 7 therefore do not demonstrate solutions of (7) or (16) as written. Please correct H, rederive the reformulation, and either confirm that the reported experiments used the corrected formulation or regenerate the fused Dantzig experiments.
- [Introduction and Table 4] The introduction claims that DantzigLP can solve instances of the ℓ1-DS problem with n≈10^3 and p≈10^6 "within a few minutes," but the only p≈10^6 experiment reported, Boston1M in Table 4 with n=200, takes 3,702 seconds (about 62 minutes), and no synthetic instance with p≈10^6 is shown. As written, this scalability claim is not supported by the evidence. Please either temper the claim, distinguish the regimes in which the stated runtimes are observed, or provide an experiment with n≈10^3 and p≈10^6 that completes in the claimed time.
- [§2.2 and Table 3] The efficiency of DantzigLP depends on the premise that the Dantzig selector solution is sparse and close to the Lasso support. Table 3 shows this dependence clearly: DantzigLP (Path) rises from 13.6 seconds at tau=1.0 to 209 seconds at tau=0.1 as the solution becomes denser, and the text acknowledges that DantzigLP can be slower than the full LP for small p. Because this is a central boundary condition on the method, the abstract and conclusion should state the sparsity regime explicitly rather than relying on the phrase "in many settings."
minor comments (6)
- [Tables 1–3, 5, 7] Runtimes are reported only as means over replications; given the variability typical of simplex-based and column/constraint-generation methods, reporting standard deviations or median/interquartile ranges would make the comparisons substantially more informative.
- [§2.1] The sentence "Combining column and constraint generation LP techniques with methods for finding good initializations for the initial column and constraint sets I and J, we develop DantzigLP, a general framework for solving large-scale versions of the the Dantzig Selector-type problems" contains a duplicated "the."
- [§2.4] The heading "F used Dantzig Selector" contains an extra space and should be "Fused Dantzig Selector."
- [§3.3] The sentence "the initial solution obtained from the Lasso problem (5)" should refer to the fused Lasso problem (5) rather than the Lasso problem (3), to avoid confusion between the two regularizers.
- [Algorithms 2–3 and §2.2] The pseudocode in Algorithms 2 and 3 uses exact violation checks, while Section 2.2 defines a relaxed version with tolerance epsilon=10^{-4} that is used in the experiments; the pseudocode should incorporate this tolerance to match the implementation.
- [Table 4] The DantzigLP runtimes include the Lasso initialization, but the table reports only the combined time; decomposing the runtime into initialization versus column/constraint generation would clarify where the 3,702 seconds for Boston1M is spent.
Circularity Check
No circularity: the Lasso warm start is an initialization, not a fitted prediction, and all LP solutions are checked against full optimality conditions.
full rationale
The paper's central derivation—column and constraint generation applied to LP formulations of l1-DS, Basis Pursuit, and the Fused Dantzig Selector—does not reduce to its inputs. The Lasso solution is used only to initialize the index sets I and J (Algorithm 2) and the Basis Pursuit starting support (Algorithm 3); every reported solution is accepted only after the full optimality checks Iv=∅ and Jv=∅ (Section 2.1 and eq. (12)). Thus the Lasso is a warm start, not a fitted parameter disguised as a prediction, and the runtime claims are measured outcomes (Tables 1–7), not consequences of a definitional identity. The only self-citation, [12] (the authors' prior SVM column/constraint-generation paper), is contextual and non-load-bearing: the DS LP formulations, optimality conditions, and numerical comparisons with Gurobi, PSM, and ADMM stand independently. The cited Lasso–DS path equivalence results [20] and [1] are external works. Section 2.2's discussion that efficiency depends on sparsity and on the Lasso's similarity to DS is a stated assumption about computational performance, not a circular derivation; Table 3 even shows the method slowing when the assumption weakens. The suspicious H=D^{-1} formula in Section 2.4 (which would make DH≠I) is a mathematical correctness issue that may undermine Tables 6–7, but it is not a circularity, so it is not scored here. Overall: no significant circularity.
Assumptions & free parameters
free parameters (3)
- Violation tolerance epsilon =
1e-4
- Max columns added per iteration =
30
- Max constraints added per iteration =
50
assumptions (3)
- standard math LP duality and simplex optimality conditions
- domain assumption Lasso solution provides a good approximation to the Dantzig selector support and active constraint set
- domain assumption The optimal Dantzig selector solution is sparse
Cite this review
Pith. "Pith review of Computing Estimators of Dantzig Selector type via Column and Constraint Generation." pith.science (2026). https://pith.science/paper/BA2QSUYJ
@misc{pith2026190806515,
author = {Pith},
title = {Pith review of: Computing Estimators of Dantzig Selector type via Column and Constraint Generation},
year = {2026},
howpublished = {\url{https://pith.science/paper/BA2QSUYJ}},
note = {Machine review of arXiv:1908.06515}
}
read the original abstract
We consider a class of linear-programming based estimators in reconstructing a sparse signal from linear measurements. Specific formulations of the reconstruction problem considered here include Dantzig selector, basis pursuit (for the case in which the measurements contain no errors), and the fused Dantzig selector (for the case in which the underlying signal is piecewise constant). In spite of being estimators central to sparse signal processing and machine learning, solving these linear programming problems for large scale instances remains a challenging task, thereby limiting their usage in practice. We show that classic constraint- and column-generation techniques from large scale linear programming, when used in conjunction with a commercial implementation of the simplex method, and initialized with the solution from a closely-related Lasso formulation, yields solutions with high efficiency in many settings.
Figures
Reference graph
Works this paper leans on
-
[1]
M. S. Asif and J. Romberg. On the lasso and dantzig selector equivalence. In 2010 44th Annual Conference on Information Sciences and Systems (CISS) , pages 1–6, 3 2010
work page 2010
-
[2]
M. S. Asif and J. Romberg. Dantzig selector homotopy with dynamic measurements. Proc. SPIE 7246, Computational Imaging VII, 72460E , 7246, 2009
work page 2009
-
[3]
Beck and M
A. Beck and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences , 2(1):183–202, 2009
2009
-
[4]
S. R. Becker, E. J. Cand` es, and M. C. Grant. Templates for convex cone problems with applications to sparse signal recovery. Mathematical Programming Computation, 3(3):165, 7 2011
work page 2011
-
[5]
D. Bertsimas and J. N. Tsitsiklis. Introduction to Linear Optimization . Athena, 1997
work page 1997
-
[6]
P. J. Bickel, Y. Ritov, and A. B. Tsybakov. Simultaneous analysis of lasso and dantzig selector. The Annals of Statistics , 37(4):1705–1732, 2009
work page 2009
-
[7]
S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein. Distributed optimization and statisti- cal learning via the alternating direction method of multipliers. Found. Trends Mach. Learn., 3(1):1–122, January 2011
work page 2011
- [8]
Show all 31 references
-
[9]
Candes and T
E. Candes and T. Tao. The dantzig selector: Statistical estimation when p is much larger than n. Ann. Statist., 35(6):2313–2351, 12 2006
2006
-
[10]
Chen and D
S. Chen and D. Donoho. Basis pursuit. In Proceedings of 1994 28th Asilomar Conference on Signals, Systems and Computers , volume 1, pages 41–44 vol.1, 10 1994
1994
-
[11]
G. B. Dantzig and P. Wolfe. Decomposition principle for linear programs. Operations research, 8(1):101–111, 1960
1960
-
[12]
Dedieu and R
A. Dedieu and R. Mazumder. Solving large-scale l1-regularized svms and cousins: the sur- prising effectiveness of column and constraint generation. arXiv preprint arXiv:1901.01585 , 2019
1901 arXiv
-
[13]
D. L. Donoho, A. Maleki, and A. Montanari. Message-passing algorithms for compressed sensing. Proceedings of the National Academy of Sciences , 106(45):18914–18919, 2009
2009
-
[14]
Efron, T
B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani. Least angle regression. The Annals of statistics, 32(2):407–499, 2004
2004
-
[15]
Efron, T
B. Efron, T. Hastie, and R. Tibshirani. Discussion: The dantzig selector: Statistical estimation when p is much larger than n. The Annals of Statistics , 35(6):2358–2364, 2007. 21
2007
-
[16]
L. R. Ford Jr and D. R. Fulkerson. A suggested computation for maximal multi-commodity network flows. Management Science, 5(1):97–101, 1958
1958
-
[17]
Friedman, T
J. Friedman, T. Hastie, and R. Tibshirani. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software , 33(1):1–22, 2010
2010
-
[18]
Harrison and D
D. Harrison and D. L. Rubinfeld. Hedonic housing prices and the demand for clean air. Journal of Environmental Economics and Management , 5(1):81 – 102, 1978
1978
-
[19]
Hazimeh and R
H. Hazimeh and R. Mazumder. Fast Best Subset Selection: Coordinate Descent and Local Combinatorial Optimization Algorithms. ArXiv e-prints, March 2018
2018
-
[20]
G. M. James, P. Radchenko, and J. Lv. Dasso: connections between the dantzig selector and lasso. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 71(1): 127–142, 2009
2009
-
[21]
N. A. Johnson. A dynamic programming algorithm for the fused lasso and l 0-segmentation. Journal of Computational and Graphical Statistics , 22(2):246–260, 2013
2013
-
[22]
X. Li, T. Zhao, X. Yuan, and H. Liu. The flare package for high dimensional linear regression and precision matrix estimation in r. Journal of Machine Learning Research, 16:553–557, 2015
2015
-
[23]
Z. Lu, T. K. Pong, and Y. Zhang. An alternating direction method for finding dantzig selectors. Computational Statistics & Data Analysis , 56(12):4037 – 4046, 2012
2012
-
[24]
Mammen and S
E. Mammen and S. Geer. Locally adaptive regression splines. The Annals of Statistics , 25(1): 387–413, 1997
1997
-
[25]
H. Pang, H. Liu, and R. Vanderbei. The fastclime package for linear programming and large- scale precision matrix estimation in r. Journal of Machine Learning Research , 15(1):489–493, January 2014
2014
-
[26]
H. Pang, H. Liu, R. J. Vanderbei, and T. Zhao. Parametric simplex method for sparse learning. In Advances in Neural Information Processing Systems , pages 188–197, 2017
2017
-
[27]
L. I. Rudin, S. Osher, and E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena , 60(1):259 – 268, 1992
1992
-
[28]
Tibshirani
R. Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological) , 58(1):267–288, 1996
1996
-
[29]
Tibshirani, M
R. Tibshirani, M. Saunders, S. Rosset, J. Zhu, and K. Knight. Sparsity and smoothness via the fused lasso. Journal of the Royal Statistical Society Series B , pages 91–108, 2005
2005
-
[30]
Wang and X
X. Wang and X. Yuan. The linearized alternating direction method of multipliers for dantzig selector. SIAM Journal on Scientific Computing , 34(5):A2792–A2811, 2012
2012
-
[31]
S. J. Wright, R. D. Nowak, and M. A. Figueiredo. Sparse reconstruction by separable approx- imation. IEEE Transactions on Signal Processing , 57(7):2479–2493, 2009. 22
2009
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.