REVIEW 2 major objections 2 minor 20 references
Fast Robust Regression via Orthogonal Block Updates
T0 review · 2 major / 2 minor · reviewed 2026-06-26 · grok-4.3
Pith's one-line read The ROBU algorithm allows scalable computation of robust MM-estimators by partitioning predictors into lower-dimensional blocks that require fewer subsamples.
desk verdict ROBU tries to scale random-subset initialization for MM-estimators by splitting predictors into blocks, but the aggregation step lacks an obvious guarantee that the combined initial estimator stays clean with high probability. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The Robust Orthogonal Block Updates (ROBU) algorithm, which performs block-coordinate descent on partitions of the explanatory variables to obtain initial regression estimators.
What would settle it
If experiments show that for a given number of predictors the ROBU initial point does not yield MM-estimates with breakdown points or efficiencies matching those from the standard subsampling method.
Extended reading notes
Core claim
The ROBU algorithm uses lower-dimensional blocks of explanatory variables, which require much fewer sub-samples to find a good initial point with high probability. This makes it possible to compute MM-estimators that are robust to heavy-tailed noise and high-leverage outliers even when the number of predictors is moderate to large.
Load-bearing premise
That solutions from the individual lower-dimensional blocks can be combined into an initial estimator that allows the MM-algorithm to reach competitive robust estimates.
Editorial extensions
If this is right
- Robust regression becomes feasible for high-dimensional models without prohibitive computation.
- The initial estimators from blocks lead to MM-estimates with good robustness properties.
- Performance is competitive with existing algorithms in both simulated and real data.
- The method scales better as the number of variables increases.
Reading between the lines
- The block approach might be combined with other initial point strategies for further gains.
- It could be tested on datasets with thousands of predictors to see the scaling benefits.
- Applications to generalized linear models or other robust procedures may be possible.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes the ROBU (Robust Orthogonal Block Updates) algorithm, a block-coordinate descent method for computing MM-estimators in robust linear regression. It partitions the p predictors into lower-dimensional blocks so that the number of random subsets required to obtain a high-probability clean initial estimator grows with block size rather than full p; block solutions are then combined via orthogonal updates to produce a scalable initial point for the subsequent MM iteration. The manuscript reports simulation studies and a proteogenomics application showing competitive robustness and speed relative to existing methods.
Significance. If the per-block initialization and aggregation step preserve the claimed high-probability robustness, the method would remove a major computational barrier to MM-estimation when p is moderate to large, extending the practical reach of robust regression beyond what random-subset or non-stochastic initializers currently allow. The reported simulations and real-data example provide empirical support for the scalability claim.
major comments (2)
- [§3.2] §3.2 (Algorithm 1 and surrounding text): the manuscript states that block-wise clean-subset searches require far fewer trials than the full-p search, yet supplies no explicit joint-probability argument showing that the concatenated or orthogonally projected initial estimator remains outlier-free with probability that scales with block dimension rather than p. Different blocks may identify different clean observation sets; an observation that is clean within its block but contaminated for variables outside the block can still bias the combined β̂₀. Without a union-bound or intersection argument, the central scalability claim rests on an unproven aggregation step.
- [§4] §4 (Simulation design): the reported contamination scenarios and performance metrics compare ROBU-initialized MM-estimators to competitors, but do not include a controlled ablation that isolates the effect of the block-aggregation rule on the probability of obtaining a clean initial point. Consequently it is unclear whether the observed robustness gains are attributable to the block construction or to other implementation choices.
minor comments (2)
- Notation for the block partition and the orthogonal projection operator is introduced without a compact reference table; a small display equation summarizing the update rule would improve readability.
- The abstract claims the method 'compare[s] favourably' to existing algorithms; the simulation tables should report both point estimates and variability (e.g., standard errors across replications) so that the reader can judge whether differences are statistically meaningful.
Simulated Author's Rebuttal
We thank the referee for the constructive comments on our manuscript. We address each major comment below and indicate the revisions we intend to make.
read point-by-point responses
-
Referee: [§3.2] §3.2 (Algorithm 1 and surrounding text): the manuscript states that block-wise clean-subset searches require far fewer trials than the full-p search, yet supplies no explicit joint-probability argument showing that the concatenated or orthogonally projected initial estimator remains outlier-free with probability that scales with block dimension rather than p. Different blocks may identify different clean observation sets; an observation that is clean within its block but contaminated for variables outside the block can still bias the combined β̂₀. Without a union-bound or intersection argument, the central scalability claim rests on an unproven aggregation step.
Authors: We agree that the manuscript does not supply a formal joint-probability argument (via union bound or otherwise) establishing that the aggregated initial estimator remains clean with probability governed by block size. The current text motivates the approach via the per-block reduction in subset-search cost and describes the orthogonal updates as a mechanism for combining block solutions, but it stops short of proving that cross-block contamination is controlled. We will revise §3.2 to state this limitation explicitly, to clarify the assumptions under which the aggregation is expected to preserve robustness, and to indicate that a rigorous proof of the overall success probability remains open. This constitutes a partial revision. revision: partial
-
Referee: [§4] §4 (Simulation design): the reported contamination scenarios and performance metrics compare ROBU-initialized MM-estimators to competitors, but do not include a controlled ablation that isolates the effect of the block-aggregation rule on the probability of obtaining a clean initial point. Consequently it is unclear whether the observed robustness gains are attributable to the block construction or to other implementation choices.
Authors: The referee correctly notes the absence of a targeted ablation isolating the block-aggregation rule. While the existing simulations demonstrate end-to-end performance, they do not separate the contribution of the orthogonal aggregation step from other algorithmic choices. We will add a controlled ablation experiment in the revised §4 that compares the full ROBU procedure against a block-wise initialization variant that omits the orthogonal updates, thereby clarifying the source of the observed robustness gains. revision: yes
Circularity Check
No circularity: ROBU presented as independent algorithmic construction
full rationale
The paper introduces ROBU as a novel block-coordinate descent procedure that partitions predictors into lower-dimensional blocks to reduce the number of random subsets needed for a clean initial estimator. No equation, theorem, or performance claim is shown to reduce by construction to a fitted quantity, self-citation chain, or renamed input; the method is offered as a new scalable construction whose properties are to be verified by simulation and application. The aggregation step is described as part of the algorithm rather than derived from prior results that would create circularity.
Assumptions & free parameters
Cite this review
Pith. "Pith review of Fast Robust Regression via Orthogonal Block Updates." pith.science (2026). https://pith.science/paper/2EJX42SR
@misc{pith2026260626611,
author = {Pith},
title = {Pith review of: Fast Robust Regression via Orthogonal Block Updates},
year = {2026},
howpublished = {\url{https://pith.science/paper/2EJX42SR}},
note = {Machine review of arXiv:2606.26611}
}
read the original abstract
Robust regression methods, particularly MM-estimators, are essential for analyzing datasets where heavy-tailed noise or high-leverage outliers may be present. Algorithms to compute these estimators are iterative and rely on having a good initial point. A widely-used probabilistic approach to obtaining an initial regression estimator that is not affected by outliers consists of fitting linear regression models to many random subsets of the training data. Unfortunately, the number of subsets that need to be considered in order to have a high probability of finding one that is clean of outliers grows exponentially with the number of predictors. This renders the approach unfeasible for models with a moderate to large numbers of variables. Alternative non-stochastic strategies that have been proposed recently also fail to scale well when the number of parameters and observations are large. To overcome this problem, we propose a highly scalable algorithm based on block-coordinate descent. Our Robust Orthogonal Block Updates (ROBU) algorithm uses lower-dimensional blocks of explanatory variables, which require much fewer sub-samples to find a good initial point with high probability. Extensive simulations and a proteogenomics data application illustrate the robustness properties of the estimators computed with ROBU and demonstrate that they compare favourably to those calculated with existing algorithms.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
V., M¨ uller, S., Schoenfeld, D., Song, J., Wang, J., Willett, W., and Zviran, A
Cohen Freue, G. V., M¨ uller, S., Schoenfeld, D., Song, J., Wang, J., Willett, W., and Zviran, A. (2019). Robust elastic net estimators for variable selection and identification of proteomic biomarkers.The Annals of Applied Statistics, 13(4):2455–2479
2019
-
[2]
(2014).An introduction to mvnfast
Fasiolo, M. (2014).An introduction to mvnfast. R package version 0.2.8.University of Bristol
2014
-
[3]
Golub, G. H. and Van Loan, C. F. (1996).Matrix Computations. The Johns Hopkins University
1996
-
[4]
J., and Verdonck, T
Hubert, M., Rousseeuw, P. J., and Verdonck, T. (2012). A deterministic algorithm for robust location and scatter.Journal of Computational and Graphical Statistics, 21(3):618–637
2012
-
[5]
Kepplinger, D. (2023). Robust prediction and protein selection with adaptive pense.Journal of Computational Biology, 30(5):555–570
2023
-
[6]
Maechler, M., Rousseeuw, P., Croux, C., Todorov, V., Ruckstuhl, A., Salibian-Barrera, M., Verbeke, T., Koller, M., Conceicao, E. L. T., and di Palma, M. A. (2025).robustbase: Basic Robust Statistics. R package version 0.99-6
2025
-
[7]
A., Martin, R
Maronna, R. A., Martin, R. D., Yohai, V. J., and Salibi´ an-Barrera, M. (2019).Robust Statistics: Theory and Methods (with R). John Wiley & Sons, Hoboken, NJ, 2nd edition
2019
-
[8]
Nesterov, Y. (2012). Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM Journal on Optimization, 22(2):341–362
2012
Show all 20 references
-
[9]
Pan, V. Y. and Chen, Z. Q. (1999). The complexity of the matrix eigenproblem. InProceedings of the Thirty-First Annual ACM Symposium on Theory of Computing, STOC ’99, page 507–516, New York, NY, USA. Association for Computing Machinery. Pe˜ na, D. and Yohai, V. (1999). A fast ...
1999
-
[10]
Ramos, M. et al. (2020). Multiomic integration of public oncology databases in Bioconductor.JCO Clinical Cancer Informatics, 1(4):958–971
2020
-
[11]
Rousseeuw, P. J. (1984). Least median of squares regression.Journal of the American Statistical Association, 79(388):871–880
1984
-
[12]
Rousseeuw, P. J. and van Zomeren, B. C. (1990). Unmasking multivariate outliers and leverage points.Journal of the American Statistical Association, 85(411):633–639
1990
-
[13]
Rousseeuw, P. J. and Yohai, V. J. (1984). Robust regression by means of S-estimators. InRobust and nonlinear time series analysis, pages 256–272. Springer, New York
1984
-
[14]
and Yohai, V
Salibian-Barrera, M. and Yohai, V. J. (2006). A fast algorithm for S-regression estimates.Journal of Computational and Graphical Statistics, 15(2):414–427
2006
-
[15]
Tseng, P. (2001). Convergence of a block coordinate descent method for nondifferentiable mini- mization.Journal of Optimization Theory and Applications, 109:475—-494
2001
-
[16]
and Marcotte, E
Vogel, C. and Marcotte, E. M. (2012). Insights into the regulation of protein abundance from proteomic and transcriptomic analyses.Nature Reviews Genetics, 13(4):227–232
2012
-
[17]
Weinstein, J. N. et al. (2013). The Cancer Genome Atlas pan-cancer analysis project.Nature Genetics, 45(10):1113–1120. 22
2013
-
[18]
Wright, S. J. (2015). Coordinate descent algorithms.Mathematical Programming, 151(1):3–34
2015
-
[19]
Yohai, V., Maronna, R., Martin, D., Brownson, G., Konis, K., and Salibian-Barrera, M. (2024). RobStatTM: Robust Statistics: Theory and Methods. R package version 1.0.11
2024
-
[20]
Yohai, V. J. (1987). High breakdown-point and high efficiency robust estimates for regression.The Annals of Statistics, 15(2):642–656. 23
1987
Reviewed June 26, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.