Reservoir Zero-Coordinatewise Projected Subspace Search for Minimization Over Sparse Symmetric Sets in Machine Learning

Michael Breuss; Morteza Kimiaei; Shima Shabani

arxiv: 2606.28124 · v1 · pith:Y6Z74ZJMnew · submitted 2026-06-26 · 🧮 math.OC

Reservoir Zero-Coordinatewise Projected Subspace Search for Minimization Over Sparse Symmetric Sets in Machine Learning

Morteza Kimiaei , Shima Shabani , Michael Breuss This is my paper

Pith reviewed 2026-06-29 03:01 UTC · model grok-4.3

classification 🧮 math.OC

keywords sparse optimizationcardinality constraintsnonconvex optimizationreservoir methodcoordinatewise searchBeck-Hallak stationaritysubspace searchmachine learning

0 comments

The pith

The RZCW-PSS algorithm produces accumulation points that are Beck-Hallak zero-coordinatewise stationary almost surely under regularity assumptions.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops the Reservoir Zero-Coordinatewise Projected Subspace Search algorithm to solve nonconvex cardinality-constrained optimization problems that arise in sparse machine learning. It augments coordinatewise moves and symmetry-aware swaps with randomized low-dimensional subspace searches drawn from a maintained reservoir of prior feasible points, plus an optional safeguard for full-support stabilization. The central result establishes that every full-support accumulation point of the iterates satisfies Beck-Hallak zero-coordinatewise stationarity almost surely when the regularity, sampling, and subproblem-accuracy conditions hold; the safeguard and full-support start extend the guarantee to every accumulation point. The work also derives a conditional local linear convergence rate once the support stabilizes, along with the associated logarithmic iteration complexity. These properties matter because they supply convergence assurances for a class of NP-hard sparse problems without requiring exhaustive enumeration of supports.

Core claim

We introduce a Reservoir Zero-Coordinatewise Projected Subspace Search (RZCW-PSS) algorithm, a simplex-style method on sparse manifolds that integrates coordinatewise search, symmetry-aware swap-based support updates, randomized low-dimensional subspace exploration, and zero-coordinatewise reservoir injection. Under the stated regularity, sampling, and subproblem-accuracy assumptions, every full-support accumulation point of the RZCW-PSS iterates is Beck--Hallak zero-coordinatewise stationary almost surely; with the safeguard and full-support initialization, this conclusion applies to all accumulation points. We further prove a conditional local linear convergence rate after support stabiliz

What carries the argument

The RZCW-PSS algorithm, which maintains a reservoir of accepted feasible points to construct sparse-compatible subspace searches while targeting Beck-Hallak zero-coordinatewise stationarity of accumulation points.

If this is right

Every full-support accumulation point of the RZCW-PSS iterates is Beck-Hallak zero-coordinatewise stationary almost surely.
With the support-identification safeguard and full-support initialization, every accumulation point satisfies Beck-Hallak zero-coordinatewise stationarity.
After support stabilization the method achieves conditional local linear convergence.
The local iteration complexity after stabilization is logarithmic.
On synthetic sparse learning problems the method improves robustness and solution quality relative to Partial Simplex Search, Basic Feasible Search, and Zero-Coordinatewise Search.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The reservoir construction could be adapted to maintain diversity on other discrete feasible sets arising in combinatorial machine learning tasks.
Beck-Hallak zero-coordinatewise stationarity may provide a useful practical termination test for coordinate-based sparse solvers even when global optimality is out of reach.
Symmetry-aware swap updates might extend naturally to problems whose feasible sets are invariant under additional group actions beyond the current symmetric setting.

Load-bearing premise

The regularity, sampling, and subproblem-accuracy assumptions invoked to establish Beck-Hallak zero-coordinatewise stationarity of accumulation points hold for the given problem and iterates.

What would settle it

A concrete problem instance satisfying all stated assumptions in which some full-support accumulation point of the RZCW-PSS iterates fails to be Beck-Hallak zero-coordinatewise stationary.

Figures

Figures reproduced from arXiv: 2606.28124 by Michael Breuss, Morteza Kimiaei, Shima Shabani.

**Figure 1.** Figure 1: Logical roadmap of the convergence analysis. Fixed-support regime Step S1g + full support Lemma 6 Restricted regularity Assumptions 4, 5 Candidate and inexactness Assumption 6 Assumption 3(ii) Inexact restricted descent Proposition 2 Local linear rate Theorem 4 [PITH_FULL_IMAGE:figures/full_fig_p020_1.png] view at source ↗

**Figure 2.** Figure 2: Logical roadmap of the local convergence-rate analysis. To clarify the logical structure of the local convergence-rate analysis, Figure 2 summarizes the dependencies among the fixed-support regime, the restricted regularity assumptions, the candidate/inexactness conditions, and the descent estimate leading to Theorem 4. 4.1.1. Problem Setting and Basic Assumptions. Assumption 1. Let Ω = C ∩ Cs. The suble… view at source ↗

**Figure 3.** Figure 3: Performance profiles {ρsi (τ )} 2 i=1 of the solvers {si} 2 i=1 = {RZCW-PSS-qr, ZCWS} in terms of nf2g (left) and sec (right), using the objective-quality criterion qsol ≤ ε = 10−4 . A more detailed analysis of solution quality is provided in [PITH_FULL_IMAGE:figures/full_fig_p038_3.png] view at source ↗

**Figure 4.** Figure 4: Comparison of ZCWS and RZCW-PSS-qr under cardinality constraints. (Left) Final support size achieved by each method. (Middle) Matched-problem comparison of the final objective value versus support size, where each connected pair corresponds to the same problem instance; RZCW-PSS-qr consistently attains lower objective values, often with different but structurally related supports. (Right) Relaxed support … view at source ↗

read the original abstract

We study a class of nonconvex cardinality-constrained optimization problems arising in sparse learning. These problems are NP-hard due to the combinatorial nature of sparsity constraints. We introduce a Reservoir Zero-Coordinatewise Projected Subspace Search (RZCW-PSS) algorithm, a simplex-style method on sparse manifolds that integrates coordinatewise search, symmetry-aware swap-based support updates, randomized low-dimensional subspace exploration, and zero-coordinatewise reservoir injection. The proposed method augments classical coordinate and swap moves with sparse-compatible subspace searches constructed from a dynamically maintained reservoir of previously accepted feasible points. A key feature of the approach is a refined reservoir initialization strategy that embeds sparse projection directly into a uniform sampling procedure, preserving geometric diversity within the feasible set. The algorithm also includes an optional support-identification safeguard that enforces full-support stabilization under a fixed support-change decrease threshold. We establish that, under the stated regularity, sampling, and subproblem-accuracy assumptions, every full-support accumulation point of the RZCW-PSS iterates is Beck--Hallak zero-coordinatewise stationary almost surely; with the safeguard and full-support initialization, this conclusion applies to all accumulation points. We further prove a conditional local linear convergence rate after support stabilization and derive the corresponding logarithmic local iteration complexity. Numerical experiments on synthetic sparse learning problems demonstrate that RZCW-PSS improves robustness and solution quality while remaining computationally competitive with Partial Simplex Search, Basic Feasible Search, and Zero-Coordinatewise Search methods.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

RZCW-PSS is a new simplex-style method that adds reservoir-based subspace search and symmetry swaps to prior coordinatewise approaches, with stationarity guarantees under standard assumptions.

read the letter

The paper introduces RZCW-PSS, which combines a maintained reservoir of feasible points with randomized low-dimensional subspace exploration, symmetry-aware support swaps, and zero-coordinatewise moves for cardinality-constrained nonconvex problems. The main addition is the reservoir initialization that embeds sparse projection into uniform sampling, plus an optional safeguard for full-support stabilization. It proves that full-support accumulation points are Beck-Hallak zero-coordinatewise stationary almost surely under regularity, sampling, and subproblem-accuracy assumptions, and gives a conditional local linear rate after stabilization.

This is genuinely new relative to the cited Partial Simplex Search, Basic Feasible Search, and Zero-Coordinatewise Search methods; those lack the reservoir-subspace integration and the stated stationarity result. The numerical section on synthetic sparse learning instances shows improved robustness and solution quality while staying competitive in runtime.

The soft spots are the assumptions themselves, which are not shown to be especially mild, and the fact that all experiments are synthetic. Real data behavior and sensitivity to the subproblem solver accuracy are not addressed. The analysis is standard iterative convergence work rather than a parameter-free or fully reproducible derivation.

This is for people working on sparse manifold optimization and cardinality-constrained ML. It is coherent on its own terms and deserves a serious referee, even if revisions will be needed on the assumption strength and experimental scope.

Referee Report

0 major / 3 minor

Summary. The manuscript introduces the Reservoir Zero-Coordinatewise Projected Subspace Search (RZCW-PSS) algorithm for nonconvex cardinality-constrained optimization arising in sparse learning. The method augments coordinatewise and swap moves with randomized low-dimensional subspace searches drawn from a dynamically maintained reservoir of feasible points, together with an optional support-identification safeguard. Under stated regularity, sampling, and subproblem-accuracy assumptions, every full-support accumulation point is shown to be Beck--Hallak zero-coordinatewise stationary almost surely; with the safeguard and full-support initialization the conclusion extends to all accumulation points. A conditional local linear rate after support stabilization is also derived, together with the corresponding logarithmic iteration complexity. Numerical experiments on synthetic sparse learning problems report improved robustness and solution quality relative to Partial Simplex Search, Basic Feasible Search, and Zero-Coordinatewise Search.

Significance. If the stated convergence result holds, the work supplies a new algorithmic framework with stationarity guarantees for a practically relevant class of NP-hard sparse optimization problems. The reservoir-plus-subspace construction and the safeguard mechanism appear to be the primary technical contributions; the local linear-rate analysis after stabilization is a standard but useful addition. The numerical comparisons, while limited to synthetic instances, indicate practical competitiveness.

minor comments (3)

[Abstract / §1] The abstract and introduction invoke Beck--Hallak zero-coordinatewise stationarity without a self-contained definition or pointer to the precise stationarity condition used in the analysis; adding a short displayed definition in §2 would improve readability.
[Numerical experiments] The numerical section reports improvements but does not specify the precise dimensions of the synthetic instances, the distribution used to generate the data, or the termination criteria applied to all compared methods; these details are needed to assess reproducibility and fairness of the comparison.
[Algorithm description] The reservoir initialization procedure is described at a high level; a pseudocode block or explicit statement of the uniform sampling step that embeds the sparse projection would clarify how geometric diversity is preserved.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive assessment of the manuscript, the clear summary of its contributions, and the recommendation for minor revision. No major comments were provided in the report.

Circularity Check

0 steps flagged

No significant circularity; standard convergence analysis

full rationale

The paper's central claim is a conditional convergence theorem establishing that accumulation points of the proposed RZCW-PSS algorithm are Beck-Hallak zero-coordinatewise stationary under explicit regularity, sampling, and subproblem-accuracy assumptions. This is a standard-style iterative algorithm analysis whose proof structure relies on those external assumptions rather than reducing any prediction or stationarity notion to a fitted parameter or self-referential definition by construction. No self-citation chain is shown to be load-bearing for the main result, and the derivation does not rename or smuggle in prior results via ansatz in a way that collapses the claim. The result remains falsifiable against the stated assumptions and is therefore self-contained.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claims rest on regularity, sampling, and subproblem-accuracy assumptions that are standard in nonconvex optimization but not independently verified here.

axioms (1)

domain assumption Regularity, sampling, and subproblem-accuracy assumptions
Invoked to guarantee that accumulation points are Beck--Hallak zero-coordinatewise stationary.

pith-pipeline@v0.9.1-grok · 5800 in / 1258 out tokens · 63380 ms · 2026-06-29T03:01:09.332870+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

35 extracted references · 2 canonical work pages

[1]

Bauschke and Patrick L

Heinz H. Bauschke and Patrick L. Combettes. Convex Analysis and Monotone Oper- ator Theory in Hilbert Spaces . Springer International Publishing (2017)

2017
[2]

Introduction to Nonlinear Optimization: Theory, Algorithms, and Ap- plications with MATLAB

Amir Beck. Introduction to Nonlinear Optimization: Theory, Algorithms, and Ap- plications with MATLAB . MOS-SIAM Series on Optimization. Society for Industrial and Applied Mathematics, Philadelphia, PA (2014). RESER VOIR ZCW PROJECTED SUBSPACE SEARCH 41

2014
[3]

Amir Beck and Yonina C. Eldar. Sparsity constrained nonlinear optimization: Opti- mality conditions and algorithms. SIAM Journal on Optimization 23 (January 2013), 1480–1509

2013
[4]

On the minimization over sparse symmetric sets: Pro- jections, optimality conditions, and algorithms

Amir Beck and Nadav Hallak. On the minimization over sparse symmetric sets: Pro- jections, optimality conditions, and algorithms. Mathematics of Operations Research 41 (February 2016), 196–223

2016
[5]

A fast iterative shrinkage-thresholding algorithm for linear inverse problems

Amir Beck and Marc Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences 2 (2009), 183–202

2009
[6]

Probabilistic iterative hard thresholding for sparse learning

Matteo Bergamaschi, Andrea Cristofari, Vyacheslav Kungurtsev, and Francesco Ri- naldi. Probabilistic iterative hard thresholding for sparse learning. Computational Optimization and Applications 93 (August 2025), 5783

2025
[7]

Best subset selection via a modern optimization lens

Dimitris Bertsimas, Angela King, and Rahul Mazumder. Best subset selection via a modern optimization lens. The Annals of Statistics 44 (April 2016)

2016
[8]

Thomas Blumensath and Mike E. Davies. Iterative hard thresholding for compressed sensing. Applied and Computational Harmonic Analysis 27 (November 2009), 265– 274

2009
[9]

Burdakov, Christian Kanzow, and Alexandra Schwartz

Oleg P. Burdakov, Christian Kanzow, and Alexandra Schwartz. Mathematical pro- grams with cardinality constraints: Reformulation by complementarity-type condi- tions and a regularization method. SIAM Journal on Optimization 26 (January 2016), 397–425

2016
[10]

Scalable subspace methods for derivative-free nonlinear least-squares optimization

Coralia Cartis and Lindon Roberts. Scalable subspace methods for derivative-free nonlinear least-squares optimization. Mathematical Programming 199 (June 2022), 461524

2022
[11]

Randomized subspace derivative-free opti- mization with quadratic models and second-order convergence

Coralia Cartis and Lindon Roberts. Randomized subspace derivative-free opti- mization with quadratic models and second-order convergence. arXiv preprint arXiv:2412.14431 (2024)

work page arXiv 2024
[12]

Random subspace cubic- regularization methods, with applications to low-rank functions

Coralia Cartis, Zhen Shao, and Edward Tansley. Random subspace cubic- regularization methods, with applications to low-rank functions. arXiv preprint arXiv:2501.09734 (2025)

work page arXiv 2025
[13]

Donoho, and Michael A

Scott Shaobing Chen, David L. Donoho, and Michael A. Saunders. Atomic decompo- sition by basis pursuit. SIAM Journal on Scientiﬁc Computing 20 (January 1998), 3361

1998
[14]

Conn, Nicholas I

Andrew R. Conn, Nicholas I. M. Gould, and Philippe L. Toint. Trust Region Methods. MPS-SIAM Series on Optimization. SIAM (2000)

2000
[15]

The greedy strategy for optimizing the Perron eigenvalue

Aleksandar Cvetković and Vladimir Yu Protasov. The greedy strategy for optimizing the Perron eigenvalue. Mathematical Programming 193 (October 2022), 1–31

2022
[16]

A new generalized shrinkage conjugate gradient method for sparse recovery

Hamid Esmaeili, Shima Shabani, and Morteza Kimiaei. A new generalized shrinkage conjugate gradient method for sparse recovery. Calcolo 56 (December 2018)

2018
[17]

Sparse regression at scale: branch- and-bound rooted in ﬁrst-order optimization

Hussein Hazimeh, Rahul Mazumder, and Ali Saab. Sparse regression at scale: branch- and-bound rooted in ﬁrst-order optimization. Mathematical Programming 196 (Oc- tober 2021), 347–388

2021
[18]

On convergence of iterative thresholding algorithms to approximate sparse solution for composite nonconvex optimization

Yaohua Hu, Xinlin Hu, and Xiaoqi Yang. On convergence of iterative thresholding algorithms to approximate sparse solution for composite nonconvex optimization. Mathematical Programming 211 (March 2025), 181–206

2025
[19]

Revisiting Frank-Wolfe: Projection-free sparse convex optimization

Martin Jaggi. Revisiting Frank-Wolfe: Projection-free sparse convex optimization. In Proceedings of the 30th International Conference on Machine Learning , Vol. 28 of Proceedings of Machine Learning Research , pp. 427–435. PMLR (2013)

2013
[20]

Inexact penalty decomposition methods for optimization problems with geometric constraints

Christian Kanzow and Matteo Lapucci. Inexact penalty decomposition methods for optimization problems with geometric constraints. Computational Optimization and Applications 85 (March 2023), 937–971

2023
[21]

Jinhak Kim, Mohit Tawarmalani, and Jean-Philippe P. Richard. Convexiﬁcation of permutation-invariant sets and an application to sparse principal component analysis. Mathematics of Operations Research 47 (November 2022), 25472584

2022
[22]

A new limited memory method for uncon- strained nonlinear least squares

Morteza Kimiaei and Arnold Neumaier. A new limited memory method for uncon- strained nonlinear least squares. Soft Computing 26 (2022), 465–490. 42 MORTEZA KIMIAEI, SHIMA SHABANI, AND MICHAEL BREUSS

2022
[23]

MATRS: heuristic methods for noisy derivative-free bound-constrained mixed-integer optimization

Morteza Kimiaei and Arnold Neumaier. MATRS: heuristic methods for noisy derivative-free bound-constrained mixed-integer optimization. Mathematical Pro- gramming Computation 17 (May 2025), 505546

2025
[24]

LMBOPT: a limited memory method for bound-constrained optimization

Morteza Kimiaei, Arnold Neumaier, and Behzad Azmi. LMBOPT: a limited memory method for bound-constrained optimization. Mathematical Programming Computa- tion 14 (January 2022), 271318

2022
[25]

New subspace method for unconstrained derivative-free optimization

Morteza Kimiaei, Arnold Neumaier, and Parvaneh Faramarzi. New subspace method for unconstrained derivative-free optimization. ACM Transactions on Mathematical Software 49 (2023), 1–28

2023
[26]

Supplementary Material: Reservoir zero-coordinatewise projected subspace search for minimization over sparse symmetric sets in machine learning

Morteza Kimiaei, Shima Shabani, and Michael Breuß. Supplementary Material: Reservoir zero-coordinatewise projected subspace search for minimization over sparse symmetric sets in machine learning. https://github.com/GS1400/RZCWPSS/ blob/main/suppMat.pdf (2026)

2026
[27]

Convergent inexact penalty decomposition methods for cardinality-constrained problems

Matteo Lapucci, Tommaso Levato, and Marco Sciandrone. Convergent inexact penalty decomposition methods for cardinality-constrained problems. Journal of Op- timization Theory and Applications 188 (December 2020), 473–496

2020
[28]

Sparse approximation via penalty decomposition methods

Zhaosong Lu and Yong Zhang. Sparse approximation via penalty decomposition methods. SIAM Journal on Optimization 23 (January 2013), 2448–2478

2013
[29]

An eﬃcient penalty decomposition algorithm for minimization over sparse sym- metric sets (2026)

Ahmad Mousavi, Morteza Kimiaei, Saman Babaie-Kafaki, and Vyacheslav Kungurt- sev. An eﬃcient penalty decomposition algorithm for minimization over sparse sym- metric sets (2026)

2026
[30]

Linear convergence of ﬁrst or- der methods for non-strongly convex optimization

Ion Necoara, Yurii Nesterov, and François Glineur. Linear convergence of ﬁrst or- der methods for non-strongly convex optimization. Mathematical Programming 175 (2019), 69–107

2019
[31]

Regression shrinkage and selection via the lasso

Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology 58 (January 1996), 267288

1996
[32]

Randomized or- thogonal matching pursuit algorithm with adaptive partial selection for sparse signal recovery

Jinming Wen, Changhao Li, Qianyu Shu, and Zhengchun Zhou. Randomized or- thogonal matching pursuit algorithm with adaptive partial selection for sparse signal recovery. SIAM Journal on Imaging Sciences 18 (April 2025), 1028–1057

2025
[33]

Probability with Martingales

David Williams. Probability with Martingales . Cambridge University Press (February 1991)

1991
[34]

A review on subspace methods for nonlinear optimization

Ya-xiang Yuan. A review on subspace methods for nonlinear optimization. In Pro- ceedings of the International Congress of Mathematics , pp. 807–827 (2014)

2014
[35]

Optimal k-thresholding algorithms for sparse optimization problems

Yun–Bin Zhao. Optimal k-thresholding algorithms for sparse optimization problems. SIAM Journal on Optimization 30 (January 2020), 31–55. F akultät für Mathematik, Universität Wien, Oskar-Morgenstern-Platz 1, A-1090 Wien, Austria Email address : morteza.kimiaei@univie.ac.at Institute for Mathematics, Brandenburg University of Technology, Platz der Deutsche...

2020

[1] [1]

Bauschke and Patrick L

Heinz H. Bauschke and Patrick L. Combettes. Convex Analysis and Monotone Oper- ator Theory in Hilbert Spaces . Springer International Publishing (2017)

2017

[2] [2]

Introduction to Nonlinear Optimization: Theory, Algorithms, and Ap- plications with MATLAB

Amir Beck. Introduction to Nonlinear Optimization: Theory, Algorithms, and Ap- plications with MATLAB . MOS-SIAM Series on Optimization. Society for Industrial and Applied Mathematics, Philadelphia, PA (2014). RESER VOIR ZCW PROJECTED SUBSPACE SEARCH 41

2014

[3] [3]

Amir Beck and Yonina C. Eldar. Sparsity constrained nonlinear optimization: Opti- mality conditions and algorithms. SIAM Journal on Optimization 23 (January 2013), 1480–1509

2013

[4] [4]

On the minimization over sparse symmetric sets: Pro- jections, optimality conditions, and algorithms

Amir Beck and Nadav Hallak. On the minimization over sparse symmetric sets: Pro- jections, optimality conditions, and algorithms. Mathematics of Operations Research 41 (February 2016), 196–223

2016

[5] [5]

A fast iterative shrinkage-thresholding algorithm for linear inverse problems

Amir Beck and Marc Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences 2 (2009), 183–202

2009

[6] [6]

Probabilistic iterative hard thresholding for sparse learning

Matteo Bergamaschi, Andrea Cristofari, Vyacheslav Kungurtsev, and Francesco Ri- naldi. Probabilistic iterative hard thresholding for sparse learning. Computational Optimization and Applications 93 (August 2025), 5783

2025

[7] [7]

Best subset selection via a modern optimization lens

Dimitris Bertsimas, Angela King, and Rahul Mazumder. Best subset selection via a modern optimization lens. The Annals of Statistics 44 (April 2016)

2016

[8] [8]

Thomas Blumensath and Mike E. Davies. Iterative hard thresholding for compressed sensing. Applied and Computational Harmonic Analysis 27 (November 2009), 265– 274

2009

[9] [9]

Burdakov, Christian Kanzow, and Alexandra Schwartz

Oleg P. Burdakov, Christian Kanzow, and Alexandra Schwartz. Mathematical pro- grams with cardinality constraints: Reformulation by complementarity-type condi- tions and a regularization method. SIAM Journal on Optimization 26 (January 2016), 397–425

2016

[10] [10]

Scalable subspace methods for derivative-free nonlinear least-squares optimization

Coralia Cartis and Lindon Roberts. Scalable subspace methods for derivative-free nonlinear least-squares optimization. Mathematical Programming 199 (June 2022), 461524

2022

[11] [11]

Randomized subspace derivative-free opti- mization with quadratic models and second-order convergence

Coralia Cartis and Lindon Roberts. Randomized subspace derivative-free opti- mization with quadratic models and second-order convergence. arXiv preprint arXiv:2412.14431 (2024)

work page arXiv 2024

[12] [12]

Random subspace cubic- regularization methods, with applications to low-rank functions

Coralia Cartis, Zhen Shao, and Edward Tansley. Random subspace cubic- regularization methods, with applications to low-rank functions. arXiv preprint arXiv:2501.09734 (2025)

work page arXiv 2025

[13] [13]

Donoho, and Michael A

Scott Shaobing Chen, David L. Donoho, and Michael A. Saunders. Atomic decompo- sition by basis pursuit. SIAM Journal on Scientiﬁc Computing 20 (January 1998), 3361

1998

[14] [14]

Conn, Nicholas I

Andrew R. Conn, Nicholas I. M. Gould, and Philippe L. Toint. Trust Region Methods. MPS-SIAM Series on Optimization. SIAM (2000)

2000

[15] [15]

The greedy strategy for optimizing the Perron eigenvalue

Aleksandar Cvetković and Vladimir Yu Protasov. The greedy strategy for optimizing the Perron eigenvalue. Mathematical Programming 193 (October 2022), 1–31

2022

[16] [16]

A new generalized shrinkage conjugate gradient method for sparse recovery

Hamid Esmaeili, Shima Shabani, and Morteza Kimiaei. A new generalized shrinkage conjugate gradient method for sparse recovery. Calcolo 56 (December 2018)

2018

[17] [17]

Sparse regression at scale: branch- and-bound rooted in ﬁrst-order optimization

Hussein Hazimeh, Rahul Mazumder, and Ali Saab. Sparse regression at scale: branch- and-bound rooted in ﬁrst-order optimization. Mathematical Programming 196 (Oc- tober 2021), 347–388

2021

[18] [18]

On convergence of iterative thresholding algorithms to approximate sparse solution for composite nonconvex optimization

Yaohua Hu, Xinlin Hu, and Xiaoqi Yang. On convergence of iterative thresholding algorithms to approximate sparse solution for composite nonconvex optimization. Mathematical Programming 211 (March 2025), 181–206

2025

[19] [19]

Revisiting Frank-Wolfe: Projection-free sparse convex optimization

Martin Jaggi. Revisiting Frank-Wolfe: Projection-free sparse convex optimization. In Proceedings of the 30th International Conference on Machine Learning , Vol. 28 of Proceedings of Machine Learning Research , pp. 427–435. PMLR (2013)

2013

[20] [20]

Inexact penalty decomposition methods for optimization problems with geometric constraints

Christian Kanzow and Matteo Lapucci. Inexact penalty decomposition methods for optimization problems with geometric constraints. Computational Optimization and Applications 85 (March 2023), 937–971

2023

[21] [21]

Jinhak Kim, Mohit Tawarmalani, and Jean-Philippe P. Richard. Convexiﬁcation of permutation-invariant sets and an application to sparse principal component analysis. Mathematics of Operations Research 47 (November 2022), 25472584

2022

[22] [22]

A new limited memory method for uncon- strained nonlinear least squares

Morteza Kimiaei and Arnold Neumaier. A new limited memory method for uncon- strained nonlinear least squares. Soft Computing 26 (2022), 465–490. 42 MORTEZA KIMIAEI, SHIMA SHABANI, AND MICHAEL BREUSS

2022

[23] [23]

MATRS: heuristic methods for noisy derivative-free bound-constrained mixed-integer optimization

Morteza Kimiaei and Arnold Neumaier. MATRS: heuristic methods for noisy derivative-free bound-constrained mixed-integer optimization. Mathematical Pro- gramming Computation 17 (May 2025), 505546

2025

[24] [24]

LMBOPT: a limited memory method for bound-constrained optimization

Morteza Kimiaei, Arnold Neumaier, and Behzad Azmi. LMBOPT: a limited memory method for bound-constrained optimization. Mathematical Programming Computa- tion 14 (January 2022), 271318

2022

[25] [25]

New subspace method for unconstrained derivative-free optimization

Morteza Kimiaei, Arnold Neumaier, and Parvaneh Faramarzi. New subspace method for unconstrained derivative-free optimization. ACM Transactions on Mathematical Software 49 (2023), 1–28

2023

[26] [26]

Supplementary Material: Reservoir zero-coordinatewise projected subspace search for minimization over sparse symmetric sets in machine learning

Morteza Kimiaei, Shima Shabani, and Michael Breuß. Supplementary Material: Reservoir zero-coordinatewise projected subspace search for minimization over sparse symmetric sets in machine learning. https://github.com/GS1400/RZCWPSS/ blob/main/suppMat.pdf (2026)

2026

[27] [27]

Convergent inexact penalty decomposition methods for cardinality-constrained problems

Matteo Lapucci, Tommaso Levato, and Marco Sciandrone. Convergent inexact penalty decomposition methods for cardinality-constrained problems. Journal of Op- timization Theory and Applications 188 (December 2020), 473–496

2020

[28] [28]

Sparse approximation via penalty decomposition methods

Zhaosong Lu and Yong Zhang. Sparse approximation via penalty decomposition methods. SIAM Journal on Optimization 23 (January 2013), 2448–2478

2013

[29] [29]

An eﬃcient penalty decomposition algorithm for minimization over sparse sym- metric sets (2026)

Ahmad Mousavi, Morteza Kimiaei, Saman Babaie-Kafaki, and Vyacheslav Kungurt- sev. An eﬃcient penalty decomposition algorithm for minimization over sparse sym- metric sets (2026)

2026

[30] [30]

Linear convergence of ﬁrst or- der methods for non-strongly convex optimization

Ion Necoara, Yurii Nesterov, and François Glineur. Linear convergence of ﬁrst or- der methods for non-strongly convex optimization. Mathematical Programming 175 (2019), 69–107

2019

[31] [31]

Regression shrinkage and selection via the lasso

Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology 58 (January 1996), 267288

1996

[32] [32]

Randomized or- thogonal matching pursuit algorithm with adaptive partial selection for sparse signal recovery

Jinming Wen, Changhao Li, Qianyu Shu, and Zhengchun Zhou. Randomized or- thogonal matching pursuit algorithm with adaptive partial selection for sparse signal recovery. SIAM Journal on Imaging Sciences 18 (April 2025), 1028–1057

2025

[33] [33]

Probability with Martingales

David Williams. Probability with Martingales . Cambridge University Press (February 1991)

1991

[34] [34]

A review on subspace methods for nonlinear optimization

Ya-xiang Yuan. A review on subspace methods for nonlinear optimization. In Pro- ceedings of the International Congress of Mathematics , pp. 807–827 (2014)

2014

[35] [35]

Optimal k-thresholding algorithms for sparse optimization problems

Yun–Bin Zhao. Optimal k-thresholding algorithms for sparse optimization problems. SIAM Journal on Optimization 30 (January 2020), 31–55. F akultät für Mathematik, Universität Wien, Oskar-Morgenstern-Platz 1, A-1090 Wien, Austria Email address : morteza.kimiaei@univie.ac.at Institute for Mathematics, Brandenburg University of Technology, Platz der Deutsche...

2020