Pith. sign in

REVIEW 3 major objections 5 minor 65 references

A single-loop SPIDER-type stochastic subgradient method for expectation-constrained nonconvex nonsmooth optimization

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes a single-loop SPIDER-type stochastic subgradient method, 3S-Econ, and claims it finds an (ε, ε)-KKT point of a nonconvex nonsmooth expectation-constrained problem with O(ε⁻⁴) objective and constraint subgradient…

desk verdict A genuinely useful single-loop method with a clean complexity story, but the quantitative Slater condition is load-bearing and the weakly convex experiments do not verify it. read the letter →

arxiv 2501.19214 v2 pith:K2V3726Y submitted 2025-01-31 math.OC cs.CCcs.LGcs.NAmath.NA

Wei Liu , Yangyang Xu This is my paper · ORCID
classification math.OCcs.CCcs.LGcs.NAmath.NA MSC 90C0690C1549M3765K05
keywords expectationconstraintsnonconvexnonsmoothoptimizationstochasticsubgradientmethodSPIDERvariancereductionexactpenaltysmoothingMoreauenvelopeKKTcomplexityfairness-constrainedclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a single-loop stochastic subgradient method, 3S-Econ, for solving nonconvex nonsmooth optimization problems with expectation constraints. The central claim is that the method reaches an (ε, ε)-KKT point of the original problem in expectation with O(ε⁻⁴) objective and constraint subgradient evaluations. For general stochastic constraints the constraint function-value complexity is O(ε⁻⁶); for finite-sum constraints it improves to O(N + √N ε⁻⁴). If correct, this is the first single-loop method to attain the ε⁻⁴ iteration complexity for this problem class without an inner solver.

What carries the argument

The engine of the argument is the smoothed penalty objective F_ν(x) = f(x) + βh_ν(g(x)), where h_ν is a Huber-type smoothing of the positive-part penalty, together with a SPIDER-type estimator u^(k) of the constraint value g(x^(k)) that resets to a large batch every q iterations and recursively updates in between. The descent lemma for the Moreau envelope F_ν_{1/(2ρ)} (Lemma 3.3) bounds the variance-induced bias by α_k β²(l_g² + σ₂²)/ν² · r_g^(k) and yields a decrease proportional to α_k ‖∇F_ν_{1/(2ρ)}(x^(k))‖², which telescopes to the O(ε⁻⁴) iteration complexity.

What would settle it

Construct a weakly convex expectation-constrained instance satisfying Assumptions 1 and 3 where the quantitative Slater-type condition Bg > mρgD² fails (for example, a scalar weakly convex constraint with large ρg and D such that the maximum constraint violation at the relevant points is below mρgD² but the uniform Slater condition still holds), run 3S-Econ, and test whether the computed iterates converge to an (ε, ε)-KKT point with the claimed O(ε⁻⁴) complexity; a failure would falsify the sufficiency of the stated condition.

Watch

Extended reading notes

Core claim

On the paper's own terms, it establishes that near-stationarity of the smoothed exact penalty problem minₓ f(x) + βh_ν(g(x)) translates into an (ε, ε)-KKT point of the original expectation-constrained problem. The method uses a Huber-type smoothing h_ν whose gradient is Proj[0,1](z/ν), and a SPIDER/SARAH estimator for the constraint function values. Theorem 3.4 shows the penalized problem is solved to near-ε stationarity in expectation within O(ε⁻⁴) iterations, matching the known lower bound for nonsmooth nonconvex stochastic optimization. Theorem 2.5 then converts that stationarity into an (ε, ε)-KKT point of problem (P), provided the penalty parameter is sufficiently large and a quantitative Slater-type constraint qualification holds with Bg > mρgD².

Load-bearing premise

The argument depends on Assumption 2 holding with the quantitative strengthening Bg > mρgD²; this condition is used in the proof of Theorem 2.5, specifically inequality (2.15), to guarantee that the smoothed penalty at the Moreau-envelope point stays within the range where the Slater-type constraint qualification applies.

Editorial extensions

If this is right

  • If the central claim is correct, expectation-constrained nonconvex nonsmooth problems can be solved by a simple single-loop update without an inner subsolver, eliminating the need to tune inner iterations or design stopping criteria for subproblems.
  • The O(ε⁻⁴) iteration complexity matches the lower bound for nonsmooth nonconvex stochastic optimization, so no first-order method of this type can do better in iteration count.
  • For finite-sum constraints, the constraint value complexity O(N + √N ε⁻⁴) improves on the previous best known O(min{N ε⁻⁴, ε⁻⁶}) bound, which is relevant to large-scale fairness and Neyman-Pearson classification problems.
  • The equivalence between near-stationarity of the smoothed penalty problem and an (ε, ε)-KKT point of the original problem provides a template for other smoothed exact penalty methods with stochastic constraints.
  • The numerical experiments suggest speedups up to hundreds of times in data passes against switching subgradient and inexact proximal point methods on fairness-constrained and Neyman-Pearson problems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper notes that the complexity results remain valid for other smoothings such as Softplus or log-sum-exp; a natural testable extension is whether those smoothings preserve the same constants and empirical speedups, or only the asymptotic rates.
  • The deterministic-case relaxation (Remark 3.8) rests on a near-feasible initialization guarantee deferred to the paper's v1; a concrete check is whether random or standard initializations in practice satisfy that closeness assumption for the tested problems.
  • The paper conjectures a total complexity lower bound of Ω(ε⁻⁵) for penalty-based methods on this problem class; if true, the gap between the ε⁻⁴ iteration complexity and the ε⁻⁶ constraint-value complexity would be intrinsic rather than an artifact of the analysis.
  • The method's reliance on the quantitative Slater condition Bg > mρgD² suggests that for problems with large diameter D or many constraints m, the penalty parameter may need to be impractically large; experiments with such ill-conditioned instances would clarify the practical reach of the theory.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies nonconvex nonsmooth stochastic optimization problems with expectation constraints of the form (P), proposes a smoothed exact penalty reformulation (PP), and analyzes a single-loop SPIDER-type stochastic subgradient method (3S-Econ) for solving (PP). The main theoretical claims are: near-stationary points of the penalized problem yield (epsilon,epsilon)-KKT points of the original problem under a quantitative Slater-type constraint qualification, and the method reaches O(epsilon^-4) objective/constraint subgradient complexity with O(epsilon^-6) constraint value complexity in the general stochastic case and O(N + sqrt(N) epsilon^-4) in the finite-sum case. Numerical experiments on fairness-constrained classification and Neyman-Pearson classification report substantial speedups over switching subgradient and inexact proximal point methods.

Significance. If the central claims are correct, this is the first single-loop SPIDER-type stochastic subgradient method for expectation-constrained nonconvex nonsmooth problems that reaches epsilon^-4 iteration complexity without an inner solver, and the reported epsilon^-2 improvement in either objective subgradient or constraint value complexity over state-of-the-art methods is a meaningful contribution. The convergence analysis of the method on the penalized problem is detailed and largely self-contained: Lemmas 3.1 and 3.3 and Theorem 3.4 give explicit parameter choices and a careful variance-reduction argument. The exact-penalization bridge, however, currently has unresolved technical gaps concerning the quantitative constraint qualification and the reuse of the symbol rho in Theorem 2.5; these need to be fixed before the headline complexity claims are fully supported.

major comments (3)
  1. [Theorem 2.5 and Proposition 3.7 (Section 2.3 and Section 3.3)] The quantitative condition Bg > m*rho_g*D^2 is load-bearing: inequality (2.15) requires the denominator Bg - m*nu - m*rho_g*D^2 to be positive, and the subsequent bound on g+(xhat*) is what enables Lemma 2.2 and the construction of the KKT multiplier. For the weakly convex numerical problems (Problems 2 and 3, where rho_g > 0), Appendix B verifies the standard Slater-type CQ with upper bounds on Bg, but it does not verify the lower bound Bg > m*rho_g*D^2. Consequently, the (epsilon,epsilon)-KKT guarantee claimed in Proposition 3.7 is not actually validated for the tested weakly convex instances. Please either verify the quantitative lower bound for these examples, add an explicit condition that can be checked, or clearly restrict the theoretical claims to cases where the inequality is known to hold.
  2. [Theorem 2.5, Eq. (2.10) and Eq. (2.17)] The proof reuses the symbol rho for both the Assumption 2 parameter and the weak convexity modulus rho_f + beta*m*rho_g of the penalized objective. Lemma 2.2 and inequality (2.17) require the CQ parameter rho_A from Assumption 2, because the lower bound theta = sqrt(2B(rho - rho_g)) is derived from condition (2.1) with that particular rho. The theorem instead defines theta with rho = rho_f + beta*m*rho_g, but no argument shows that condition (2.1) holds for this larger rho; increasing rho makes the left-hand side of (2.1) larger and can destroy the inequality. This makes the lower bound in (2.17) unsupported as written. Please separate the two parameters, for example rho_A for the constraint qualification and rho_F = rho_f + beta*m*rho_g for the Moreau envelope, and restate the conditions in (2.10) in terms of theta_A = sqrt(2B(rho_A - rho_g)).
  3. [Remark 3.8] The deterministic relaxation of the condition Bg > m*rho_g*D^2 is not proved in this manuscript: the remark asserts that the condition can be dropped by citing the authors' own arXiv first version [44] for a special m = 1 result and a near-feasible initialization guarantee. Since this premise is needed to ensure max_i g_i(xhat(k)) <= Bg for all iterates, it is load-bearing for the deterministic claim. A peer-reviewed version should contain a complete proof of this step or should state the deterministic result only under an explicit near-feasible initialization assumption.
minor comments (5)
  1. [Algorithm 3.1, line 1] The word 'intergers' should be 'integers'.
  2. [Section 1.2 and title page] There are spacing artifacts in the headings ('T ools' in Section 1.2 and 'EXPECT A TION' in the title); these should be corrected.
  3. [Figure 1] The y-axis label 'Constaints Violation' contains a typo; it should be 'Constraints Violation'.
  4. [Section 3.1 and Theorem 3.4] The text before Algorithm 3.1 refers to a tolerance bar-epsilon, while Theorem 3.4 and Section 3.3 use epsilon; please unify the notation.
  5. [Section 5] The lower-bound discussion based on the Lipschitz constant of nabla F^nu is heuristic and is presented as a conjecture; it would be clearer to move it to a dedicated discussion section and explicitly separate it from the proven upper bounds.

Circularity Check

1 steps flagged · score 2.0 of 10

Core stochastic complexity proof is self-contained; only the deterministic relaxation in Remark 3.8 leans on the authors' own unpublished v1, making the circularity minor.

  1. self citation load bearing [Remark 3.8, Section 3.3; reference [44]]
    "In contrast, for the deterministic case, the constraints gi(x) ≤ 0, ∀ i ∈ [m] can be equivalently expressed as maxi∈[m] gi(x) ≤ 0. Leveraging this reformulation, we directly utilize the established convergence result for the special case of m = 1, detailed in our arXiv first version [44]. This prior result ensures that, provided the initial point is sufficiently close to the feasible set X, all iterates produced by our method satisfy max i∈[m] gi(bx(k)) ≤ Bg."

    The deterministic claim that the condition Bg > mρgD² is unnecessary rests on a premise — near-feasible initialization and iterates staying within max_i gi(bx(k)) ≤ Bg — that is not proved in this paper but is imported from the authors' own arXiv v1 [44]. Since [44] is the same work in earlier version rather than an independent, externally verified source, the support for this remark reduces to a self-citation. This is not load-bearing for the main stochastic complexity theorem, so the circularity is minor; the central derivation chain does not otherwise reduce to its own inputs.

full rationale

The central derivation chain — Theorem 3.4 (SPIDER-type convergence to a near-ε stationary point of the smoothed penalty problem PP), Theorem 2.5 (exact-penalization bridge from PP to an (ε,ε)-KKT point of P), and Proposition 3.7 (complexity bookkeeping) — is an internal analysis against external benchmarks and does not fit the target result. Theorem 2.5 uses the quantitative constraint qualification Bg > mρgD² as an explicit assumption, and inequality (2.15) derives g+(bx*) ≤ Bg from the stated β and ν choices rather than assuming the conclusion. Whether that quantitative CQ is actually satisfied for the weakly convex test problems is a verification/correctness issue — Appendix B checks the Slater-type CQ from [27] but not the stronger Bg > mρgD² — not a circular reduction. The only self-citation carrying argumentative weight is Remark 3.8, where the deterministic relaxation's near-feasibility guarantee is taken from the authors' own v1 [44]; this does not affect the main stochastic Proposition 3.7. Numerical speedups are reported measurements rather than fitted predictions, and the lower-bound matching cites external work [29]. Overall, the paper's principal claim is independently derived; the minor self-citation in a side remark justifies a low score.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The theoretical complexity does not fit any free parameter to the target result; it relies on standard weak-convexity calculus plus three regularity assumptions. The only hand-chosen experimental parameters are hyperparameters for the numerical study, and no new physical or mathematical entities are postulated.

free parameters (1)
  • Experimental algorithmic hyperparameters (beta, nu, alpha, S1, S2, q) = beta=10, nu=1e-5, alpha=0.01 (deterministic) or 1/(100*ceil(sqrt(k/q))) (stochastic), S1=n, S2=q=ceil(sqrt(n))
    The theoretical bound uses formulas for these parameters, but the numerical comparisons use defaults selected by performance testing on Problem 1 with a9a (footnote 3); these tuned choices affect reported speedups but not the O(epsilon^-4) complexity theorem.
assumptions (5)
  • domain assumption Assumption 1: X is bounded and f, g_i are weakly convex with Lipschitz-bounded subdifferentials on X.
    States the problem class in Section 1 and is used throughout the analysis.
  • domain assumption Assumption 2: uniform Slater-type constraint qualification with the quantitative strengthening Bg > m*rho_g*D^2.
    Introduced in Section 2.1; necessary for the exact-penalization equivalence in Theorem 2.5, especially inequality (2.15).
  • domain assumption Assumption 3: unbiased stochastic subgradients, unbiased g-value estimates with variance bound sigma^2, and mean-squared Lipschitz continuity of g.
    Section 3.1; enables the SPIDER-type estimator analysis in Lemma 3.1 and Theorem 3.4.
  • standard math Standard Moreau envelope and weak convexity calculus, including Danskin's theorem and results of Davis-Drusvyatskiy [20] and Drusvyatskiy-Paquette [21].
    Used in Lemmas 2.4, 3.3 and Theorem 3.4 without reproving.
  • ad hoc to paper Near-feasible initialization guarantee for the deterministic case, deferred to the authors' own v1 [44].
    Remark 3.8 invokes [44] to remove Bg > m*rho_g*D^2 when g is deterministic; not derived in v2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A single-loop SPIDER-type stochastic subgradient method for expectation-constrained nonconvex nonsmooth optimization." pith.science (2026). https://pith.science/paper/K2V3726Y

@misc{pith2026250119214,
  author       = {Pith},
  title        = {Pith review of: A single-loop SPIDER-type stochastic subgradient method for expectation-constrained nonconvex nonsmooth optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K2V3726Y}},
  note         = {Machine review of arXiv:2501.19214}
}
abstract

Many real-world problems, such as those with fairness constraints, involve complex expectation constraints and large datasets, necessitating the design of efficient stochastic methods to solve them. Most existing research focuses on cases with no {constraint} or easy-to-project constraints or deterministic constraints. In this paper, we consider nonconvex nonsmooth stochastic optimization problems with expectation constraints, for which we build a novel exact penalty model. We first show the relationship between the penalty model and the original problem. Then on solving the penalty problem, we present a single-loop SPIDER-type stochastic subgradient method, which utilizes the subgradients of both the objective and constraint functions, as well as the constraint function value at each iteration. Under certain regularity conditions (weaker than Slater-type constraint qualification or strong feasibility assumed in existing works), we establish an iteration complexity result of $O(\epsilon^{-4})$ to reach a near-$\epsilon$ stationary point of the penalized problem in expectation, matching the lower bound for such tasks. Building on the exact penalization, an $(\epsilon,\epsilon)$-KKT point of the original problem is obtained. For a few scenarios, our complexity of either the {objective} sample subgradient or the constraint sample function values can be lower than the state-of-the-art results by a factor of $\epsilon^{-2}$. Moreover, on solving two fairness-constrained problems and a multi-class Neyman-Pearson classification problem, our method is significantly (up to 466 times) faster than the state-of-the-art algorithms, including switching subgradient method and inexact proximal point methods.

Figures

Figures reproduced from arXiv: 2501.19214 by the authors.

Figure 1
Figure 1. Performance of 3S-Econ-D with different β and ν on solving Problem 1 by using the a9a dataset. Top row: different values of β and fixed ν = 10−5 ; Bottom row: different values of ν and fixed β = 10. depicted in [PITH_FULL_IMAGE:figures/full_fig_p020_1.png] view at source ↗
Figure 2
Figure 2. Comparisons between 3S-Econ-D (deterministic version of our method) and [PITH_FULL_IMAGE:figures/full_fig_p021_2.png] view at source ↗
Figure 3
Figure 3. Comparisons between the deterministic and stochastic versions of our method [PITH_FULL_IMAGE:figures/full_fig_p022_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Comparisons of different methods on Problem 3 by using [PITH_FULL_IMAGE:figures/full_fig_p022_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 59 canonical work pages

  1. [44]

    Liu and Y

    W. Liu and Y. Xu , A single-loop spider-type stochastic subgradient method for expectation- constrained nonconvex nonsmooth optimization, Preprint, arXiv:2501.19214, v1, (2025)

    arXiv 2025

  2. [1]

    Agarwal, A

    A. Agarwal, A. Beygelzimer, M. Dud ´ık, J. Langford, and H. W allach , A reductions approach to fair classification , in International Conference on Machine Learning, PMLR, 2018, pp. 60–69

    work page 2018

  3. [2]

    Alacaoglu and S

    A. Alacaoglu and S. J. Wright, Complexity of single loop algorithms for nonlinear program- ming with stochastic objective and constraints , in International Conference on Artificial Intelligence and Statistics, PMLR, 2024, pp. 4627–4635

    work page 2024

  4. [3]

    Andreani, G

    R. Andreani, G. Haeser, and J. Mart ´ınez, On sequential optimality conditions for smooth constrained optimization, Optimization, 60 (2011), pp. 627–641

    work page 2011

  5. [4]

    Andreani, J

    R. Andreani, J. Mart ´ınez, A. Ramos, and P. Silva , Strict constraint qualifications and sequential optimality conditions for constrained optimization , Mathematics of Operations Research, 43 (2018), pp. 693–717

    work page 2018

  6. [5]

    Angwin, J

    J. Angwin, J. Larson, S. Mattu, and L. Kirchner , Machine bias , in Ethics of Data and Analytics, Auerbach Publications, 2022, pp. 254–264

    work page 2022

  7. [6]

    Arjevani, Y

    Y. Arjevani, Y. Carmon, J. C. Duchi, D. J. Foster, N. Srebro, and B. Woodworth , Lower bounds for non-convex stochastic optimization , Mathematical Programming, 199 (2023), pp. 165–214

    work page 2023

  8. [7]

    N. S. Aybat and G. Iyengar, A first-order smoothed penalty method for compressed sensing , SIAM Journal on Optimization, 21 (2011), pp. 287–313

    work page 2011

Show all 65 references
  1. [8]

    Bayandina, P

    A. Bayandina, P. Dvurechensky, A. Gasnikov, F. Stonyakin, and A. Titov , Mirror de- scent and convex optimization problems with non-smooth inequality constraints , Large- scale and Distributed Optimization, (2018), pp. 181–213

    2018

  2. [9]

    D. P. Bertsekas , Nonlinear programming, Journal of the Operational Research Society, 48 (1997), pp. 334–334

    1997

  3. [10]

    D. Boob, Q. Deng, and G. Lan , Stochastic first-order methods for convex and nonconvex functional constrained optimization, Mathematical Programming, 197 (2023), pp. 215–279

    2023

  4. [11]

    J. V. Burke and T. Hoheisel, Epi-convergent smoothing with applications to convex composite functions, SIAM Journal on Optimization, 23 (2013), pp. 1457–1479

    2013

  5. [12]

    J. V. Burke and T. Hoheisel, Epi-convergence properties of smoothing by infimal convolution, Set-Valued and Variational Analysis, 25 (2017), pp. 1–23

    2017

  6. [13]

    Cartis, N

    C. Cartis, N. Gould, and P. L. Toint, On the complexity of finding first-order critical points in constrained nonlinear optimization , Mathematical Programming, 144 (2014), pp. 93– 106

    2014

  7. [14]

    Chen and O

    C. Chen and O. L. Mangasarian , A class of smoothing functions for nonlinear and mixed complementarity problems, Computational Optimization and Applications, 5 (1996), pp. 97–138

    1996

  8. [15]

    Chen , Smoothing methods for nonsmooth, nonconvex minimization , Mathematical pro- gramming, 134 (2012), pp

    X. Chen , Smoothing methods for nonsmooth, nonconvex minimization , Mathematical pro- gramming, 134 (2012), pp. 71–99

    2012

  9. [16]

    X. Chen, Z. Lu, and T. K. Pong , Penalty methods for a class of non-lipschitz optimization problems, SIAM Journal on Optimization, 26 (2016), pp. 1465–1492

    2016

  10. [17]

    Chouldechova, Fair prediction with disparate impact: A study of bias in recidivism pre- diction instruments , Big Data, 5 (2017), pp

    A. Chouldechova, Fair prediction with disparate impact: A study of bias in recidivism pre- diction instruments , Big Data, 5 (2017), pp. 153–163

    2017

  11. [18]

    F. H. Clarke , Optimization and nonsmooth analysis , SIAM, 1990

    1990

  12. [19]

    Y. Cui, X. W ang, and X. Xiao, A two-phase stochastic momentum-based algorithm for non- convex expectation-constrained optimization, Journal of Scientific Computing, 104 (2025), pp. 1–27

    2025

  13. [20]

    Davis and D

    D. Davis and D. Drusvyatskiy, Stochastic model-based minimization of weakly convex func- tions, SIAM Journal on Optimization, 29 (2019), pp. 207–239

    2019

  14. [21]

    Drusvyatskiy and C

    D. Drusvyatskiy and C. Paquette , Efficiency of minimizing compositions of convex func- tions and smooth maps , Mathematical Programming, 178 (2019), pp. 503–558

    2019

  15. [22]

    F acchinei, V

    F. F acchinei, V. Kungurtsev, L. Lampariello, and G. Scutari , Ghost penalties in non- convex constrained optimization: Diminishing stepsizes and iteration complexity , Mathe- matics of Operations Research, 46 (2021), pp. 595–627

    2021

  16. [23]

    F an and R

    J. F an and R. Li, Variable selection via nonconcave penalized likelihood and its oracle prop- STOCHASTIC METHOD FOR EXPECTATION-CONSTRAINED OPTIMIZATION 25 erties, Journal of the American statistical Association, 96 (2001), pp. 1348–1360

    2001

  17. [24]

    F ang, C

    C. F ang, C. J. Li, Z. Lin, and T. Zhang , Spider: Near-optimal non-convex optimization via stochastic path-integrated differential estimator, Advances in Neural Information Process- ing Systems, 31 (2018)

    2018

  18. [25]

    Grimmer and Z

    B. Grimmer and Z. Jia , Goldstein stationarity in lipschitz constrained optimization , Opti- mization Letters, 19 (2025), pp. 425–435

    2025

  19. [26]

    Hardt, E

    M. Hardt, E. Price, and N. Srebro, Equality of opportunity in supervised learning, Advances in Neural Information Processing Systems, 29 (2016)

    2016

  20. [27]

    Huang and Q

    Y. Huang and Q. Lin, Oracle complexity of single-loop switching subgradient methods for non- smooth weakly convex functional constrained optimization, Advances in Neural Information Processing Systems, 36 (2023), pp. 61327–61340

    2023

  21. [28]

    Jia and B

    Z. Jia and B. Grimmer, First-order methods for nonsmooth nonconvex functional constrained optimization with or without slater points , SIAM Journal on Optimization, 35 (2025), pp. 1300–1329

    2025

  22. [29]

    Jordan, G

    M. Jordan, G. Kornowski, T. Lin, O. Shamir, and M. Zampetakis , Deterministic non- smooth nonconvex optimization, in The Thirty Sixth Annual Conference on Learning The- ory, PMLR, 2023, pp. 4570–4597

    2023

  23. [30]

    Kohavi et al

    R. Kohavi et al. , Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid. , in Kdd, vol. 96, 1996, pp. 202–207

    1996

  24. [31]

    Lan and R

    G. Lan and R. D. Monteiro , Iteration-complexity of first-order penalty methods for convex programming, Mathematical Programming, 138 (2013), pp. 115–139

    2013

  25. [32]

    Lan and R

    G. Lan and R. D. Monteiro, Iteration-complexity of first-order augmented Lagrangian meth- ods for convex programming, Mathematical Programming, 155 (2016), pp. 511–547

    2016

  26. [33]

    Lan and Z

    G. Lan and Z. Zhou , Algorithms for stochastic optimization with function or expectation constraints, Computational Optimization and Applications, 76 (2020), pp. 461–498

    2020

  27. [34]

    LeCun, The MNIST database of handwritten digits , (1998)

    Y. LeCun, The MNIST database of handwritten digits , (1998)

    1998

  28. [35]

    G. Li, W. Yu, Y. Yao, W. Tong, Y. Liang, Q. Lin, and T. Yang , Model developmental safety: A safety-centric method and applications in vision-language models , (2024)

    2024

  29. [36]

    M. Li, P. Grigas, and A. Atamt¨urk, On the softplus penalty for large-scale convex optimiza- tion, Operations Research Letters, 51 (2023), pp. 666–672

    2023

  30. [37]

    M. Li, P. Grigas, and A. Atamt ¨urk, New penalized stochastic gradient methods for linearly constrained strongly convex optimization , Journal of Optimization Theory and Applica- tions, 205 (2025), p. 29

    2025

  31. [38]

    Li, P.-Y

    Z. Li, P.-Y. Chen, S. Liu, S. Lu, and Y. Xu , Stochastic inexact augmented Lagrangian method for nonconvex expectation constrained optimization , Computational Optimization and Applications, 87 (2024), pp. 117–147

    2024

  32. [39]

    Q. Lin, R. Ma, and Y. Xu , Complexity of an inexact proximal-point penalty method for con- strained smooth non-convex optimization , Computational Optimization and Applications, 82 (2022), pp. 175–224

    2022

  33. [40]

    Q. Lin, R. Ma, and T. Yang, Level-set methods for finite-sum constrained convex optimization, in International Conference on Machine Learning, 2018, pp. 3112–3121

    2018

  34. [41]

    W. Liu, M. Khan, G. Mancino-Ball, and Y. Xu , A stochastic smoothing framework for nonconvex-nonconcave min-sum-max problems with applications to wasserstein distribu- tionally robust optimization , Preprint, arXiv:2502.17602, (2025)

    2025 arXiv

  35. [42]

    W. Liu, X. Liu, and X. Chen , Linearly constrained nonsmooth optimization for training autoencoders, SIAM Journal on Optimization, 32 (2022), pp. 1931–1957

    2022

  36. [43]

    W. Liu, X. Liu, and X. Chen , An inexact augmented Lagrangian algorithm for training leaky ReLU neural network with group sparsity , Journal of Machine Learning Research, 24 (2023), pp. 1–43

    2023

  37. [45]

    Y. Liu, X. Liu, and S. Ma , On the non-ergodic convergence rate of an inexact augmented Lagrangian framework for composite convex programming , Preprint, arXiv:1603.05738, (2016)

    2016 arXiv

  38. [46]

    R. Ma, Q. Lin, and T. Yang , Quadratically regularized subgradient methods for weakly con- vex optimization with weakly convex constraints , in International Conference on Machine Learning, PMLR, 2020, pp. 6554–6564

    2020

  39. [47]

    Mart´ınez and B

    J. Mart´ınez and B. Svaiter, A practical optimality condition without constraint qualifications for nonlinear programming, Journal of Optimization Theory and Applications, 118 (2003), pp. 117–133

    2003

  40. [48]

    Nedi´c and T

    A. Nedi´c and T. Tatarenko, Huber loss-based penalty approach to problems with linear con- straints, arXiv preprint arXiv:2311.00874, (2023). 26 WEI LIU, YANGYANG XU

    2023 arXiv

  41. [49]

    L. M. Nguyen, J. Liu, K. Scheinberg, and M. Tak ´aˇc, SARAH: A novel method for ma- chine learning problems using stochastic recursive gradient, in International Conference on Machine Learning, PMLR, 2017, pp. 2613–2621

    2017

  42. [50]

    Nocedal and S

    J. Nocedal and S. Wright , Numerical optimization , Springer Science & Business Media, 2006

    2006

  43. [51]

    G. D. Pillo and L. Grippo , Exact penalty functions in constrained optimization , SIAM Jour- nal on control and optimization, 27 (1989), pp. 1333–1360

    1989

  44. [52]

    Polyak, A general method for solving extremum problems , Soviet Mathematics

    B. Polyak, A general method for solving extremum problems , Soviet Mathematics. Doklady, 8 (1967)

    1967

  45. [53]

    Rigollet and X

    P. Rigollet and X. Tong , Neyman-pearson classification, convexity and stochastic con- straints, Journal of Machine Learning Research, (2011)

    2011

  46. [54]

    Q. Shi, X. W ang, and H. W ang, A momentum-based linearized augmented lagrangian method for nonconvex constrained stochastic optimization , Mathematics of Operations Research, (2025)

    2025

  47. [55]

    Tatarenko and A

    T. Tatarenko and A. Nedich , A smooth inexact penalty reformulation of convex problems with linear constraints, SIAM Journal on Optimization, 31 (2021), pp. 2141–2170

    2021

  48. [56]

    Vogel, A

    R. Vogel, A. Bellet, and S. Cl ´emenc ¸on, Learning fair scoring functions: Bipartite ranking under ROC-based fairness constraints, in International Conference on Artificial Intelligence and Statistics, PMLR, 2021, pp. 784–792

    2021

  49. [57]

    W ang, S

    X. W ang, S. Ma, and Y. Yuan, Penalty methods with stochastic approximation for stochastic nonlinear programming, Mathematics of Computation, 86 (2017), pp. 1793–1820

    2017

  50. [58]

    N. Xiao, K. Ding, X. Hu, and K.-C. Toh , Developing Lagrangian-based methods for non- smooth nonconvex optimization , Preprint, arXiv:2404.09438, (2024)

    2024 arXiv

  51. [59]

    Xu , Primal-dual stochastic gradient method for convex programs with many functional constraints, SIAM Journal on Optimization, 30 (2020), pp

    Y. Xu , Primal-dual stochastic gradient method for convex programs with many functional constraints, SIAM Journal on Optimization, 30 (2020), pp. 1664–1692

    2020

  52. [60]

    Xu, First-order methods for constrained convex programming based on linearized augmented Lagrangian function, INFORMS Journal on Optimization, 3 (2021), pp

    Y. Xu, First-order methods for constrained convex programming based on linearized augmented Lagrangian function, INFORMS Journal on Optimization, 3 (2021), pp. 89–117

    2021

  53. [61]

    Yan and Y

    Y. Yan and Y. Xu , Adaptive primal-dual stochastic gradient method for expectation- constrained convex stochastic programs , Mathematical Programming Computation, 14 (2022), pp. 319–363

    2022

  54. [62]

    M. Yang, G. Li, Q. Hu, Q. Lin, and T. Yang, Single-loop algorithms for stochastic non-convex optimization with weakly-convex constraints , Preprint, arXiv:2504.15243, (2025)

    2025

  55. [63]

    H. Yu, M. Neely, and X. Wei , Online convex optimization with stochastic constraints , Ad- vances in Neural Information Processing Systems, 30 (2017)

    2017

  56. [64]

    Yu and M

    H. Yu and M. J. Neely , A simple parallel algorithm with an o(1/t) convergence rate for general convex programs, SIAM Journal on Optimization, 27 (2017), pp. 759–783

    2017

  57. [65]

    Zhang and L

    J. Zhang and L. Xiao, Stochastic variance-reduced prox-linear algorithms for nonconvex com- posite optimization, Mathematical Programming, (2022), pp. 1–43. Appendix A. Proof of Lemma 2.2. Proof. Let J3 = {i : gi(x) ≥ 0}. Then by the condition on J1, it holds J1∩J3 ̸= ∅, and t...

    2022

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.