REVIEW 2 major objections 6 minor 14 references
Composite Optimization with Indicator Functions: Stationary Duality and a Semismooth Newton Method
T0 review · 2 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that minimizing a strongly convex function plus a zero-one loss can be recast as a sparse dual problem, with local minimizers in one-to-one correspondence, and that a subspace semismooth Newton method on that dual…
desk verdict The stationary-duality frame is new and the equivalence theorem holds up, but the global convergence claim is conditional on a boundedness assumption that can fail on simple valid instances, so the 'first algorithm' claim is overstated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the stationary-duality function $g(z)=\mu\lVert z\rVert_0+\delta_+(z)$, defined so that $z\in\partial I(u)$ if and only if $u\in\partial g(z)$; this replaces the classical conjugate, which is degenerate for the zero-one indicator. Its proximal operator has a closed form: each coordinate is kept when it exceeds $\sqrt{2\tau\mu}$ and otherwise set to zero, which lets SGSN identify the support $T_k$ and restrict computation to that subspace. The second ingredient is the semismooth Newton step in the reduced subspace, regularized by $\gamma_k I$ and accepted only when sufficient-descent and residual conditions hold; the P LK sharpness inequality then converts local descent into global convergence, and semismoothness plus positive definiteness of the generalized Hessian on the support subspace produces the superlinear rate.
What would settle it
Construct data with a nonzero $d\ge 0$ satisfying $A^\top d=0$ and $\langle b,d\rangle>0$, then run SGSN: $F(td)\to -\infty$ as $t\to\infty$, so no bounded sequence exists and global convergence cannot hold. To test the rate claim instead, run SGSN on an instance satisfying the semismooth and positive-definiteness assumptions and check whether the support $T_k$ eventually equals the true support and whether $\lVert z^{k+1}-z^*\rVert/\lVert z^k-z^*\rVert\to 0$; a single instance where $T_k$ keeps changing indefinitely would contradict the support-identification lemma.
Extended reading notes
Core claim
The central claim is Theorem 10: a point $w^*=(x^*,u^*)$ is a local minimizer of $\min f(x)+I(u)$ subject to $Ax+b=u$ if and only if the associated multiplier $z^*$ is a local minimizer of the dual problem $\min_z h(z)+g(z)$, where $h(z)=f^*(-A^\top z)-\langle b,z\rangle$ and $g(z)=\mu\lVert z\rVert_0+\delta_+(z)$; the correspondence is explicit, with $x^*=\nabla f^*(-A^\top z^*)$. Because the dual's nonsmooth term is a sparse regularizer with a closed-form proximal operator, a subspace can be identified cheaply, and the paper's SGSN method alternates a proximal gradient step and a regularized semismooth Newton step in that subspace. Under boundedness of the generated sequence, the P LK property, semismoothness of $\nabla f^*$, and a reduced positive-definiteness condition, the iteration converges to a P-stationary point at a superlinear rate, and the paper claims this combination is the first for Problem (1).
Load-bearing premise
Everything about global convergence rests on the assumption that the iterates $\{z^k\}$ stay bounded; the problem data alone do not imply this, and if the dual objective is unbounded below along a nonnegative direction in the nullspace of $A^\top$, the sequence can diverge and the theorem's conclusion is unavailable.
Editorial extensions
If this is right
- Solving the sparse dual yields a local minimizer of the original zero-one problem, so combinatorial search over configurations of the loss is replaced by continuous sparse optimization.
- Per-iteration cost is $O(|T_k|n)$ for the gradient step and $O(n|T_k|^2)$ for the Newton system when $f$ is separable, making problems with millions of pairs such as AUC maximization tractable.
- The method does not require $A$ to have full row rank, so it covers composite $\ell_0$ norms written as $I(Bx+b)+I(-Bx-b)$.
- Once the support of the dual solution is identified, the iteration is essentially a smooth Newton method on a fixed subspace, so numerically the final digits converge superlinearly or quadratically.
- In AUC, the dual variable lives over sample pairs and its support selects the informative pairs; in multi-label classification with elastic net, SGSN returns sparse classifiers with low Hamming loss.
Reading between the lines
- The stationary-duality construction is not tied to the specific indicator $1_{(0,\infty)}$: any loss whose limiting subdifferential has the same pattern (all nonnegative multipliers at zero, zero elsewhere) should admit an analogous $g$, so the approach may extend to other discontinuous losses.
- The dual support set is an active-set certificate. In AUC it identifies the margin pairs that constrain the solution, suggesting an online or incremental variant that adds pairs only when they are violated.
- The weakest point is boundedness: adding an explicit constraint or proximal term in the dual to force boundedness even when $h$ is unbounded below along the nullspace of $A^\top$ would make the global theorem depend only on problem data rather than on iterate behavior.
- The one-to-one correspondence plus explicit primal reconstruction $x^*=\nabla f^*(-A^\top z^*)$ gives a natural hot-starting mechanism for primal-dual methods on zero-one problems, which the paper leaves for future work.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies the nonconvex composite problem min_x f(x) + I(Ax+b), where I is a sum of zero-one indicators on positive intervals and f is strongly convex. Because the Fenchel conjugate of I is trivial, the authors construct a 'stationary dual' g(z)=μ||z||_0+δ_+(z) satisfying the subdifferential inversion relation z∈∂I(u) iff u∈∂g(z). They show that local minimizers of the primal problem correspond to local minimizers of the dual problem min_z F(z)=h(z)+g(z) with h(z)=f^*(-A^T z)-⟨b,z⟩ (Theorem 10). They then propose a subspace gradient semismooth Newton method (SGSN) that alternates a proximal gradient step, which identifies the support T_k, with a semismooth Newton step on that support. Global convergence is proved under a boundedness assumption and a PLK assumption (Theorem 16), and local superlinear convergence is proved under a semismoothness and reduced positive-definiteness assumption (Theorem 19). Numerical experiments on AUC maximization and sparse multi-label classification show competitive speed and accuracy.
Significance. The stationary duality idea is genuinely interesting: replacing the unavailable conjugate of the zero-one loss by an ℓ0-plus-nonnegativity function whose subdifferential inverts that of the indicator is a useful construction, and the main equivalence theorem is sound once a small typo in the proof of Proposition 5 is fixed. The algorithm has low per-iteration complexity, and the numerical study is extensive and largely supports the practical value of the method. However, the paper's central 'global convergence' claim is conditional on Assumption 1, which is not implied by the problem data and can fail on a valid instance with row-rank-deficient A; this weakens the claim that SGSN is the first algorithm with both global convergence and local superlinear rate for Problem (1). The local analysis under Assumption 3 appears correct and is a genuine contribution.
major comments (2)
- [§4.2, Assumption 1 and Theorem 16] The boundedness assumption on the generated sequence {z^k} is not a mild or verifiable condition, and SGSN can fail it on a valid instance even when the stationary set is nonempty. Consider f(x)=x^2/2, A=[1;-1]^T, b=(1,0)^T, μ=1, τ=0.25, γ=0.1, c1=1/6, c2=6. Here ℓ_h=2, z*=0 is a P-stationary point of (21), and x*=0 is a local minimizer of (1). Starting at z^0=(T,T), the proximal gradient step (32) gives v^0=(T+0.25,T); the Newton direction is approximately (6.46,6.22), and condition (C1) fails (F(v)-F(ez)≈6.37 while c1||ez-v||^2≈13.4), so z^1=v^0. The same pattern repeats and the iterate diverges along the nullspace ray of A^T while F(z)→-∞. Thus Assumption 1 is not implied by the data and selects exactly those runs that happen not to escape. The global convergence theorem should be restated with explicit sufficient conditions that can be checked from the problem data (for example, exclusion of b-components in the nullspace of A^T that make F unbounded below), or the algorithm should be safeguarded against such rays. As written, the abstract's and Section 1.2's claim of 'global convergence' is substantially weaker than stated.
- [Section 4.3, Theorem 19 and the 'first algorithm' claim] The local superlinear convergence result is conditional not only on Assumption 3 but also on the same Assumption 1, since the proof first uses Assumption 1 to obtain convergence of {v^k} to z* (see Remark 3(iii)). In the counterexample above, the sequence does not converge, so the local rate theorem does not apply even though the dual objective is semismooth and the reduced positive-definiteness condition is satisfied at z*=0 (vacuously, since S*=∅). The contribution statement in Section 1.2 should therefore be qualified: SGSN is the first algorithm with local superlinear rate and global convergence under an a priori boundedness condition on the iterates, not for Problem (1) in general. This is a genuine limitation, not merely a presentational issue.
minor comments (6)
- [Proposition 5, equation (20)] The displayed inclusion {i:u_i>0} ⊇ J_-^* appears to have the wrong set. To justify I(u)≥I(u*) for u near u*, one needs all indices with u*_i>0 to remain positive, i.e., {i:u_i>0} ⊇ [m]\J_-^*. As written, the proof's claim in the F*∩N(w*,δ*) case does not follow, although the argument is easily repaired.
- [Lemma 1, proof of part (iv)] At the end of the proof of part (iv), the text says 'we complete the proof of (iii)' but should say '(iv)'. This is a minor cross-reference error.
- [Theorem 19 heading] The heading 'converges to z* with local superliner rate' contains a typo; it should read 'superlinear'.
- [Section 1.2, Theorem 10 description] The phrase 'one-to-one correspondence between local minimizers' is stronger than what Theorem 10 establishes. The theorem gives mutual implication with a recovery formula, but when the KKT multiplier is nonunique, the correspondence need not be a literal bijection between solution sets. The wording should be softened to 'equivalence'.
- [Section 5.1, parameter setting] The experiments set c2=3ℓ_h, whereas Lemma 18(iii) proves the Newton acceptance condition under c2≥ℓ_h+η4+1, which in the AUC setting is 3ℓ_h+1. The numerical choice slightly violates the proven sufficient condition; either the condition should be stated with a smaller constant or the experiment should use the theory-consistent value.
- [Table 5] The column headers 'RSF', 'LLSF', and 'MFDS' are typos for RFS, LLFS, and MDFS used elsewhere in the text and in Table 3.
Circularity Check
The primal–dual stationarity equivalence is encoded in the definition of g; SGSN convergence is independent, keeping circularity partial.
-
self definitional
[Section 1.2, Eqs. (3)-(5); Lemma 1(iii); Theorem 10 proof]
"We would like to find a function g(·) that plays the role of I∗(·) and satisfies the stationary duality property: z∈∂I(u)⇐⇒u∈∂g(z), (3) ... Extension of the one-dimensional case to the multi-dimensional case leads to the following function: g(z):=µ∥z∥0+δ+(z), (4) ... Using the characterizations in (6) and (7), it is easy to verify that the stationary dual relationship (3) holds for g(·)."
The dual function g is explicitly chosen so that the subdifferential inversion (3) holds. Lemma 1(iii) then verifies (3) by comparing formulas (6) and (7), and Theorem 10's proof concludes the primal-dual KKT equivalence with 'This directly follows from Lem. 1(iii).' Thus the claimed one-to-one correspondence between primal and dual local minimizers is not an independent duality theorem; it is a property baked into the definition of g and the dual objective (5). The local-minimizer characterizations in Propositions 5 and 8 are genuine, but the bridge between the two problems reduces to the designed identity (3), so this part of the derivation is self-definitional rather than independently derived.
full rationale
The paper's central algorithmic contribution, SGSN, is not circular: its descent, global convergence, and local superlinear analysis (Prop. 13, Lem. 18, Thms. 16 and 19) are genuine proofs under standard conditions, and the numerical comparisons are against external solvers. The circularity is confined to the 'stationary duality' foundation. The authors openly construct g so that z∈∂I(u) iff u∈∂g(z), and the dual problem is defined with that g. Consequently, the equivalence of KKT points in Theorem 10 is a restatement of the construction rather than a consequence of a pre-existing dual theory. This is a legitimate design choice, but it means the paper's claim of 'establishing the equivalence between primal and dual solutions' is weaker than a derivation from independent principles. The Assumption 1 boundedness issue raised in the reader's take is a correctness/robustness concern about the global-convergence theorem, not a circularity; similarly, Assumption 3 is a standard second-order condition. Overall, the duality equivalence reduces by construction, while the algorithmic and experimental content remains independent, justifying a partial-circularity score of 5.
Assumptions & free parameters
free parameters (3)
- μ (dual ℓ0 penalty) =
1/(2ℓ_h) in AUC experiments; 10^{-5} in MLC experiments
- τ (proximal gradient step length) =
1/(2ℓ_h) in AUC; adaptive backtracking in MLC
- Algorithm constants γ, c1, c2 =
γ=10^{-1} (AUC), 10^{-2} (MLC); c1=1/(3ℓ_h) or 10^{-4}; c2=3ℓ_h or 10^8
assumptions (5)
- standard math Standard variational analysis results: limiting subdifferential calculus, Fermat's rule, proximal behavior theorem, PLK property, semismooth calculus.
- domain assumption Problem data assumptions: f is σ_f-strongly convex (possibly nonsmooth), A and b are given, I(u)=λΣ1_{>0}(u_i).
- ad hoc to paper Assumption 1: the sequence {z^k} generated by SGSN is bounded.
- domain assumption Assumption 2: F is a PLK function.
- ad hoc to paper Assumption 3: ∇f* is semismooth and the reduced generalized Hessian is positive definite on the identified support (49).
invented entities (1)
-
Stationary dual function g(z)=μ||z||_0+δ_+(z)
Cite this review
Pith. "Pith review of Composite Optimization with Indicator Functions: Stationary Duality and a Semismooth Newton Method." pith.science (2026). https://pith.science/paper/25VSCTQW
@misc{pith2026250608374,
author = {Pith},
title = {Pith review of: Composite Optimization with Indicator Functions: Stationary Duality and a Semismooth Newton Method},
year = {2026},
howpublished = {\url{https://pith.science/paper/25VSCTQW}},
note = {Machine review of arXiv:2506.08374}
}
abstract
Indicator functions of taking values of zero or one are essential to numerous applications in machine learning and statistics. The corresponding primal optimization model has been researched in several recent works. However, its dual problem is a more challenging topic that has not been well addressed. One possible reason is that the Fenchel conjugate of any indicator function is finite only at the origin. This work aims to explore the dual optimization for the sum of a strongly convex function and a composite term with indicator functions on positive intervals. For the first time, a dual problem is constructed by extending the classic conjugate subgradient property to the indicator function. This extension further helps us establish the equivalence between the primal and dual solutions. The dual problem turns out to be a sparse optimization with a $\ell_0$ regularizer and a nonnegative constraint. The proximal operator of the sparse regularizer is used to identify a dual subspace to implement gradient and/or semismooth Newton iteration with low computational complexity. This gives rise to a dual Newton-type method with both global convergence and local superlinear (or quadratic) convergence rate under mild conditions. Finally, when applied to AUC maximization and sparse multi-label classification, our dual Newton method demonstrates satisfactory performance on computational speed and accuracy.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
[1]F. J. Arag ´on-Artacho, B. S. Mordukhovich, and P. P ´erez-Aros,Coderivative- based semi-Newton method in nonsmooth difference programming, Math. Program., (2024), pp. 1–48. [2]H. Attouch, J. Bolte, P. Redont, and A. Soubeyran,Proximal alternating min- imization and projection methods for nonconvex problems: An approach based on the Kurdyka- Lojasiewic...
work page 2024
-
[295]
[28]J. A. Hanley and B. J. McNeil,The meaning and use of the area under a receiver operating characteristic (ROC) curve., Radiology, 143 (1982), pp. 29–36. 38 [29]J. Huang, G. Li, Q. Huang, and X. Wu,Learning label-specific features and class- dependent labels for multi-label classification, IEEE Trans. Knowl. Data Eng., 28 (2016), pp. 3309–3323. [30]C. K...
arXiv 1982
-
[297]
[19]Y. Cui, J. Liu, and J.-S. Pang,The minimization of piecewise functions: pseudo sta- tionarity, arXiv preprint arXiv:2305.14798, (2023). [20]Y. Cui and J.-S. Pang,Modern nonconvex nondifferentiable optimization, SIAM,
arXiv 2023
-
[737]
[49]F. Nie, H. Huang, X. Cai, and C. Ding,Efficient and robust feature selection via joint ℓ2,1-norms minimization, Advances in neural information processing systems, 23 (2010). [50]J. Nocedal and S. Wright,Numerical optimization, Springer series in operations re- search and financial engineering, Springer, New York,
work page 2010
- [1987]
-
[1997]
[41]B. S. Mordukhovich,Maximum principle in the problem of time optimal response with nonsmooth constraints, J. Appl. Math. Mech., 40 (1976), pp. 960–969. [42]B. S. Mordukhovich,Sensitivity analysis in nonsmooth optimization, Theor. Asp. Ind. Des., 58 (1992), pp. 32–46. [43]B. S. Mordukhovich,Variational Analysis and Generalized Differentiation. I. Basic ...
work page 1976
-
[1998]
[58]K. Sechidis, G. Tsoumakas, and I. Vlahavas,On the stratification of multi-label data, in Machine Learning and Knowledge Discovery in Databases: European Conference, Springer, 2011, pp. 145–158. [59]P. K. Shivaswamy and T. Jebara,Maximum relative margin and data-dependent regu- larization., J. Mach. Learn. Res., 11 (2010), pp. 747–788. [60]V. V apnik,T...
work page 2010
-
[1999]
[61]F. W ang, W. Cao, and Z. Xu,Convergence of multi-block Bregman ADMM for noncon- vex composite problems, Sci. China Inform. Sci., 61 (2018), pp. 1–12. [62]G. Wu, Y. Tian, and C. Zhang,A unified framework implementing linear binary rele- vance for multi-label learning, Neurocomputing, 289 (2018), pp. 86–100. [63]W. Xie and H. Yang,Group sparse recovery ...
work page 2018
Show all 14 references
-
[2003]
[22]M. Feng, J. E. Mitchell, J.-S. Pang, X. Shen, and A. W ¨achter,Complementarity formulations ofℓ 0-norm optimization problems, Pac. J. Optim., 14 (2018), pp. 273–305. [23]W. Gao, R. Jin, S. Zhu, and Z.-H. Zhou,One-pass AUC optimization, in International conference on machin...
2018
-
[2006]
Poliquin and R
[51]R. Poliquin and R. T. Rockafellar,Tilt stability of a local minimum, SIAM J. Optim., 8 (1998), pp. 287–299. [52]B. T. Polyak,Gradient methods for the minimisation of functionals, USSR Comput. Math. Math. Phys., 3 (1963), pp. 864–878. [53]B. T. Polyak,Sharp minima, Institut...
1998
-
[2009]
Cortes and V
[18]C. Cortes and V. V apnik,Support-vector networks, Mach. Learn., 20 (1995), pp. 273–
1995
-
[2013]
Bolte, A
[7]J. Bolte, A. Daniilidis, A. Lewis, and M. Shiota,Clarke subgradients of stratifiable functions, SIAM J. Optim., 18 (2007), pp. 556–572. [8]J. Bolte, S. Sabach, and M. Teboulle,Proximal alternating linearized minimization for nonconvex and nonsmooth problems, Math. Program.,...
2007
-
[2017]
Beck and M
[4]A. Beck and M. Teboulle,A fast dual proximal gradient algorithm for convex mini- mization and applications, Oper. Res. Lett., 42 (2014), pp. 1–6. [5]G. Bento, B. Mordukhovich, T. Mota, and Y. Nesterov,Conver- gence of descent optimization algorithms under Polyak- Lojasiewic...
2014
-
[2024]
[46]B. S. Mordukhovich and M. E. Sarabi,Generalized Newton algorithms for tilt-stable minimizers in nonsmooth optimization, SIAM J. Optim., 31 (2021), pp. 1184–1214. [47]B. S. Mordukhovich, X. Yuan, S. Zeng, and J. Zhang,A globally convergent prox- imal Newton-type method in n...
2021
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.