Pith. sign in

REVIEW 4 major objections 5 minor 54 references

Difference-of-Convex Regularization for Graph Learning by Differentiable Programming

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DCR replaces the dense graph Laplacian pseudoinverse with a learnable spectral approximation and reconstructs nonnegative least-squares solutions faster than direct convex solvers.

desk verdict A novel Phase 1 for learning L† via Tyler estimation, but Phase 2 fits the CVXPY reference directly, so the speedup claims don't hold. read the letter →

arxiv 2608.12757 v1 pith:6AQ7EXOS submitted 2026-08-13 math.OC cs.LG

Liping Tao , Chee Wei Tan This is my paper · ORCID
classification math.OCcs.LG MSC 90C2690C2505C50
keywords graphLaplacianpseudoinversedifference-of-convexregularizationTyler'sestimatorCCCPLaplacian-regularizednonnegativeleastsquaresTolanddualitydifferentiableprogramminglearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that the dense, ill-conditioned graph Laplacian pseudoinverse can be replaced by a learnable approximation that is spectrally accurate on the Laplacian's range space, so that Laplacian-regularized nonnegative least squares can be solved without ever forming $L^\dagger$. The proposed two-phase scheme first learns $\tilde L^\dagger$ through regularized maximum likelihood estimation written as a difference-of-convex program and solved by CCCP iterations, then uses a differentiable dual-guided map to reconstruct the primal solution. If the claim holds, the practical payoff is a graph-learning method whose online cost is a few matrix-vector products instead of an $O(n^3)$ pseudoinverse, with speedups over a standard convex solver that grow with problem size.

What carries the argument

The central object is the two-phase DCR pipeline. Phase 1 learns $\tilde L^\dagger = Q \sqrt{\Sigma^{-1}_\star} Q^\top$, where $Q$ is an orthonormal basis of the Laplacian range space and $\Sigma_\star$ solves a scale-invariant regularized MLE of Tyler type; the objective is written as a difference of convex functions, $f(X) - h(X)$, and solved by CCCP, whose update is the fixed-point iteration (34), equivalently a Frank–Wolfe step. Phase 2 uses the dual representation $x(\lambda,\mu,c) = \tilde L^\dagger(\mu - A^\top \lambda) + c\mathbf{1}$ as a fully differentiable map, trains $(\lambda,\tilde\mu,c)$ against a reference solution with loss (58), then returns the projected reconstruction (59). The load-bearing identities are the KKT solution formula (11) and the Tyler-type fixed-point mapping (43), whose nonlinear Perron–Frobenius structure gives existence and stability guarantees.

What would settle it

Take a fixed graph and a new pair $(A,b)$ not used in Phase 2 training, run only Phase 2 from random dual initialization with the learned $\tilde L^\dagger$, and compare $x_{\mathrm{DCR}}$ against the true optimum from an independent solver: the decoupling claim collapses if accuracy degrades materially across instances. A sharper test is to check the KKT residual of the learned dual variables—if $\|L x_{\mathrm{DCR}} + A^\top \lambda^\star - \mu^\star\|$ stays far from zero while the objective gap is small, the reconstruction is fitting the reference rather than solving the problem.

Watch

Extended reading notes

Core claim

For the LR-NNLS problem (3), the paper derives from the KKT conditions that any primal solution has the form $x^\star = L^\dagger(\mu^\star - A^\top \lambda^\star) + c\mathbf{1}$, with the constant $c$ determined by complementary slackness. It then constructs $\tilde L^\dagger$ as an approximation of $L^\dagger$ on the range space of $L$, learned by a scale-invariant regularized MLE of Tyler type whose stationary point is reached by a CCCP fixed-point iteration, and shows that Phase 2's differentiable training of $(\lambda, \mu, c)$ against a high-accuracy reference solution yields a reconstructed $x_{\mathrm{DCR}} = [\tilde L^\dagger(\mu^\star - A^\top\lambda^\star) + c^\star \mathbf{1}]_+$ that matches the convex solver's solution to high accuracy. The paper also establishes fixed-point existence and stability guarantees for the iteration. Numerically, on grid2d and Erdős–Rényi graphs with $n$ up to 2000, DCR reaches a $10^{-3}$ relative objective gap in seconds and achieves speedups up to 13.2$\times$ over CVXPY, while the Chebyshev polynomial baseline fails to reach the threshold.

Load-bearing premise

The method's online accuracy depends on having a high-accuracy reference solution from a convex solver to train the dual variables; without such a reference, the paper provides no objective for choosing $(\lambda,\mu,c)$ at inference, so DCR cannot yet act as a standalone solver for a new instance.

Editorial extensions

If this is right

  • For a fixed graph, Phase 1's learned $\tilde L^\dagger$ can be reused across many instances $(A,b)$, amortizing the expensive pseudoinverse approximation.
  • The dual-guided reconstruction turns primal recovery into a differentiable program whose dimension is $O(m+n)$, not $O(n^2)$.
  • Accuracy holds on both grid2d and Erdős–Rényi topologies, indicating the learned operator adapts to spectral structure rather than relying on a fixed polynomial approximation.
  • Time-to-accuracy grows slowly with $n$, and speedups over CVXPY increase with problem size, so the method becomes more attractive exactly where direct inversion is prohibitive.
  • The Chebyshev baseline's large, irreducible error suggests fixed polynomial approximations of $L^\dagger$ carry a bias that a learned spectral approximation can remove.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because Phase 2 supervises the dual variables with a high-accuracy reference solution, the pipeline is not yet a standalone solver for a fresh instance; a natural test is zero-shot transfer of the trained dual map to unseen $(A,b)$ on the same graph.
  • The same KKT structure extends beyond NNLS: any Laplacian-regularized problem whose solution is characterized through $L^\dagger$ and constraint duals could reuse the learned operator, so DCR may port to ranking, nonnegative matrix factorization, and hyperspectral unmixing.
  • One could replace the reference-supervised loss with an unsupervised objective that minimizes the LR-NNLS objective directly in the dual parameters, removing the dependence on a convex solver.
  • A stress test is whether the learned dual variables satisfy the KKT conditions exactly or only approximately in a way the final nonnegativity projection hides.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a Difference-of-Convex Regularizer (DCR) framework for Laplacian-regularized nonnegative least squares (LR-NNLS), with the Laplacian given. The method has two phases. Phase 1 learns an approximation of the Laplacian pseudoinverse on the range space via a regularized Tyler-type MLE solved by a CCCP fixed-point iteration. Phase 2 learns auxiliary dual variables (λ, μ, c) by minimizing the squared distance between the reconstruction x(λ, μ, c) = L~†(μ − A^T λ) + c1 and a high-accuracy CVXPY reference solution x*_cvxpy, followed by a final nonnegativity projection. The paper states theoretical guarantees on fixed-point existence and reports numerical experiments showing that DCR achieves high solution accuracy and speedups over CVXPY and Chebyshev baselines across graph topologies and problem sizes.

Significance. The paper contains a few genuinely useful components: the KKT-based primal reconstruction formula in Theorem 1 is clean and correctly identifies the role of the null-space constant; the connection between CCCP and Frank-Wolfe in Remark 1 is a nice observation; the Phase 1 MLE formulation for approximating the spectral action of the Laplacian pseudoinverse is interesting and is validated on a small example against a CVXPY baseline. The authors also state that source code is available. However, the central claim that DCR is a fast solver for (3) that outperforms convex solvers is not supported by the evidence: Phase 2 is a supervised fitting procedure trained on the CVXPY solution of the very same instance, so the reported accuracy and speedups do not demonstrate that (3) is solved faster. The theoretical contribution is also weaker than advertised, as only existence, not uniqueness, of fixed points is proved. If the paper were reframed as a learning-to-predict method conditioned on reference solutions, the significance would be different, but as a solver the contribution is not established.

major comments (4)
  1. [Section IV-C, Eq. (58), Algorithm 1 lines 15–18] The Phase 2 objective is min_{λ, μ~, c} ||x(λ, φ(μ~), c) − x*_cvxpy||², where x*_cvxpy is the CVXPY solution of the same instance and x(λ, μ, c) = L~†(μ − A^T λ) + c1. Consequently, the reported relative solution errors and objective gaps in Table III and Figs. 5–7 are training errors, not measures of how accurately (3) is solved. The method also provides no rule for selecting (λ, μ, c) at inference time for a new instance without a reference solution, so DCR, as presented, is not a standalone solver for (3). This circularity is the central issue and it invalidates the paper's main claim of improved performance over convex solvers.
  2. [Section V-A, runtime metric, and Fig. 6] The runtime comparison counts only Phase 2 time, with the text stating 'For DCR, only Phase 2 (online) time is counted; Phase 1 is excluded as it is amortized across instances with the same graph.' However, Phase 2 requires the CVXPY solution of the same instance as its training target (Algorithm 1 line 15), so the full pipeline cost is at least the CVXPY solve time plus the Phase 2 optimization time. The reported speedups of up to 13.2× over CVXPY therefore do not reflect end-to-end wall-clock performance and are not a fair comparison with a direct solver.
  3. [Abstract and Theorems 2–3] The abstract claims 'the existence of a unique fixed point' for the DCR algorithm, but Theorem 2 only establishes existence of an eigenvector fixed point α*Σ* for M_γ and hence a fixed point of the normalized map; it does not prove uniqueness. Theorem 3 similarly establishes at least one fixed point via Brouwer's theorem. No uniqueness result is proved anywhere in the manuscript, so the abstract overstates the theoretical guarantee. This is secondary to the Phase 2 circularity but still requires correction.
  4. [Section IV-A2 and Eq. (60)] The number of random samples used in Phase 1 is set to K(n) = min(Kmax, max(Kmin, ⌈2√n⌉)) in Eq. (60), but Section IV-A2 justifies the method only by saying that 'with sufficiently large K, the set {r_k} spans R(L) with high probability.' For n = 2000, this gives K ≈ 89 samples to approximate a pseudoinverse on an (n−1)-dimensional range space, which is far below any standard sample-complexity guarantee for covariance or Tyler-type estimation. Because Phase 2 fits the reconstruction to the reference solution, the experiments do not isolate whether such a small K actually provides a faithful approximation of L†; this weakens the claimed decoupling of pseudoinverse learning from instance-specific inference.
minor comments (5)
  1. [Section IV-C, line 16] The symbol L is used both for the graph Laplacian and for the training loss in Algorithm 1 line 15 and in the text 'Update (λ, μ~, c) by one gradient step on L'; this overloading is confusing and should be resolved by renaming the loss, for example as J or ℓ.
  2. [Fig. 5 caption] The caption states that the vertical dashed line marks the CVXPY solve time, but the DCR curves begin at Phase 2 start; a note that Phase 1 time is excluded from the DCR curves would help the reader interpret the comparison.
  3. [Section V-A, Chebyshev baseline] The Chebyshev baseline is described as being 'integrated into a gradient-based iterative solver for (3)', but no details are given on the step-size rule or number of inner iterations, making it hard to assess whether the comparison is favorable to DCR or the baseline.
  4. [Throughout] Several equations and displayed expressions are typeset with missing or ambiguous spacing (e.g., Eq. (17) and the normalization in Theorem 2), which makes the mathematics harder to read; a careful proofreading pass would improve clarity.
  5. [References] References [29] and [31] are arXiv preprints dated 2026; if these are not yet published, the authors should indicate whether they are under review or should cite accessible versions.

Circularity Check

2 steps flagged · score 8.0 of 10

Phase 2 is supervised fitting against the CVXPY reference, so the reported accuracy and speedups measure the training loss; no inference rule for (λ, μ, c) exists without x*_cvxpy.

  1. fitted input called prediction [Section IV-C 'Dual-Driven Differentiable Primal Reconstruction', Eq. (58); Algorithm 1 lines 11-18]
    "With the ˜L† fixed from previous phase, we learn auxiliary dual variables that induce a primal candidate matching a high-accuracy reference solution x ⋆ cvxpy. Specifically, we solve the smooth differentiable training objective: min λ,˜µ,c L(λ,˜µ,c) = x(λ,ϕ(˜µ),c)−x ⋆ cvxpy 2 2."

    Algorithm 1 line 15 evaluates exactly this loss against x*_cvxpy, line 17 refits c to x*_cvxpy, and line 18 returns the projected fit. Therefore the relative solution errors and objective gaps in Table III and Figs. 5-8 measure the very quantity that Phase 2 explicitly minimizes on the same instance. The agreement with CVXPY is enforced by construction, not independently predicted. Moreover, for a new instance b there is no mechanism to select (λ, μ, c) without either x*_cvxpy or another solve, so DCR is not a standalone solver for (3).

  2. fitted input called prediction [Section V-A 'Experimental Setup', runtime metric; Section V-B 'Runtime and Speedup Analysis']
    "High-accuracy reference solutions are obtained via the convex solver CVXPY [41]. ... For DCR, only Phase 2 (online) time is counted; Phase 1 is excluded as it is amortized across instances with the same graph."

    The speedup claim compares only Phase 2 fitting time with the full CVXPY solve, but Phase 2 is defined as fitting to the CVXPY output: Algorithm 1 line 15 uses x*_cvxpy as the loss target and line 17 fits c to it. The end-to-end cost for a new instance is therefore at least the CVXPY solve time plus Phase 2. Counting only Phase 2 makes the reported speedups (e.g., 13.2× in Table III) a comparison of a supervised fitting stage against the very solver that generated its labels, not an end-to-end solve-time comparison.

full rationale

Phase 1 (regularized MLE/CCCP pseudoinverse approximation) is self-contained: it learns L̃† from random samples r_k = Lz_k on R(L) without any reference primal solution, and Example 2 validates it against the MLE objective with normalized Frobenius inner product 1.0000. This independent content prevents a score of 10. The circularity is concentrated in Phase 2 and in the runtime accounting. Eq. (58) minimizes ||x(λ,ϕ(µ̃),c)−x*_cvxpy||², and Algorithm 1 lines 15-18 use that same reference for the loss, for the c-refit, and for the returned projection [·]_+. Consequently, the solution errors and objective gaps in Table III and Figs. 5-8 are training losses on the same instance, not independent predictions. The runtime comparison compounds the problem: only Phase 2 time is counted, but Phase 2 requires the CVXPY solution as its label, so the reported speedups do not represent a faster end-to-end solve. The abstract's 'unique fixed point' phrase also exceeds Theorems 2-3, which prove existence (NPF eigenvector in Theorem 2 and Brouwer fixed point in Theorem 3) but not uniqueness; this is a correctness overclaim rather than a circular step. No load-bearing self-citation was found: the nonlinear Perron-Frobenius citation is standard external mathematics also given in reference [51]. Overall the central accuracy and speed claims reduce, at least partially, to a supervised fit against the benchmark they claim to outperform.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The Phase 1 estimator introduces free hyperparameters γ, ρ, K whose values are not fully specified, and relies on standard Tyler-consistency plus fixed-point theorems. No new physical or model entities are introduced.

free parameters (6)
  • gamma
    Regularization weight in (18) controlling spectral shrinkage; no value is reported in Table II or the text.
  • rho
    Shrinkage weight in (50)-(51), adaptively set by (61) with unspecified ρ_plugin, c_γ, ρ_min, ρ_max.
  • K
    Number of random directions K(n)=min(Kmax, max(Kmin, round(2√n))) in (60); Kmin and Kmax values are not specified.
  • softplus_temperature_beta = 0.5
    Softplus temperature in the reparameterization μ=ϕ(μ~), listed in Table II.
  • phase2_learning_rate_eta_D = 5e-3
    Adam learning rate for dual variables in Phase 2, listed in Table II.
  • phase1_iterations = min{3000, 500 + 2(n-1)}
    CCCP iteration count in Phase 1, listed in Table II.
assumptions (4)
  • domain assumption Connected graph Laplacian has one-dimensional null space span{1}
    Used in Theorem 1 to write x* = L†(...) + c1; standard for connected graphs but not stated for disconnected graphs.
  • domain assumption Tyler's estimator consistency: normalized random directions Lz/||Lz|| identify the scatter matrix L^2 on R(L)
    The paper relies on E[r r^T] = L^2 and asymptotic Tyler consistency to claim Σ* shares eigenvectors with L; finite-sample and undersampled validity is not proved.
  • standard math Nonlinear Perron-Frobenius and Brouwer fixed-point theorems
    Used in Theorems 2 and 3 to establish existence of fixed points for the normalized CCCP mapping.
  • ad hoc to paper K random directions with K≈2√n suffice to approximate L† on R(L) at scale
    The experiments use K << n-1, while Theorem 2 requires sum_k y_k y_k^T ≻ 0, which needs at least n-1 directions; no theoretical justification is given for the undersampled regime.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Difference-of-Convex Regularization for Graph Learning by Differentiable Programming." pith.science (2026). https://pith.science/paper/6AQ7EXOS

@misc{pith2026260812757,
  author       = {Pith},
  title        = {Pith review of: Difference-of-Convex Regularization for Graph Learning by Differentiable Programming},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6AQ7EXOS}},
  note         = {Machine review of arXiv:2608.12757}
}
read the original abstract

Laplacian-regularized minimization is fundamental in signal processing and machine learning, but is limited by the dense and ill-conditioned nature of the graph Laplacian pseudoinverse. While the Laplacian itself is sparse, its pseudoinverse is dense and often ill-conditioned, rendering direct computation impractical at scale. Moreover, pseudoinverse learning is more challenging than Laplacian learning. To address this challenge, this paper considers the setting where the graph Laplacian is given and proposes a Difference-of-Convex Regularizer (DCR) graph learning framework that approximates the spectral action of the Laplacian pseudoinverse without direct inversion via regularized Maximum Likelihood Estimation (MLE). By reformulating Laplacian-Regularized Nonnegative Least Squares (LR-NNLS) through a dual representation, DCR decouples pseudoinverse learning from instance-specific inference and enables efficient primal solution reconstruction via a differentiable dual-guided learning scheme. We establish theoretical guarantees on stability and the existence of a unique fixed point for DCR algorithm. Numerical experiments demonstrate improved performance over convex solvers and graph filtering baselines and robust performance across diverse graph topologies.

Figures

Figures reproduced from arXiv: 2608.12757 by the authors.

Figure 1
Figure 1. While the graph Laplacian L is sparse, its pseudoinverse L† is dense, rendering direct computation memory- and compute-intensive at scale. and interpretability. A representative example is the Laplacian￾Regularized Nonnegative Least-Squares (LR-NNLS) problem: min x⪰0 1 2 ∥Ax − b∥ 2 2 + 1 2 x ⊤Lx, (2) which is widely used in applications such as semi-supervised ranking on graphs [3] and graph-guided nonnegative codin… view at source ↗
Figure 2
Figure 2. Spectral and convergence properties of the LR-NNLS problem ( [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. DCR algorithm solves (3) in two phases. Phase 1 learns a Laplacian pseudoinverse approximation L˜† via regularized MLE and CCCP iteration with shrinkage regularization, while Phase 2 recovers the primal solution through dual-driven differentiable learning. The graph Laplacian L is symmetric positive semidefinite with L1 = 0, where 1 ∈ R n denotes the all-ones vector. For a connected graph, N (L) = span{1} is one-dim… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: MLE on R(L). (a) Σ−1 follows the theoretical eigenvalue scaling of (L2 ) † on R(L). (b): after L˜† = √ Σ−1, L˜† aligns with L† in spectral coordinates, confirming spectrum approximation of L† . Theorem 3. P Let ε > 0, ρ ∈ (0, 1], and γ ≥ 0. Assume K k=1 yky ⊤ k ≻ 0 and…
Figure 5
Figure 5. Figure 5: Convergence curves (objective gap vs. solve time) for DCR and Chebyshev on the LR-NNLS problem ( [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Runtime and speedup comparison of DCR and Chebyshev relative to [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 8
Figure 8. Figure 8: Signal reconstruction comparison between DCR and CVXPY on [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Convergence comparison between the CVXPY baseline and the fixed [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 37 canonical work pages

  1. [1]

    Distributed majorization-minimization for Laplacian regularized problems,

    J. Tuck, D. Hallac, and S. Boyd, “Distributed majorization-minimization for Laplacian regularized problems,”IEEE/CAA Journal of Automatica Sinica, vol. 6, no. 1, pp. 45–52, 2019

    work page 2019

  2. [2]

    Laplacian regularized low-rank represen- tation and its applications,

    M. Yin, J. Gao, and Z. Lin, “Laplacian regularized low-rank represen- tation and its applications,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 3, pp. 504–517, 2016

    work page 2016

  3. [3]

    Semi-supervised ranking on very large graphs with rich metadata,

    B. Gao, T.-Y . Liu, W. Wei, T. Wang, and H. Li, “Semi-supervised ranking on very large graphs with rich metadata,” inProceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2011, pp. 96–104

    work page 2011

  4. [4]

    Representing docu- ments through their readers,

    K. El-Arini, M. Xu, E. B. Fox, and C. Guestrin, “Representing docu- ments through their readers,” inProceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2013, pp. 14–22

    work page 2013

  5. [5]

    A graph Laplacian regular- ization for hyperspectral data unmixing,

    R. Ammanouil, A. Ferrari, and C. Richard, “A graph Laplacian regular- ization for hyperspectral data unmixing,” in2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2015, pp. 1637–1641

    work page 2015

  6. [6]

    Graph regularized nonnegative matrix factorization for data representation,

    D. Cai, X. He, J. Han, and T. S. Huang, “Graph regularized nonnegative matrix factorization for data representation,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 8, pp. 1548– 1560, 2010

    work page 2010

  7. [7]

    Laplacian regularized nonnegative representation for clustering and dimensionality reduction,

    Y .-P. Zhao, L. Chen, and C. P. Chen, “Laplacian regularized nonnegative representation for clustering and dimensionality reduction,”IEEE Trans- actions on Circuits and Systems for Video Technology, vol. 31, no. 1, pp. 1–14, 2020

    work page 2020

  8. [8]

    Boyd and L

    S. Boyd and L. Vandenberghe,Convex Optimization. Cambridge University Press, 2004

    2004

Show all 54 references
  1. [9]

    Incremental computation of pseudoinverse of Laplacian,

    G. Ranjan, Z.-L. Zhang, and D. Boley, “Incremental computation of pseudoinverse of Laplacian,” inInternational Conference on Combina- torial Optimization and Applications. Springer, 2014, pp. 729–749

    2014

  2. [10]

    Maintainability and scal- ability in machine learning: Challenges and solutions,

    K. Shivashankar, G. Al Hajj, and A. Martini, “Maintainability and scal- ability in machine learning: Challenges and solutions,”ACM Computing Surveys, vol. 57, no. 12, 2025

    2025

  3. [11]

    Wavelets on graphs via spectral graph theory,

    D. K. Hammond, P. Vandergheynst, and R. Gribonval, “Wavelets on graphs via spectral graph theory,”Applied and Computational Harmonic Analysis, vol. 30, no. 2, pp. 129–150, 2011. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 12

    2011

  4. [12]

    Dis- tributed signal processing via Chebyshev polynomial approximation,

    D. I. Shuman, P. Vandergheynst, D. Kressner, and P. Frossard, “Dis- tributed signal processing via Chebyshev polynomial approximation,” IEEE Transactions on Signal and Information Processing over Networks, vol. 4, no. 4, pp. 736–751, 2018

    2018

  5. [13]

    Iterative polynomial approximation algorithms for inverse graph filters,

    C. Cheng, Q. Sun, and C. Zheng, “Iterative polynomial approximation algorithms for inverse graph filters,” in2025 International Conference on Sampling Theory and Applications (SampTA). IEEE, 2025, pp. 1–5

    2025

  6. [14]

    Graph convolutions enrich the self-attention in transformers!

    J. Choi, H. Wi, J. Kim, Y . Shin, K. Lee, N. Trask, and N. Park, “Graph convolutions enrich the self-attention in transformers!”Advances in Neural Information Processing Systems, vol. 37, pp. 52 891–52 936, 2024

    2024

  7. [15]

    Learning spectral graph transforma- tions for link prediction,

    J. Kunegis and A. Lommatzsch, “Learning spectral graph transforma- tions for link prediction,” inProceedings of the 26th Annual Interna- tional Conference on Machine Learning, 2009, pp. 561–568

    2009

  8. [16]

    Learning the kernel via convex optimization,

    S.-J. Kim, A. Zymnis, A. Magnani, K. Koh, and S. Boyd, “Learning the kernel via convex optimization,” in2008 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 2008, pp. 1997– 2000

    2008

  9. [17]

    Graph learning,

    F. Xia, C. Peng, J. Ren, F. G. Febrinanto, R. Luo, V . Saikrishna, S. Yu, and X. Kong, “Graph learning,”Foundations and Trends® in Signal Processing, vol. 19, no. 4, pp. 362–519, 2026

    2026

  10. [18]

    Duality in DC (difference of convex functions) optimization. Subgradient methods,

    P. D. Tao and E. B. Souad, “Duality in DC (difference of convex functions) optimization. Subgradient methods,” inTrends in Mathe- matical Optimization: 4th French-German Conference on Optimization. Springer, 1988, pp. 277–293

    1988

  11. [19]

    Disciplined convex-concave programming,

    X. Shen, S. Diamond, Y . Gu, and S. Boyd, “Disciplined convex-concave programming,” in2016 IEEE 55th Conference on Decision and Control (CDC). IEEE, 2016, pp. 1009–1014

    2016

  12. [20]

    A globally convergent difference-of-convex algorithmic framework and application to log-determinant optimization problems,

    C. Yao and X. Jiang, “A globally convergent difference-of-convex algorithmic framework and application to log-determinant optimization problems,”arXiv preprint arXiv:2306.02001, 2023

    2023 arXiv

  13. [21]

    Minimizing oracle-structured composite functions,

    X. Shen, A. Ali, and S. Boyd, “Minimizing oracle-structured composite functions,”Optimization and Engineering, vol. 24, no. 2, pp. 743–777, 2023

    2023

  14. [22]

    Computing one-bit compressive sensing via zero-norm regularized DC loss model and its surrogate,

    K. Chen, L. Liang, and S. Pan, “Computing one-bit compressive sensing via zero-norm regularized DC loss model and its surrogate,”Journal of Global Optimization, vol. 92, no. 3, pp. 775–807, 2025

    2025

  15. [23]

    Direct-optimization-based DC dictionary learning with the MCP regularizer,

    Z. Li, Z. Yang, H. Zhao, and S. Xie, “Direct-optimization-based DC dictionary learning with the MCP regularizer,”IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 7, pp. 3568–3579, 2021

    2021

  16. [24]

    A DC optimization algorithm for solving the trust-region subproblem,

    P. D. Tao and L. T. H. An, “A DC optimization algorithm for solving the trust-region subproblem,”SIAM Journal on Optimization, vol. 8, no. 2, pp. 476–505, 1998

    1998

  17. [25]

    Duality in nonconvex optimization,

    J. F. Toland, “Duality in nonconvex optimization,”Journal of Mathe- matical Analysis and Applications, vol. 66, no. 2, pp. 399–415, 1978

    1978

  18. [26]

    The concave-convex procedure,

    A. L. Yuille and A. Rangarajan, “The concave-convex procedure,” Neural Computation, vol. 15, no. 4, pp. 915–936, 2003

    2003

  19. [27]

    Variations and extension of the convex–concave procedure,

    T. Lipp and S. Boyd, “Variations and extension of the convex–concave procedure,”Optimization and Engineering, vol. 17, no. 2, pp. 263–287, 2016

    2016

  20. [28]

    The concave-convex procedure (CCCP),

    A. L. Yuille and A. Rangarajan, “The concave-convex procedure (CCCP),”Advances in Neural Information Processing Systems, vol. 14, 2001

    2001

  21. [29]

    Accelerating regularized attention kernel regression for spectrum cartography,

    L. Tao and C. W. Tan, “Accelerating regularized attention kernel regression for spectrum cartography,”arXiv preprint arXiv:2604.25138, 2026

    2026 arXiv

  22. [30]

    The elements of differentiable program- ming,

    M. Blondel and V . Roulet, “The elements of differentiable program- ming,”arXiv preprint arXiv:2403.14606, 2024

    2024 arXiv

  23. [31]

    Learning to optimize by differentiable programming,

    L. Tao, X. Tong, and C. W. Tan, “Learning to optimize by differentiable programming,”arXiv preprint arXiv:2601.16510, 2026

    2026 arXiv

  24. [32]

    A multivariate regression approach to association analysis of a quantitative trait network,

    S. Kim, K.-A. Sohn, and E. P. Xing, “A multivariate regression approach to association analysis of a quantitative trait network,”Bioinformatics, vol. 25, no. 12, pp. i204–i212, 2009

    2009

  25. [33]

    Smoothing proximal gradient method for general structured sparse regression,

    X. Chen, Q. Lin, S. Kim, J. G. Carbonell, and E. P. Xing, “Smoothing proximal gradient method for general structured sparse regression,” 2012

    2012

  26. [34]

    Generalized Laplacian precision matrix estima- tion for graph signal processing,

    E. Pavez and A. Ortega, “Generalized Laplacian precision matrix estima- tion for graph signal processing,” in2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2016, pp. 6350–6354

    2016

  27. [35]

    Non-negative low rank and sparse graph for semi-supervised learning,

    L. Zhuang, H. Gao, Z. Lin, Y . Ma, X. Zhang, and N. Yu, “Non-negative low rank and sparse graph for semi-supervised learning,” in2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2012, pp. 2328–2335

    2012

  28. [36]

    Forging the graphs: A low rank and positive semidefinite graph learning approach,

    D. Luo, H. Huang, F. Nie, and C. Ding, “Forging the graphs: A low rank and positive semidefinite graph learning approach,”Advances in Neural Information Processing Systems, vol. 25, 2012

    2012

  29. [37]

    Pseudoinverse graph convolutional networks,

    D. Alfke and M. Stoll, “Pseudoinverse graph convolutional networks,” Data Mining and Knowledge Discovery, vol. 35, no. 4, pp. 1318–1341, 2021

    2021

  30. [38]

    An efficient implementation to compute the pseudoinverse for the incremental broad learning system on added inputs,

    H. Zhu, Z. Liu, C. P. Chen, and Y . Liang, “An efficient implementation to compute the pseudoinverse for the incremental broad learning system on added inputs,”International Journal of Wavelets, Multiresolution and Information Processing, vol. 22, no. 06, p. 2450026, 2024

    2024

  31. [39]

    Diagonal of pseudoinverse of graph Laplacian: Fast estimation and exact results,

    Z. Lu, W. Xu, and Z. Zhang, “Diagonal of pseudoinverse of graph Laplacian: Fast estimation and exact results,”Theoretical Computer Science, vol. 1032, p. 115102, 2025

    2025

  32. [40]

    Strang,Introduction to Linear Algebra

    G. Strang,Introduction to Linear Algebra. SIAM, 2022

    2022

  33. [41]

    CVXPY: A Python-embedded modeling lan- guage for convex optimization,

    S. Diamond and S. Boyd, “CVXPY: A Python-embedded modeling lan- guage for convex optimization,”Journal of Machine Learning Research, vol. 17, no. 83, pp. 1–5, 2016

    2016

  34. [42]

    A distribution-free M-estimator of multivariate scatter,

    D. E. Tyler, “A distribution-free M-estimator of multivariate scatter,” The Annals of Statistics, pp. 234–251, 1987

    1987

  35. [43]

    Regularized Tyler’s scatter estimator: Existence, uniqueness, and algorithms,

    Y . Sun, P. Babu, and D. P. Palomar, “Regularized Tyler’s scatter estimator: Existence, uniqueness, and algorithms,”IEEE Transactions on Signal Processing, vol. 62, no. 19, pp. 5143–5156, 2014

    2014

  36. [44]

    Tyler’s estimator performance analysis,

    I. Soloveychik and A. Wiesel, “Tyler’s estimator performance analysis,” in2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2015, pp. 5688–5692

    2015

  37. [45]

    D. P. Palomar,Portfolio Optimization: Theory and Application. Cam- bridge, United Kingdom: Cambridge University Press, 2025

    2025

  38. [46]

    On the mathematical foundations of theoretical statistics,

    R. A. Fisher, “On the mathematical foundations of theoretical statistics,” Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of A Mathematical or Physical Character, vol. 222, no. 594-604, pp. 309–368, 1922

    1922

  39. [47]

    An algorithm for quadratic programming,

    M. Frank and P. Wolfe, “An algorithm for quadratic programming,” Naval Research Logistics Quarterly, vol. 3, no. 1-2, pp. 95–110, 1956

    1956

  40. [48]

    Frank-Wolfe algorithm for DC optimization problem,

    R. D. Mill ´an, O. P. Ferreira, and J. Ugon, “Frank-Wolfe algorithm for DC optimization problem,”arXiv preprint arXiv:2308.16444, 2023

    2023

  41. [49]

    CCCP is Frank-Wolfe in disguise,

    A. Yurtsever and S. Sra, “CCCP is Frank-Wolfe in disguise,”Advances in Neural Information Processing Systems, vol. 35, pp. 35 352–35 364, 2022

    2022

  42. [50]

    Wireless network optimization by Perron-Frobenius theory,

    C. W. Tan, “Wireless network optimization by Perron-Frobenius theory,” Foundations and Trends in Networking, vol. 9, no. 2-3, pp. 107–218, 2015

    2015

  43. [51]

    Lemmens and R

    B. Lemmens and R. Nussbaum,Nonlinear Perron-Frobenius Theory. Cambridge University Press, 2012, vol. 189

    2012

  44. [52]

    Unified framework to regularized covariance estimation in scaled Gaussian models,

    A. Wiesel, “Unified framework to regularized covariance estimation in scaled Gaussian models,”IEEE Transactions on Signal Processing, vol. 60, no. 1, pp. 29–38, 2011

    2011

  45. [53]

    Robust shrinkage estimation of high-dimensional covariance matrices,

    Y . Chen, A. Wiesel, and A. O. Hero, “Robust shrinkage estimation of high-dimensional covariance matrices,”IEEE Transactions on Signal Processing, vol. 59, no. 9, pp. 4097–4107, 2011

    2011

  46. [54]

    Granas, J

    A. Granas, J. Dugundjiet al.,Fixed Point Theory. Springer, 2003, vol. 14. APPENDIX This appendix derives CCCP iterations for two representa- tive spectral regularizers in Table I. As in the main text, we setX= Σ −1 and formulate the objective as a DC program, withS k =y ky⊤ k ...

    2003

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.