pith. sign in

arxiv: 1906.12015 · v2 · pith:3TIBFCK4new · submitted 2019-06-28 · 🧮 math.NA · cs.NA

Accelerated Symmetric ADMM and Its Applications in Signal Processing

Jianchao Bai , Junli Liang , Ke Guo , Yang Jing This is my paper

Pith reviewed 2026-05-25 14:16 UTC · model grok-4.3

classification 🧮 math.NA cs.NA
keywords alternating direction method of multipliersnonconvex optimizationsymmetric ADMMsignal processingconvergence analysisaugmented Lagrangianiteration complexitysparse regularization
0
0 comments X

The pith

A symmetric ADMM converges for nonsmooth nonconvex equality-constrained problems by updating dual variables twice with different stepsizes.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper develops an accelerated symmetric alternating direction method of multipliers for nonsmooth nonconvex optimization problems subject to equality constraints. The method updates the dual variables twice using different stepsizes and provides convergence guarantees along with pointwise iteration complexity bounds. These results are established using proper assumptions rather than the Kurdyka-Lojasiewicz inequality, and the approach is tested on sparse regularization problems arising in signal processing. A sympathetic reader would care because many practical signal processing tasks involve nonconvex objectives where standard ADMM convergence theory does not apply.

Core claim

The authors propose a symmetric ADMM based on different acceleration techniques for potentially nonsmooth nonconvex programming problems with equality constraints. The dual variables are updated twice with different stepsizes. Under proper assumptions, they analyze the convergence of the algorithm in terms of the augmented Lagrangian function and the pointwise iteration-complexity in terms of the primal-dual residuals. The performance is verified on applications in sparse nonconvex/convex regularized minimization signal processing problems.

What carries the argument

Symmetric ADMM that updates dual variables twice with different stepsizes combined with acceleration techniques.

If this is right

  • Convergence of the augmented Lagrangian function holds under the assumptions.
  • Pointwise iteration-complexity bounds apply to the primal-dual residuals.
  • The algorithm applies directly to sparse regularized minimization problems in signal processing.
  • Preliminary numerical tests confirm performance on both nonconvex and convex instances.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The twice-update structure with differing stepsizes could be tuned for faster practical performance in other constrained problems.
  • Avoiding the Kurdyka-Lojasiewicz inequality may allow the method to cover cases where that inequality does not hold.
  • The same acceleration pattern might transfer to related primal-dual splitting schemes outside signal processing.

Load-bearing premise

The proof requires unspecified proper assumptions that replace the Kurdyka-Lojasiewicz inequality to ensure convergence and complexity bounds.

What would settle it

A nonsmooth nonconvex equality-constrained problem satisfying the proper assumptions where the augmented Lagrangian fails to converge or the primal-dual residuals violate the stated pointwise iteration-complexity bound.

Figures

Figures reproduced from arXiv: 1906.12015 by Jianchao Bai, Junli Liang, Ke Guo, Yang Jing.

Figure 1
Figure 1. Figure 1: Convergence tendency of the equality constraint err [PITH_FULL_IMAGE:figures/full_fig_p015_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparison between the original signal and reconstr [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Original signal and reconstructed signal by Algorit [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The left and right are the errors and average run time v [PITH_FULL_IMAGE:figures/full_fig_p017_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: DOAs at SNR= 5dB. • The implementation of the proposed method is faster than that of the CVX method. • In terms of DOA resolution and the estimation accuracy of the incoming signal power, the proposed method is better than that of CVX. 5 Conclusion remarks In this paper, we construct a symmetric alternating direction method of multipliers for solving a family of possibly nonconvex nonxmooth optimization pr… view at source ↗
read the original abstract

The alternating direction method of multipliers (ADMM) were extensively investigated in the past decades for solving separable convex optimization problems. Fewer researchers focused on exploring its convergence properties for the nonconvex case although it performed surprisingly efficient. In this paper, we propose a symmetric ADMM based on different acceleration techniques for a family of potentially nonsmooth nonconvex programing problems with equality constraints, where the dual variables are updated twice with different stepsizes. Under proper assumptions instead of using the so-called Kurdyka-Lojasiewicz inequality, convergence of the proposed algorithm as well as its pointwise iteration-complexity are analyzed in terms of the corresponding augmented Lagrangian function and the primal-dual residuals, respectively. Performance of our algorithm is verified by some preliminary numerical examples on applications in sparse nonconvex/convex regularized minimization signal processing problems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper proposes a symmetric ADMM variant that accelerates via dual-variable updates with two different stepsizes for nonsmooth nonconvex equality-constrained problems. It claims global convergence of the augmented Lagrangian and pointwise iteration-complexity bounds on primal-dual residuals, proved under unspecified 'proper assumptions' that replace the Kurdyka-Łojasiewicz inequality. Numerical performance is illustrated on sparse regularized signal-processing problems.

Significance. A convergence framework for nonconvex ADMM that avoids KL and supplies explicit complexity bounds would be useful for signal-processing applications where KL fails. The paper's contribution hinges on whether the 'proper assumptions' are stated precisely, are checkable, and hold for the target problems; if so, the result would strengthen the theoretical toolkit for accelerated nonconvex splitting methods.

major comments (3)
  1. [Abstract, §3] Abstract and §3 (convergence analysis): the central claims of convergence and pointwise iteration complexity rest on 'proper assumptions' that replace the KL inequality, yet these assumptions are never listed explicitly. Without their precise statement (e.g., error-bound, boundedness, or prox-regularity conditions) and a verification that they hold for the sparse-regularized problems in §5, the theorems cannot be assessed.
  2. [§3, Theorem 3.1] §3, Theorem 3.1 (or equivalent): the proof sketch supplied in the text does not indicate how the two distinct dual step-sizes enter the descent inequality for the augmented Lagrangian; the argument therefore appears to reduce to a standard ADMM analysis whose novelty is unclear once the 'proper assumptions' are granted.
  3. [§5] §5, numerical examples: the reported residual plots and objective values lack error bars, iteration counts to a fixed tolerance, or comparison against a standard non-accelerated symmetric ADMM baseline, so it is impossible to confirm that the claimed acceleration is realized under the same assumptions used in the theory.
minor comments (2)
  1. [Abstract] Abstract: 'programing' should read 'programming'.
  2. [§2] Notation: the two dual step-sizes are introduced without a consistent symbol (e.g., β1, β2); this should be fixed at first appearance in §2.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive comments, which help strengthen the presentation. We address each major point below and will revise the manuscript accordingly where the concerns are valid.

read point-by-point responses

  1. Referee: [Abstract, §3] Abstract and §3 (convergence analysis): the central claims of convergence and pointwise iteration complexity rest on 'proper assumptions' that replace the KL inequality, yet these assumptions are never listed explicitly. Without their precise statement (e.g., error-bound, boundedness, or prox-regularity conditions) and a verification that they hold for the sparse-regularized problems in §5, the theorems cannot be assessed.

    Authors: We agree that the assumptions must be stated explicitly rather than referred to generically as 'proper assumptions.' In the revision we will add a dedicated subsection in §3 that lists them precisely (including any error-bound or boundedness conditions used), and we will verify their validity for the sparse-regularized signal-processing problems of §5. This addresses the assessability concern directly. revision: yes

  2. Referee: [§3, Theorem 3.1] §3, Theorem 3.1 (or equivalent): the proof sketch supplied in the text does not indicate how the two distinct dual step-sizes enter the descent inequality for the augmented Lagrangian; the argument therefore appears to reduce to a standard ADMM analysis whose novelty is unclear once the 'proper assumptions' are granted.

    Authors: The two distinct dual step-sizes are essential to the descent inequality because they permit a weighted combination of the two dual updates inside the Lyapunov function, yielding a stricter decrease than the single-stepsize case. We will expand the proof of Theorem 3.1 (and the supporting lemmas) to display the explicit dependence on both step-sizes in the key inequality, thereby clarifying the novelty relative to standard symmetric ADMM. revision: yes

  3. Referee: [§5] §5, numerical examples: the reported residual plots and objective values lack error bars, iteration counts to a fixed tolerance, or comparison against a standard non-accelerated symmetric ADMM baseline, so it is impossible to confirm that the claimed acceleration is realized under the same assumptions used in the theory.

    Authors: The numerical section will be augmented with (i) error bars over multiple random instances, (ii) tabulated iteration counts required to reach a fixed primal-dual residual tolerance, and (iii) direct comparison against the non-accelerated symmetric ADMM baseline run under identical problem instances and stopping criteria. These additions will allow quantitative confirmation of the acceleration effect. revision: yes

Circularity Check

0 steps flagged

No circularity: convergence claims rest on independent analysis under stated assumptions, not self-referential definitions or fitted inputs

full rationale

The paper's derivation chain consists of proposing a symmetric ADMM variant with dual updates at different stepsizes, followed by a convergence and iteration-complexity analysis expressed in terms of the augmented Lagrangian and primal-dual residuals. This analysis is explicitly conditioned on 'proper assumptions' that replace the Kurdyka-Lojasiewicz inequality, with no equations, parameters, or self-citations in the provided text that reduce the claimed rates to a redefinition or fit of the algorithm inputs themselves. Numerical examples on signal-processing problems are presented only as verification, not as the source of the theoretical bounds. The derivation therefore remains self-contained against external benchmarks and does not match any enumerated circularity pattern.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claims rest on a set of 'proper assumptions' whose precise content is not stated in the abstract; these assumptions replace the Kurdyka-Lojasiewicz inequality and are therefore load-bearing for both convergence and complexity bounds. No free parameters, invented entities, or additional axioms are visible.

axioms (1)
  • domain assumption Proper assumptions (unspecified) on the objective functions and constraint set that guarantee convergence without invoking the Kurdyka-Lojasiewicz inequality
    Abstract states convergence holds 'under proper assumptions instead of using the so-called Kurdyka-Lojasiewicz inequality'

pith-pipeline@v0.9.0 · 5667 in / 1339 out tokens · 24098 ms · 2026-05-25T14:16:46.931973+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. A family of multi-parameterized proximal point algorithms

    math.NA 2019-07 unverdicted novelty 4.0

    A family of multi-parameterized proximal point algorithms with relaxation is introduced for linearly constrained convex minimization, with global and sublinear convergence shown from the variational inequality viewpoint.

Reference graph

Works this paper leans on

39 extracted references · 39 canonical work pages · cited by 1 Pith paper · 1 internal anchor

  1. [1]

    Attouch, J

    H. Attouch, J. Bolte, P. Redont, A. Soubeyran, Proximal a lternating minimization and projection meth- ods for nonconvex problems: An approach based on the Kurdyka -Lojasiewicz inequality, Mathematics of Operations Research, 35 (2010) 438-457

    work page 2010

  2. [2]

    S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, Distr ibuted optimization and statistical learning via the alternating direction method of multipliers, Foundations and Trends in Machine Learning, 3 (2010) 1-122

    work page 2010

  3. [3]

    J. Bai, H. Zhang, J. Li, A parameterized proximal point al gorithm for separable convex optimization, Optimization Letters, 12 (2018) 1589-1608

    work page 2018

  4. [4]

    J. Bai, J. Li, F. Xu, H. Zhang, Generalized symmetric ADMM for separable convex optimization, Compu- tational Optimization and Applications, 70 (2018) 129-170 . 18

    work page 2018

  5. [5]

    J. Bai, J. Li, Z. Wu, Several variants of the primal-dual h ybrid gradient algorithm with applications, Nu- merical Mathematics: Theory, Methods and Applications, 12 (2019) 1-24

    work page 2019

  6. [6]

    Chartrand, V

    R. Chartrand, V. Staneva, Restricted isometry properti es and nonconvex compressive sensing, Inverse Prob- lems, 24 (2008) 20-35

    work page 2008

  7. [7]

    Chang, S

    X. Chang, S. Liu, P. Zhao, D. Song, A generalization of lin earized alternating direction method of multipliers for solving two-block separable convex programming, Journ al of Computational and Applied Mathematics, 357 (2019) 251-272

    work page 2019

  8. [8]

    Corman, X

    E. Corman, X. Yuan, A generalized proximal point algorit hm and its convergence rate, SIAM Journal on Optimization, 24 (2014) 1614-1638

    work page 2014

  9. [9]

    Douglas, H

    J. Douglas, H. Rachford, On the numerical solution of hea t conduction problems in two and three space variables, Transactions of the American Mathematical Soci ety, 82 (1956) 421-439

    work page 1956

  10. [10]

    Eckstein, D

    J. Eckstein, D. Bertsekas, On the Douglas-Rachford spl itting method and the proximal point algorithm for maximal monotone operators, Mathematical Programming, 55 (1992) 293-318

    work page 1992

  11. [11]

    Figueiredo, R

    M. Figueiredo, R. Nowak, S. Wright, Gradient projectio n for sparse reconstruction: application to com- pressed sensing and other inverse problems, IEEE Journal of Selected Topics in Signal Processing, 1 (2007) 586-597

    work page 2007

  12. [12]

    F. Fang, B. He, H. Liu, X. Yuan, Generalized alternating direction method of multipliers: new theoretical insights and applications, Mathematical Programming Comp utation, 7 (2015) 149-187

    work page 2015

  13. [13]

    Gabay, Applications of the method of multipliers to v ariational inequalities, Studies in Mathematics and Its Applications, 15 (1983) 299-331

    D. Gabay, Applications of the method of multipliers to v ariational inequalities, Studies in Mathematics and Its Applications, 15 (1983) 299-331

    work page 1983

  14. [14]

    Glowinski, Lectures on numerical methods for non-li near variational problems, Published for the Tata Institute of Fundamental Research, Bombay [by] Springer-V erlag, 1980

    R. Glowinski, Lectures on numerical methods for non-li near variational problems, Published for the Tata Institute of Fundamental Research, Bombay [by] Springer-V erlag, 1980

    work page 1980

  15. [15]

    Glowinski, On alternating direction methods of mult ipliers: A historical perspective, In Modeling, Simu- lation and Optimization for Science and Technology, W

    R. Glowinski, On alternating direction methods of mult ipliers: A historical perspective, In Modeling, Simu- lation and Optimization for Science and Technology, W. Fitz gibbon, Y.A. Kuznetsov, P. Neittaanmaki, O. Pironneau, eds., Computational Methods in Applied Science s, Vol. 34, Springer, Dordrecht, (2014), 59-82

    work page 2014

  16. [16]

    Glowinski, T

    R. Glowinski, T. Karkkainen, K. Majava, On the converge nce of operator-splitting methods, in Numerical Methods for Scientific Computing, Variational Problems and Applications, E. Heikkola, Y. Kuznetsov, P. Neittaanmaki, and O. Pironneau, eds., CIMNE, Barcelona, 20 03, pp. 67-79

    work page

  17. [17]

    K. Guo, D. Han, T. Wu, Convergence of alternating direct ion method for minimizing sum of two nonconvex functions with linear constraints, International Journal of Computer Mathematics, 94 (2017) 1653-1669

    work page 2017

  18. [18]

    K. Guo, D. Han, D. Wang, T. Wu, Convergence of ADMM for mul ti-block nonconvex separable optimization models, Frontier of Mathematics in China, 12 (2017) 1139-11 62

    work page 2017

  19. [19]

    K. Guo, D . Han, T. Wu, Convergence of ADMM for optimizati on problems with nonseparable nonconvex objective and linear constraints, Pacific Journal of Optimi zation, 14 (2018) 489-506

    work page 2018

  20. [20]

    Convergence rate bounds for a proximal ADMM with over-relaxation stepsize parameter for solving nonconvex linearly constrained problems

    M. Goncalves, J. Melo, R. Monteiro, Convergence rate bo unds for a proximal ADMM with over-relaxation stepsize parameter for solving nonconvex linearly constra ined problems, arXiv:1702.01850v2 (2017)

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  21. [21]

    B. He, H. Liu, Z. Wang, X. Yuan, A strictly contractive Pe aceman-Rachford splitting method for convex programming, SIAM Journal on Optimization, 24 (2014) 1011- 1040

    work page 2014

  22. [22]

    B. He, H. Yang, S. Wang, Alternating direction method wi th self-adaptive penalty parameters for monotone variational inequalities, Journal of Optimization Theory and Applications, 106 (2000) 337-356

    work page 2000

  23. [23]

    B. He, F. Ma, X. Yuan, Convergence study on the symmetric version of ADMM with larger step sizes, SIAM Journal on Imaging Science, 9 (2016) 1467-1501. 19

    work page 2016

  24. [24]

    M. Hong, Z. Luo, M. Razaviyay, Convergence analysis of a lternating direction method of multipliers for a family of nonconvex problems, SIAM Journal on Optimization , 26 (2016) 337-364

    work page 2016

  25. [25]

    S. Kim, K. Koh, M. Lustig, S. Boyd, D. Gorinvesky, An inte rior-point method for large-scale l1-regularized least squares, IEEE Journal of Selected Topics in Signal Pro cessing, 1 (2007) 606-617

    work page 2007

  26. [26]

    Lions, B

    P. Lions, B. Mercier, Splitting algorithms for the sum o f two nonlinear operators, SIAM Journal on Numerical Analysis, 16 (1979) 964-979

    work page 1979

  27. [27]

    G. Li, T. Pong, Global convergence of splitting methods for nonconvex composite optimization, SIAM Journal on Optimization, 25 (2015) 2434-2460

    work page 2015

  28. [28]

    Q. Liu, X. Shen, Y. Gu, Linearized ADMM for nonconvex non -smooth optimization with convergence analysis, IEEE Access, (2019) doi:10.1109/ACCESS.2019.2 914461

    work page doi:10.1109/access.2019.2 2019

  29. [29]

    Malioutov, M

    D. Malioutov, M. Cetin, A. Willsky, A sparse signal reco nstruction perspective for source localization with sensor arrays, IEEE Transactions on Signal Processing, 53 ( 2005) 3010-3022

    work page 2005

  30. [30]

    Mordukhovich, Variational Analysis and Generalize d Differentiation I: Basic Theory, Grundlehren der mathematischen Wissenschaften, Springer-Verlag Berlin H eidelberg, 2006

    B. Mordukhovich, Variational Analysis and Generalize d Differentiation I: Basic Theory, Grundlehren der mathematischen Wissenschaften, Springer-Verlag Berlin H eidelberg, 2006

    work page 2006

  31. [31]

    Nesterov, A method of solving a convex programming pr oblem with convergence rate O(1/k2), Soviet Mathematics Doklady, 27 (1983) 372-376

    Y. Nesterov, A method of solving a convex programming pr oblem with convergence rate O(1/k2), Soviet Mathematics Doklady, 27 (1983) 372-376

    work page 1983

  32. [32]

    Peaceman, H

    D. Peaceman, H. Rachford, The numerical solution of par abolic elliptic differential equations, SIAM Journal on Applied Mathematics, 3 (1955) 28-41

    work page 1955

  33. [33]

    Rockafellar, R

    R. Rockafellar, R. Wets, Variational Analysis, Spring er-Verlag Berlin Heidelberg, 1998

    work page 1998

  34. [34]

    T. Sun, C. Zhang, Sparse matrix inversion with scaled La sso, Journal of Machine Learning Research, 14 (2013) 3385-3418

    work page 2013

  35. [35]

    W. Wang, Y. Yin, J. Zeng, Global convergence of ADMM in no nconvex nonsmooth optimization, Journal of Scientific Computing, 78 (2019) 29-63

    work page 2019

  36. [36]

    Z. Wu, M. Li, D. Wang, D. Han, A symmetric alternating dir ection method of multipliers for separable nonconvex minimization problems, Asia-Pacific Journal of O perational Research, 34 (2017) 1750030, 27 pages

    work page 2017

  37. [37]

    Z. Xu, X. Chang, F. Xu, H. Zhang, L1/2 regularization: a thresholding representation theory and a fast solver, IEEE Transactions on Neural Networks and Learning S ystems, 23 (2012) 1013-1027

    work page 2012

  38. [38]

    L. Yang, T. K. Pong, X. Chen, Alternating direction meth od of multipliers for a class of nonconvex and nonsmooth problems with applications to background/foreg round extraction, SIAM Journal on Imaging Sciences, 10 (2017) 74-110

    work page 2017

  39. [39]

    M. Zhu, T. Chan, An efficient primal-dual hybrid gradient algorithm for total variation image restoration, CAM Report 08-34, UCLA, Los Angeles, CA, 2008. 20

    work page 2008