REVIEW 6 major objections 4 minor 26 references
Simultaneous Clustering and Optimization for Evolving Datasets
T0 review · 6 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read This paper claims that a dual-space regularizer and a drift threshold let a clustering-and-optimization model stay accurate on an evolving dataset without full recomputation.
desk verdict A genuinely useful idea for reusing SCO models on evolving data, but the advertised accuracy guarantees do not survive contact with the proofs. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the modified dual problem (6): $$\min_{\$\lambda$\in C}\; f^*\!\left(-\mathrm{vec}^T(\$\lambda$)(I_d\otimes Q);A\right)+\$\beta$\|(I_d\otimes Q)^T\mathrm{vec}(\$\lambda$)\|_s$$ with $C=\{\lambda:\|\lambda_i\|_q\leq 1\}$. This reformulation replaces the primal sum-of-norms regularizer $\alpha\sum_{e_{ij}\in E}w_{ij}\|X_i-X_j\|_p$ with simple dual constraints, and the new $\beta$-term penalizes exactly the quantity that carries drift, namely the dual residual $(I_d\otimes Q)^T\mathrm{vec}(\lambda)$, which by relation (5) equals $-\nabla f(X;A)$. The drift metric $\Delta f^*$ in (7) evaluates the old dual variable on the new data and triggers recomputation only when its change exceeds $c$. The accompanying ADMM variant has a closed-form $u$-update via Theorem 2 and, when $q=\infty$, splits the $\lambda$-update into $d$ independent subproblems, giving natural parallelization.
What would settle it
Take a fixed graph and noise-free data, run Algorithm 1 with a small $\beta$, and compare the reused $X^*$ against the exact optimum on $A+\Delta$ for many random perturbations; if the empirical error grows faster than the Theorem 4 bound as $\beta$ decreases, the central claim fails. A direct check is to evaluate the quadratic difference omitted in the proof of Theorem 4 and see whether it is nonnegative for all $A$ and $\Delta$; a negative example would invalidate the stated bound.
Extended reading notes
Core claim
The central claim is that the solution $X^*$ of the modified dual problem (6), with the added regularizer $\beta\|(I_d \otimes Q)^T\mathrm{vec}(\lambda)\|_s$ over the constraint set $C=\{\lambda:\|\lambda_i\|_q\leq 1\}$, is an accurate solution for the evolving dataset $A+\Delta$ as long as $\beta$ is large enough and the drift metric $\Delta f^*$ in (7) stays below the threshold $c$. The paper proves this through strong duality: the primal nonsmooth sum-of-norms term becomes simple convex constraints on the dual variable $\lambda$, and the new regularizer penalizes the dual residual $(I_d \otimes Q)^T\mathrm{vec}(\lambda)$, which is exactly $\nabla f(X;A)$ up to sign. Theorems 4 and 5 provide explicit error bounds for convex clustering and ridge regression, expressed in terms of $\beta$, $c$, and the perturbation $\Delta$. Algorithm 1 operationalizes the result: when drift is detected, compute $\Delta f^*$; if it exceeds $c$, update the data matrix and re-solve problem (6); otherwise reuse the existing $X^*$.
Load-bearing premise
The guarantees assume the variable graph $G$ and the matrix $Q$ stay fixed while only the data matrix $A$ changes; if the graph is rebuilt or edge weights change, the dual constraint set changes and the bounds no longer apply.
Editorial extensions
If this is right
- For convex clustering, the reused solution's error against the true solution on $A+\Delta$ is bounded by terms involving $\beta$, $c$, and $\Delta$, so a larger $\beta$ tolerates more drift before recomputation.
- For ridge regression, the analogous bound depends on the helper matrices $\Omega$, $\Phi$, and the label vector $y$, giving the same threshold-based update logic for prediction models.
- The metric $\Delta f^*$ provides a computable update trigger: the model is recomputed only when the drift in the dual objective evaluated at the old $\lambda^*$ exceeds the user-chosen threshold $c$.
- The ADMM variant inherits the standard $O(1/T)$ convergence rate for convex objectives, so each recomputation is comparatively cheap.
- In the $q=\infty$ case the $\lambda$-update decomposes into $d$ independent subproblems, enabling up to $O(d)$ parallel speedup.
Reading between the lines
- The tradeoff between $\beta$ and $c$ suggests a principled way to set update schedules: $\beta$ controls worst-case drift tolerance while $c$ controls how often recomputation fires, so both could be tuned from a cost model of recomputation.
- The scalar metric $\Delta f^*$ could serve as a change-point statistic in streaming settings, since it measures how much the old dual solution degrades on new data without requiring a full re-solve.
- The same dual-regularizer construction may transfer to other fused-lasso-style objectives, such as generalized lasso or trend filtering, whose sum-of-norms penalties admit the same conjugate treatment.
- A natural next stress test is whether the bounds survive graph evolution, because the paper fixes $G$ and $Q$; extending to edge re-weighting or vertex addition would be the logical follow-up.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper addresses simultaneous clustering and optimization (SCO) on evolving data. It introduces a dual formulation (6) with an extra regularizer β||(Id⊗Q)^T vec(λ)||_s, a threshold metric Δf* defined in (7), and Algorithm 1 that decides whether to recompute the model when the data matrix changes. A custom ADMM (Algorithm 2) is proposed to solve the regularized dual. The paper claims theoretical accuracy guarantees for two special cases, convex clustering and ridge regression, stated as Theorems 4 and 5, and reports experiments on six datasets comparing the proposed REG-CC and REG-RG methods against PRIMAL-CC, SSNAL-CC, NET-LASSO, and AMA-CC.
Significance. If the theoretical guarantees were correct, the paper would offer a practical way to reuse an SCO model across small data perturbations, with a clean dual formulation that avoids the nonsmooth sum-of-norms term and a parallelizable ADMM for q=∞. The empirical protocol is also sensible: accuracy is measured against the true re-solved optimum, and the baselines are standard. However, the proof chain that connects the threshold c to primal model error has several unsupported steps, and the ridge-regression dual in Section 4.2 appears to be the wrong conjugate of the stated loss. Because the advertised 'guarantee of model accuracy' is the central contribution, the manuscript is not publishable in its current form.
major comments (6)
- [Section 6.1, Eq. (10)] The first inequality in (10) does not follow from the definitions. Since tilde-λ* minimizes F_{A+Δ}(λ) = f^*(-vec^T(λ)(Id⊗Q); A+Δ) + β||(Id⊗Q)^Tvec(λ)||_s, minimality gives F_{A+Δ}(tilde-λ*) ≤ F_{A+Δ}(λ*). This implies f^*_{A+Δ}(tilde-λ*) ≤ f^*_{A+Δ}(λ*) + β(||(Id⊗Q)^Tvec(λ*)||_s − ||(Id⊗Q)^Tvec(tilde-λ*)||_s). The β-difference can have either sign, so the claimed inequality f^*_{A+Δ}(tilde-λ*) ≤ f^*_{A+Δ}(λ*) is unproved and generally false. Since (10) is the only bridge from the threshold c to the primal solutions, both Theorem 4 and Theorem 5 inherit this gap.
- [Section 6.2, proof of Theorem 4, between (10) and (16)] Passing from (10) to (16) deletes the quadratic term (1/4)(||(Id⊗Q)^Tvec(tilde-λ*)||² − ||(Id⊗Q)^Tvec(λ*)||²) without proving its sign. If this term is negative, the inequality c ≥ vec^T(A)(Id⊗Q)^Tvec(λ*) − vec^T(A+Δ)(Id⊗Q)^Tvec(tilde-λ*) is not implied by (10). In addition, combining (15) with (16) yields vec^T(A)(vec(tilde-X*)−vec(X*)) ≥ vec^T(A)vec(Δ) − (1/2)(c + (1/β)||vec(Δ)|| vec^T(A+Δ)vec(A+Δ)), which is a lower bound, not the upper bound stated in the theorem; the direction of the final inequality is wrong.
- [Section 6.2, Theorem 4] The left-hand side, vec^T(A)(vec(tilde-X*)−vec(X*)), is a directional projection of the model error, not the model error itself. If A has a zero row, or if vec(tilde-X*)−vec(X*) is orthogonal to vec(A), the left-hand side can be zero while ||tilde-X*−X*||_F is arbitrarily large. Therefore even a corrected proof of the displayed inequality would not establish the 'guarantee of model accuracy' promised in the abstract.
- [Section 4.2 and Theorem 5] The ridge-regression quadratic form is incorrect. The loss ∑_i ||A_i X_i^T − y_i||² has quadratic part vec(X)^T [diag(vec(A))(J_d⊗I_n)diag(vec(A))] vec(X) plus γ||vec(X)||², not vec(X)^T [diag(vec(A))² + γI] vec(X); the stated Ω omits the within-row cross terms. Consequently the dual problem and Lemma 2 are not conjugate to the stated loss. In the proof of Theorem 5, Eq. (12) drops the linear terms involving y, the substitution from (5) loses both the −Λ^T y term and the minus sign (since ∇f(X*;A) = −(Id⊗Q)^Tvec(λ*)), and the expression y^T tilde-Ω^{-1} tilde-Λ y is dimensionally inconsistent (y is n×1 while tilde-Ω is nd×nd). Theorem 5 is therefore unsupported.
- [Section 5.1, (8)–(10)] Algorithm 2 is introduced to solve (6), whose regularizer is β||u||_s, but the augmented Lagrangian is written with g(u) = ||u||_{1,p}, which is the original primal regularizer. The u-update in Theorem 2 is the proximal map for ||·||_{1,p}, not for β||·||_s (unless s=p and β=1). No argument is given that the resulting ADMM iterates minimize (6), so the correctness of the main algorithmic contribution is not established.
- [Section 3.1 and Section 6] The theoretical analysis fixes the graph G and the matrix Q, while Algorithm 1 updates only the data matrix A. Because the data graph is built from A via K-NN (Section 3.1), the edge set E, the dual dimension m, the matrix Q, and the constraint set C can all change as A evolves. When the graph changes, the dual variables have a different dimension and the statements of Theorems 4 and 5 do not apply. The manuscript does not state this as a limitation or extend the analysis to graph changes.
minor comments (4)
- [Section 5.1, Eq. (10)] The update of µ should read µ(t+1) = µ(t) + ρ(...), not µ^T + ρ(...).
- [Section 6.3, definition of tilde-Λ] The definition of tilde-Λ uses diag(vec(A+Λ)); this should presumably be diag(vec(A+Δ)).
- [Appendix, Lemma 1] The displayed equality for the supremum holds only when (Id⊗Q)^T is surjective; for the upper bound it suffices to state the inequality ≤, so the proof should be phrased as an inequality.
- [Section 6.1, Remark 3] Remark 3 is only a placeholder ('may be bounded') and does not state a theorem; it should either be removed or replaced with a precise statement.
Circularity Check
No circularity: the proposed bound is a conditional trade-off in user-set c and beta, not a fitted input; the main unsupported step is a proof gap, not a circular reduction.
full rationale
The derivation chain is not circular. The dual formulation (6) is obtained by standard Fenchel duality in the appendix (Theorem 1), and the added regularizer beta times the norm of (Id tensor Q)^T vec(lambda) is a genuinely new term whose effect is subsequently analyzed rather than being fitted to the target result. Algorithm 1's threshold c is defined through the dual-objective change Delta f* in (7), and Theorems 4 and 5 state error bounds in terms of that same c and of beta; this is a legitimate conditional trade-off, not a parameter fitted to the conclusion. The experiments also measure error against the true re-solved optimum on the evolving data, so the empirical claim has external content. The paper's self-citations, e.g., [2] and [14], are contextual and not load-bearing, and no uniqueness theorem is imported from the authors' prior work. The serious weakness is mathematical correctness rather than circularity: inequality (10) is not entailed by the definitions, because minimizing the beta-regularized objective does not order the bare f* terms, and the proof of Theorem 4 drops a quadratic term and bounds only a directional projection of the model error. Those are proof gaps that prevent the theorems from following as stated, but they do not reduce the claimed prediction to the paper's own inputs, so the circularity score remains 0.
Assumptions & free parameters
free parameters (6)
- beta =
beta=5 for convex clustering; beta=10^-2, 10^-4, 10^-5 or 50, 10, 1 for ridge regression
- c =
c=10 (default in Algorithm 1)
- alpha =
alpha in {5,10,50,100} in experiments
- gamma =
gamma=5 for ridge regression
- s =
s=1, 2, infinity tested; s=1 recommended
- K =
K=10, 5, or 4 depending on dataset
assumptions (5)
- standard math Strong duality and Slater's condition hold for the primal SCO problem (2), so the dual (4) is exact.
- domain assumption The conjugate functions f* for convex clustering and ridge regression as written in Section 4.2 are correct.
- domain assumption The variable graph G and the operator Q are fixed while A evolves; Algorithm 1 does not rebuild the K-NN graph or edge weights.
- domain assumption The non-ergodic ADMM convergence theorem of He and Yuan applies to Algorithm 2 with its approximate lambda-subproblem.
- ad hoc to paper The omitted quadratic difference in the proof of Theorem 4 is nonnegative and can be deleted from the inequality chain.
Cite this review
Pith. "Pith review of Simultaneous Clustering and Optimization for Evolving Datasets." pith.science (2026). https://pith.science/paper/4NMQVEHS
@misc{pith2026190801384,
author = {Pith},
title = {Pith review of: Simultaneous Clustering and Optimization for Evolving Datasets},
year = {2026},
howpublished = {\url{https://pith.science/paper/4NMQVEHS}},
note = {Machine review of arXiv:1908.01384}
}
read the original abstract
Simultaneous clustering and optimization (SCO) has recently drawn much attention due to its wide range of practical applications. Many methods have been previously proposed to solve this problem and obtain the optimal model. However, when a dataset evolves over time, those existing methods have to update the model frequently to guarantee accuracy; such updating is computationally infeasible. In this paper, we propose a new formulation of SCO to handle evolving datasets. Specifically, we propose a new variant of the alternating direction method of multipliers (ADMM) to solve this problem efficiently. The guarantee of model accuracy is analyzed theoretically for two specific tasks: ridge regression and convex clustering. Extensive empirical studies confirm the effectiveness of our method.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
Network lasso: Clustering and optimization in large graphs
D. Hallac, J. Leskovec, and S. Boyd, “Network lasso: Clustering and optimization in large graphs.” in ACM Proceedings of the Inter- national Conference on Knowledge Discovery & Data Mining (KDD) , 2015, p. 387
work page 2015
-
[2]
Trian- gle lasso for simultaneous clustering and optimization in graph datasets,
Y. Zhao, K. Xu, L. Xinwang, E. Zhu, X. Zhu, and J. Yin, “Trian- gle lasso for simultaneous clustering and optimization in graph datasets,” IEEE Transactions on Knowledge and Data Engineering , 2018
work page 2018
-
[3]
Least squares quantization in pcm,
S. Lloyd, “Least squares quantization in pcm,” IEEE Transactions on Information Theory, vol. 28, no. 2, pp. 129–137, 1982
1982
-
[4]
Convergence properties of the k- means algorithms,
L. Bottou, Y. Bengio et al. , “Convergence properties of the k- means algorithms,” in Proceedings of Advances in Neural Information Processing Systems, 1995, pp. 585–592
work page 1995
-
[5]
Learning joint affinity graph for multi-view subspace clustering,
C. Tang, X. Zhu, X. Liu, M. Li, P . Wang, C. Zhang, and L. Wang, “Learning joint affinity graph for multi-view subspace clustering,” IEEE Transactions on Multimedia, 2018
work page 2018
-
[6]
Late fusion incomplete multi-view clustering,
X. Liu, X. Zhu, M. Li, L. Wang, C. Tang, J. Yin, D. Shen, H. Wang, and W. Gao, “Late fusion incomplete multi-view clustering,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
work page 2018
-
[7]
An application of network lasso optimization for ride sharing prediction,
S. Ghosh, K. Page, and D. D. Roure, “An application of network lasso optimization for ride sharing prediction,” in ACM Proceed- ings of the International Conference on Knowledge Discovery & Data Mining (ICDM), 2016
work page 2016
-
[8]
Splitting methods for convex clustering,
E. C. Chi and K. Lange, “Splitting methods for convex clustering,” Professional Geographer, vol. 46, no. 1, pp. 80–89, 2014
work page 2014
Show all 26 references
-
[9]
Statistical properties of convex cluster- ing,
K. M. Tan and D. Witten, “Statistical properties of convex cluster- ing,” Electronic Journal of Statistics, vol. 9, no. 2, p. 2324, 2015
2015
-
[10]
Convex clustering via l1 fusion penalization,
P . Radchenko and G. Mukherjee, “Convex clustering via l1 fusion penalization,” Journal of the Royal Statistical Society: Series B (Statis- tical Methodology), vol. 47, p. 67, feb 2017
2017
-
[11]
Snap: A general purpose network anal- ysis and graph mining library
J. Leskovec and R. Sosi, “Snap: A general purpose network anal- ysis and graph mining library.” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 8, no. 1, p. 1, 2016
2016
-
[12]
When is network lasso accurate?
A. Jung, N. Tran, and A. Mara, “When is network lasso accurate?” Frontiers in Applied Mathematics and Statistics, vol. 3, pp. 1–11, 2018
2018
-
[13]
Just relax and come clustering! a convexification of k-means clustering,
F. Lindsten, H. Ohlsson, and L. Ljung, “Just relax and come clustering! a convexification of k-means clustering,” Institute of Technology, 2011
2011
-
[14]
Large-scale k-means clustering via variance reduction,
Y. Zhao, Y. Ming, X. Liu, E. Zhu, K. Zhao, and J. Yin, “Large-scale k-means clustering via variance reduction,” Neurocomputing, vol. 307, pp. 184 – 194, 2018
2018
-
[15]
Convex Optimization Procedure for Clustering - Theoretical Revisit
C. Zhu, H. Xu, C. Leng, and S. Yan, “Convex Optimization Procedure for Clustering - Theoretical Revisit.” in Proceedings of the Advances in Neural Information Processing Systems (NIPS) , 2014
2014
-
[16]
Clustering by sum of norms: Stochastic incremental algorithm, convergence and cluster recovery,
A. Panahi, D. Dubhashi, F. D. Johansson, and C. Bhattacharyya, “Clustering by sum of norms: Stochastic incremental algorithm, convergence and cluster recovery,” in Proceedings of the 34th Inter- national Conference on Machine Learning (ICML) , 2017
2017
-
[17]
An efficient semismooth new- ton based algorithm for convex clustering,
Y. Yuan, D. Sun, and K. C. Toh, “An efficient semismooth new- ton based algorithm for convex clustering,” in Proceedings of the Advances in Neural Information Processing Systems (NIPS) , 2018
2018
-
[18]
Shalev-Shwartz and S
S. Shalev-Shwartz and S. Ben-David, Understanding Machine Learn- ing: From Theory to Algorithms. Cambridge University Press, 2014
2014
-
[19]
Distributed optimization and statistical learning via the alternating direction method of multipliers,
S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Foundations & Trends in Machine Learning , vol. 3, no. 1, pp. 1–122, jan 2011
2011
-
[20]
On non-ergodic convergence rate of Douglas- Rachford alternating direction method of multipliers
B. He and X. Yuan, “On non-ergodic convergence rate of Douglas- Rachford alternating direction method of multipliers.” Numerische Mathematik, vol. 130, no. 3, pp. 567–577, 2015
2015
-
[21]
CVX research project,
“CVX research project,” http://cvxr.com/cvx/, accessed: 2018-07- 01
2018
-
[22]
Convex Clustering - An Attractive Alternative to Hierarchical Clustering
G. K. Chen, E. C. Chi, J. M. O. Ranola, and K. Lange, “Convex Clustering - An Attractive Alternative to Hierarchical Clustering.” PLoS Computational Biology, vol. 11, no. 5, p. e1004228, 2015
2015
-
[23]
Boyd and L
S. Boyd and L. Vandenberghe, Convex Optimization. New York, NY, USA: Cambridge University Press, 2004
2004
-
[24]
Proximal algorithms,
N. Parikh and S. Boyd, “Proximal algorithms,” Foundations & Trends in Optimization, vol. 1, no. 3, pp. 127–239, 2014
2014
-
[25]
Bertsekas, A
D. Bertsekas, A. Nedic, and A. Ozdaglar, Convex Analysis and Optimization. Athena Scientific, 2004. Yawei Zhao is currently a PhD candidate in Computer Science at the National University of Defense Technology, China. He received his B.E. degree and M.S. degree in Computer Scien...
2004
-
[2019]
Deke Guo received a B.S
His current research interests focus on building machine learning models for solving computer vision and data mining problems. Deke Guo received a B.S. degree in industry engineering from Beijing University of Aeronau- tics and Astronautics, Beijing, China, in 2001, and a PhD ...
2001
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.