REVIEW 2 major objections 4 minor 46 references
The paper claims that a single total-variation penalized least-squares estimator can jointly recover the parameters of many linear dynamical systems from one short trajectory each, with the mean squared error driven to zero by the graph's c
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 20:39 UTC pith:MXGOG4ME
load-bearing objection Genuinely new TV-penalized joint LDS estimation theory, but Lemma 5 has a factor-of-m scaling error that must be fixed before the stated conditions are certified. the 2 major comments →
Joint learning of a network of linear dynamical systems via total variation penalization
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that the TV-penalized joint estimator (1.2)/(2.3), which adds an entry-wise ℓ1 penalty on edge differences to a least-squares fit, satisfies the non-asymptotic bound ∥â−a*∥₂ ≤ (2mλ/T)(1 + 3√|S|/κ_S) + √(8λm/T ∥(eDa*)_{S^c}∥₁) with high probability. Under the uniform stability condition ∥A*_l∥₂ ≤ ρ_max < 1, this yields consistency of the MSE as m→∞ with T fixed for well-connected graphs such as the complete graph and dense Erdős–Rényi graphs, provided the parameter matrices vary smoothly (or have few jumps) across edges; for sparser graphs like the star and 2D grid, T only needs to grow polylogarithmically with m and d. The proof provides, to the authors' knowledge, the f
What carries the argument
The carrying object is the generalized-lasso formulation of the estimator, with block-diagonal design Q = blkdiag(X_lᵀ ⊗ I_d) and penalty λ‖eDa‖₁, where eD is the Kronecker product of the graph incidence matrix with I_{d²}. The argument proceeds through four components: (i) tail bounds for self-normalized vector-valued martingales to control the noise terms (Lemmas 1–2); (ii) a range–nullspace decomposition of the error h = h₁ + h₂ relative to eD, separating graph-edge differences from the global mean (the identity eD†eD = I − eΠ); (iii) a restricted eigenvalue condition for Q obtained by bounding suprema of second-order subgaussian chaos via Talagrand functionals (Lemmas 3–4) and a Hanson–W
Load-bearing premise
That every true system matrix has spectral norm bounded below 1 by a uniform constant, ∥A*_l∥₂ ≤ ρ_max < 1; if any node is marginally stable or unstable, the bounds on β and the Grammians collapse, and the stated rates and small-T consistency are no longer supported by the proof.
What would settle it
Run the estimator on a complete graph with T = 5 and m = 50, 100, 200, where every node's A*_l is a rotation matrix with spectral radius 1 (so condition (2.7) fails), but the matrices still vary smoothly across edges. If the MSE does not tend to zero as m grows — or if it grows — then the uniform Schur-stability assumption is genuinely load-bearing. Equivalently, compute the constants in Corollary 1: as ρ_max approaches 1, the required T ≳ v/Δ² diverges, so the small-T claim should fail in simulation.
If this is right
- On complete graphs and dense Erdős–Rényi graphs, a single short trajectory per node (T constant) is sufficient for weak consistency as m grows, provided the true matrices vary smoothly or have few jumps across edges.
- On star graphs and 2D grids, T need only grow poly-logarithmically with m and d for consistency; the joint estimator's MSE is o(1) while naive node-wise OLS has O(1) MSE in m.
- The choice of support set S splits into a smooth regime (S=∅) and a few-changes regime (S=supp(eDa*)), with a quantitative switch criterion: ∥eDa*∥₁ ≲ (M/√T)·(s/κ_S²) favors S=∅ when changes are many but tiny, and S=supp(eDa*) when changes are few but large.
- The bound separates bias from off-support total variation: the term √(8λm/T ‖(eDa*)_{S^c}‖₁) shows explicitly how misspecification of the smoothness (large off-support TV) degrades the rate.
- The estimator's advantage over Laplacian smoothing is most pronounced for piecewise-constant matrices, where TV preserves sharp edges; experiments confirm this on synthetic graphs and on EPA air-quality monitoring data.
Where Pith is reading between the lines
- An implication left implicit is a design principle for networked time series: when trajectories are short, increasing the number of observed nodes can substitute for longer observation windows, as long as the graph is well connected and the dynamics are stable; this suggests a testable 'more nodes, same T' experimental protocol.
- The proof structure suggests that the Fiedler eigenvalue λ_{m−1} is the fundamental resource: condition (2.9) essentially requires µ′√log(|E|d/δ) ≲ 1, which for many graphs translates to λ_{m−1} growing with m — indicating that small-T consistency should fail on graphs with bottlenecks even if they are technically connected.
- A natural extension, not addressed in the paper, is asynchronous or misaligned node trajectories; since the estimator only uses each node's own trajectory and the graph penalty couples parameter estimates, a permutation-invariant variant could be developed, though the martingale arguments would need adjustment if noise is also correlated across nodes.
- The constants in Example 1 indicate that the smoothness requirement scales with d³/Δ, so for high-dimensional states (large d) the practical benefit of graph pooling diminishes; this could be tested by varying d while keeping m, T, and the graph fixed.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies joint estimation of m linear dynamical systems on a graph, using a total-variation penalized least-squares estimator. The main theorem gives high-probability non-asymptotic bounds on the estimation error, expressed through graph-geometry parameters, controllability Grammians, and a dispersion functional. Corollaries for uniformly Schur-stable systems yield rates for smooth and few-jump regimes, and the paper claims weak consistency as m→∞ even with T fixed for well-connected graphs (complete and Erdős–Rényi), while star and grid graphs require only poly-logarithmic growth of T. The theoretical results are complemented by synthetic and EPA-air-quality experiments.
Significance. If the proof issues are fixed, this is a valuable contribution: it provides the first non-asymptotic guarantees for graph-TV-penalized joint LDS estimation with dependent design and dependent noise, and it introduces a range–nullspace decomposition of the RE condition that is tailored to the graph-TV operator. The explicit dependence on µ, µ′, κ_S, β, and Δ_G is informative, and the paper is careful to separate estimator structure from dynamics assumptions. The experiments are extensive and the code is provided. The paper is methodologically serious and the central claims are plausible, but the proof of the RE condition currently contains a load-bearing scaling inconsistency that must be corrected.
major comments (2)
- [Section 3.5, Eq. (3.37)–(3.38) and Theorem 1 condition (iii)] The displayed bound (3.37) scales as M ≤ dµ/√m Δ_G + cβ²µ√(T/m)log = (µ/√m)(dΔ_G + cβ²√T log). Multiplying by the bracket in (3.31) can only yield a cross-term bound proportional to µ/√m. However, Lemma 5's statement and Eq. (3.38) replace µ/√m by µ√m, and the 'Putting everything together' step uses this larger factor in condition (iii). This is not a harmless typo: Theorem 1 condition (iii) is not proven as stated, and Corollary 3 and Examples 1–4 inherit the wrong m-power. The lemma must be corrected to µ/√m and the proof steps reconciled. Because the corrected factor is smaller, the small-T consistency conclusion is likely preserved, but the stated theorem is not certified as written.
- [Section 2.4, (2.11)–(2.12), Corollary 3, Examples 1–4] Once Lemma 5 is corrected, the sample-size side conditions must be re-derived. Condition (C3b) should become T≥CµΦ_S dΔ_G/√m rather than T≥Cµ√mΦ_S dΔ_G. Moreover, the small-T conditions in Corollary 3 do not follow algebraically from the stated (2.11)–(2.12): substituting T≍log²(1/δ)/Δ² into (2.12) gives µΦ_S ≲ log²/(√m dΔ_GΔ²), not the √m log²/(dΔ_GΔ²) printed in Corollary 3(ii). The thresholds in Examples 1 and 2 (e.g., the m^{3/2} condition) are therefore off by a further factor of m. The quantitative small-T claims need a careful re-derivation.
minor comments (4)
- [Lemmas 3 and 4] The probability statements are written as P(...) ≤ 2exp(...); from the context and usage, the inequality should be ≥. As printed, the lemmas assert that the desired event has small probability.
- [Lemma 5 statement, Eq. (3.8)] The statement uses ∥(Da*)_{S^c}∥₁ while the proof and Theorem 1 use ∥(eDa*)_{S^c}∥₁. Use the tilde consistently.
- [Section 3.3] The sentence 'We now set t=G√v for any v≥1, t=G√v for any v≥1' is duplicated.
- [Introduction / Section 2.4] The abstract says 'MSE goes to zero ... even when T is constant' without mentioning the uniform Schur-stability condition (2.7). This condition is load-bearing and should be stated more prominently in the abstract or introduction.
Circularity Check
No circularity: the MSE bounds are oracle-type bounds in terms of true difficulty parameters, not fitted inputs or self-imported uniqueness claims.
full rationale
Walking the derivation chain: Theorem 1 follows from the optimality inequality (3.1)–(3.6), the restricted-eigenvalue decomposition in (3.7), and Lemmas 3–5. The error bounds depend on true but unknown quantities — β, Δ_G, μ, μ′, κ_S and ∥(eDa*)_{S^c}∥_1 — which are difficulty parameters, not fitted values. No step defines the target through the estimator or fits a parameter and then renames it a prediction. Corollaries 1–3 and Examples 1–4 are instantiations of Theorem 1 using graph-geometric bounds from Table 1, Propositions 1–2 and Lemma 9; they are not obtained by assuming the conclusion. The self-citations ([35], [36], [37], [38]) are contextual comparisons or standard facts; the proof’s load-bearing external tools are independent ([1], [11], [12], [13], [15]). I also note, as a correctness issue rather than a circularity, an apparent factor-of-m inconsistency between Eq. (3.37) and Lemma 5 / Theorem 1 condition (iii): (3.37) gives a cross-term bound of order μ/√m·(dΔ_G + β²√T log), while (3.38) states μ√m·(...). This affects the stated sample-size conditions but does not make the argument circular.
Axiom & Free-Parameter Ledger
free parameters (1)
- regularization parameter λ =
selected by validation in experiments; theory requires λ ≥ (2/m)max{F1,F2} and suggests (2.8)
axioms (5)
- domain assumption Noise η_{l,t} are i.i.d. centered standard Gaussian (or subgaussian with bounded norms)
- domain assumption Schur stability ∥A*_l∥₂ ≤ ρ_max < 1 for all l (for corollaries)
- domain assumption Graph G is known, undirected, connected
- domain assumption Smoothness/few-jumps of A*_l across edges (small ∥eDa*∥₁ or support S)
- domain assumption Initial state x_{l,0}=0 for each l
read the original abstract
We consider the problem of joint estimation of the parameters of $m$ linear dynamical systems, given access to single realizations of their respective trajectories, each of length $T$. The linear systems are assumed to reside on the nodes of an undirected and connected graph $G = ([m], \mathcal{E})$, and the system matrices are assumed to either vary smoothly or exhibit small number of ``jumps'' across the edges. We consider a total variation penalized least-squares estimator and derive non-asymptotic bounds on the mean squared error (MSE) which hold with high probability. In particular, the bounds imply for certain choices of well connected $G$ that the MSE goes to zero as $m$ increases, even when $T$ is constant. The theoretical results are supported by extensive experiments on synthetic and real data.
Figures
Reference graph
Works this paper leans on
-
[1]
Improved algorithms for linear stochastic bandits
Yasin Abbasi-yadkori, D´ avid P´ al, and Csaba Szepesv´ ari. Improved algorithms for linear stochastic bandits. InAdvances in Neural Information Processing Systems, volume 24, 2011
2011
-
[2]
Regularized estimation in sparse high-dimensional time series models.The Annals of Statistics, 43(4):1535–1567, 2015
Sumanta Basu and George Michailidis. Regularized estimation in sparse high-dimensional time series models.The Annals of Statistics, 43(4):1535–1567, 2015
2015
-
[3]
Network granger causality with inherent grouping structure.Journal of Machine Learning Research, 16(13):417–453, 2015
Sumanta Basu, Ali Shojaie, and George Michailidis. Network granger causality with inherent grouping structure.Journal of Machine Learning Research, 16(13):417–453, 2015
2015
-
[4]
Linearized aerodynamic and control law models of the x-29a airplane and comparison with flight data.National Aeronautics and Space Administration, Office of Management
John T Bosworth. Linearized aerodynamic and control law models of the x-29a airplane and comparison with flight data.National Aeronautics and Space Administration, Office of Management . . ., 4356, 1992
1992
-
[5]
Oxford University Press, 2013
St´ ephane Boucheron, G´ abor Lugosi, and Pascal Massart.Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford University Press, 2013
2013
-
[6]
Campi and E
M.C. Campi and E. Weyer. Finite sample properties of system identification methods.IEEE Transactions on Automatic Control, 47(8):1329–1334, 2002
2002
-
[7]
Ospina, Fabio Pasqualetti, and Emiliano Dall’Anese
Yiting Chen, Ana M. Ospina, Fabio Pasqualetti, and Emiliano Dall’Anese. Multi-task system identification of similar linear time-invariant dynamical systems.2023 62nd IEEE Conference on Decision and Control (CDC), pages 7342–7349, 2023
2023
-
[8]
Cormen, Charles E
Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein.Introduction to Algorithms, Third Edition. The MIT Press, 3rd edition, 2009
2009
-
[9]
Learning sparse dynamical systems from a single sample trajectory
Salar Fattahi, Nikolai Matni, and Somayeh Sojoudi. Learning sparse dynamical systems from a single sample trajectory. In2019 IEEE 58th Conference on Decision and Control (CDC), pages 2682–2689, 2019
2019
-
[10]
Telesford, Alfred B
Shi Gu, Fabio Pasqualetti, Matthew Cieslak, Qawi K. Telesford, Alfred B. Yu, Ari E. Kahn, John D. Medaglia, Jean M. Vettel, Michael B. Miller, Scott T. Grafton, and Danielle S. Bassett. Controllability of structural brain networks.Nature Communications, 6, 2014
2014
-
[11]
A tail inequality for quadratic forms of subgaus- sian random vectors.Electronic Communications in Probability, 17:1–6, 2012
Daniel Hsu, Sham Kakade, and Tong Zhang. A tail inequality for quadratic forms of subgaus- sian random vectors.Electronic Communications in Probability, 17:1–6, 2012
2012
-
[12]
Optimal rates for total variation denoising
Jan-Christian H¨ utter and Philippe Rigollet. Optimal rates for total variation denoising. vol- ume 49, pages 1115–1146, 2016
2016
-
[13]
Finite-time identification of stable linear systems opti- mality of the least-squares estimator
Yassir Jedra and Alexandre Proutiere. Finite-time identification of stable linear systems opti- mality of the least-squares estimator. In2020 59th IEEE Conference on Decision and Control (CDC), pages 996–1001, 2020
2020
-
[14]
Oracle inequalities for high dimensional vector autoregressions.Journal of Econometrics, 186(2):325–344, 2015
Anders Bredahl Kock and Laurent Callot. Oracle inequalities for high dimensional vector autoregressions.Journal of Econometrics, 186(2):325–344, 2015
2015
-
[15]
Suprema of chaos processes and the restricted isometry property.Communications on Pure and Applied Mathematics, 67(11):1877– 1904, 2014
Felix Krahmer, Shahar Mendelson, and Holger Rauhut. Suprema of chaos processes and the restricted isometry property.Communications on Pure and Applied Mathematics, 67(11):1877– 1904, 2014. 50
1904
-
[16]
Asymptotic properties of general autoregressive models and strong consistency of least-squares estimates of their parameters.Journal of Multivariate Analysis, 13(1):1–23, 1983
T.L Lai and C.Z Wei. Asymptotic properties of general autoregressive models and strong consistency of least-squares estimates of their parameters.Journal of Multivariate Analysis, 13(1):1–23, 1983
1983
-
[17]
Extended least squares and their applications to adaptive control and prediction in linear systems.IEEE Transactions on Automatic Control, 31(10):898–906, 1986
Tze Lai and Ching-Zong Wei. Extended least squares and their applications to adaptive control and prediction in linear systems.IEEE Transactions on Automatic Control, 31(10):898–906, 1986
1986
-
[18]
Least Squares Estimates in Stochastic Regression Models with Applications to Identification and Control of Dynamic Systems.The Annals of Statistics, 10(1):154 – 166, 1982
Tze Leung Lai and Ching Zong Wei. Least Squares Estimates in Stochastic Regression Models with Applications to Identification and Control of Dynamic Systems.The Annals of Statistics, 10(1):154 – 166, 1982
1982
-
[19]
Variable fusion: A new adaptive signal regression method.Dept
Stephanie R Land and Jerome H Friedman. Variable fusion: A new adaptive signal regression method.Dept. Statistics, Carnegie Mellon Univ. Pittsburgh, Pittsburgh, PA, USA, Rep, 656, 1997
1997
-
[20]
Graph-based regularization for regression problems with alignment and highly corre- lated designs.SIAM Journal on Mathematics of Data Science, 2(2):480–504, 2020
Yuan Li, Benjamin Mark, Garvesh Raskutti, Rebecca Willett, Hyebin Song, and David Neiman. Graph-based regularization for regression problems with alignment and highly corre- lated designs.SIAM Journal on Mathematics of Data Science, 2(2):480–504, 2020
2020
-
[21]
Wainwright
Po-Ling Loh and Martin J. Wainwright. High-dimensional regression with noisy and missing data: Provable guarantees with nonconvexity.The Annals of Statistics, 40(3):1637 – 1664, 2012
2012
-
[22]
Estimating structured vector autoregressive models
Igor Melnyk and Arindam Banerjee. Estimating structured vector autoregressive models. In Proceedings of The 33rd International Conference on Machine Learning, pages 830–839, 2016
2016
-
[23]
Joint learning of linear time-invariant dynamical systems.Automatica, 164:111635, 2024
Aditya Modi, Mohamad Kazem Shirani Faradonbeh, Ambuj Tewari, and George Michailidis. Joint learning of linear time-invariant dynamical systems.Automatica, 164:111635, 2024
2024
-
[24]
Prediction bounds for higher order total variation regularized least squares.The Annals of Statistics, 49(5):2755–2773, 2021
Francesco Ortelli and Sara van de Geer. Prediction bounds for higher order total variation regularized least squares.The Annals of Statistics, 49(5):2755–2773, 2021
2021
-
[25]
Wainwright, and Bin Yu
Garvesh Raskutti, Martin J. Wainwright, and Bin Yu. Restricted eigenvalue properties for correlated gaussian designs.J. Mach. Learn. Res., 11:2241–2259, 2010
2010
-
[26]
Total variation classes beyond 1d: Minimax rates, and the limitations of linear smoothers.Advances in Neural Information Processing Systems, 29:3521–3529, 2016
Veeranjaneyulu Sadhanala, Yu-Xiang Wang, and Ryan J Tibshirani. Total variation classes beyond 1d: Minimax rates, and the limitations of linear smoothers.Advances in Neural Information Processing Systems, 29:3521–3529, 2016
2016
-
[27]
Near optimal finite time identification of arbitrary linear dynamical systems
Tuhin Sarkar and Alexander Rakhlin. Near optimal finite time identification of arbitrary linear dynamical systems. InProceedings of the 36th International Conference on Machine Learning, ICML, volume 97, pages 5610–5618, 2019
2019
-
[28]
Finite time identification in unstable linear systems.Automatica, 96:342–353, 2018
Mohamad Kazem Shirani Faradonbeh, Ambuj Tewari, and George Michailidis. Finite time identification in unstable linear systems.Automatica, 96:342–353, 2018
2018
-
[29]
Jordan, and Benjamin Recht
Max Simchowitz, Horia Mania, Stephen Tu, Michael I. Jordan, and Benjamin Recht. Learning without mixing: Towards a sharp analysis of linear system identification. InProceedings of the 31st Conference On Learning Theory, volume 75, pages 439–473, 2018. 51
2018
-
[30]
Kim, Harang Ju, Dale Zhou, Cassiano Becker, Fabio Pasqualetti, George J
Pragya Srivastava, Erfan Nozari, Jason Z. Kim, Harang Ju, Dale Zhou, Cassiano Becker, Fabio Pasqualetti, George J. Pappas, and Danielle S. Bassett. Models of communication and control for brain networks: distinctions, convergence, and future outlook.Network Neuroscience, 4(4):1122–1159, 2020
2020
-
[31]
Wellesley-Cambridge Press, Philadel- phia, PA, 2007
Gilbert Strang.Computational Science and Engineering. Wellesley-Cambridge Press, Philadel- phia, PA, 2007
2007
-
[32]
Springer, 2005
Michel Talagrand.The Generic Chaining: Upper and Lower Bounds of Stochastic Processes. Springer, 2005
2005
-
[33]
Springer, 2014
Michel Talagrand.Upper and lower bounds for stochastic processes, volume 60. Springer, 2014
2014
-
[34]
Stanford University, 2011
Ryan J Tibshirani.The solution path of the generalized lasso. Stanford University, 2011
2011
-
[35]
Huy Tran, Sansen Wei, and Claire Donnat. The generalized elastic net for least squares regression with network-aligned signal and correlated design.IEEE Transactions on Signal and Information Processing over Networks, 2025
2025
-
[36]
Joint estimation of smooth graph signals from partial linear measurements
Hemant Tyagi. Joint estimation of smooth graph signals from partial linear measurements. arXiv:2505.23240, 2025
Pith/arXiv arXiv 2025
-
[37]
Joint learning of linear dynamical systems under smoothness constraints
Hemant Tyagi. Joint learning of linear dynamical systems under smoothness constraints. Information and Inference: A Journal of the IMA, 14(3):iaaf026, 2025
2025
-
[38]
Learning linear dynamical systems under convex constraints
Hemant Tyagi and Denis Efimov. Learning linear dynamical systems under convex constraints. arxiv:2303.15121, 2025
Pith/arXiv arXiv 2025
-
[39]
Cambridge University Press, 2025
Roman Vershynin.High-Dimensional Probability: An Introduction with Applications in Data Science (2nd ed.). Cambridge University Press, 2025
2025
-
[40]
Vidyasagar and R.L
M. Vidyasagar and R.L. Karandikar. A learning theory approach to system identification and stochastic adaptive control.Journal of Process Control, 18(3):421–430, 2008
2008
-
[41]
Di Wang and Ruey S. Tsay. Rate-optimal robust estimation of high-dimensional vector au- toregressive models.The Annals of Statistics, 51(2):846 – 877, 2023
2023
-
[42]
Fedsysid: A federated approach to sample-efficient system identification
Han Wang, Leonardo Felipe Toso, and James Anderson. Fedsysid: A federated approach to sample-efficient system identification. InProceedings of The 5th Annual Learning for Dynamics and Control Conference, volume 211, pages 1308–1320, 2023
2023
-
[43]
Trend filtering on graphs.Journal of Machine Learning Research, 17(105):1–41, 2016
Yu-Xiang Wang, James Sharpnack, Alexander J Smola, and Ryan J Tibshirani. Trend filtering on graphs.Journal of Machine Learning Research, 17(105):1–41, 2016
2016
-
[44]
Learning the dynamics of autonomous linear systems from multiple trajectories
Lei Xin, George Chiu, and Shreyas Sundaram. Learning the dynamics of autonomous linear systems from multiple trajectories. In2022 American Control Conference (ACC), pages 3955– 3960, 2022
2022
-
[45]
Chiu, and Shreyas Sundaram
Lei Xin, Lintao Ye, George T.-C. Chiu, and Shreyas Sundaram. Identifying the dynamics of a system by leveraging data from similar systems.2022 American Control Conference (ACC), pages 818–824, 2022
2022
-
[46]
Non-asymptotic identification of linear dynamical systems using multiple trajectories.IEEE Control Systems Letters, 5(5):1693–1698, 2020
Yang Zheng and Na Li. Non-asymptotic identification of linear dynamical systems using multiple trajectories.IEEE Control Systems Letters, 5(5):1693–1698, 2020. 52
2020
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.