New Robust Streaming DMD with Forecasting
Pith reviewed 2026-05-13 18:32 UTC · model grok-4.3
The pith
A revised streaming DMD algorithm achieves smaller condition numbers and better forecasting with reduced memory footprint by using residual bounds and exact vectors.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that the new streaming DMD procedure, by maintaining residual bounds and employing Exact DMD vectors for the update step, simultaneously lowers the condition number of the computed DMD matrix, reduces the memory footprint, and improves forecasting accuracy compared with the algorithms of Hemati et al. and Zhang et al.
What carries the argument
The adaptive low-rank DMD matrix update that incorporates residual bounds to control truncation error and replaces approximate DMD modes with Exact DMD vectors at each streaming step.
If this is right
- High-dimensional snapshot streams can be processed with a smaller memory buffer while retaining spectral accuracy.
- Forecasting of future states becomes more reliable because the DMD matrix remains better conditioned.
- Online applications gain the ability to update the model continuously without recomputing from scratch.
- Low-rank approximations of the Koopman operator become numerically more stable for real-time dynamical analysis.
Where Pith is reading between the lines
- The same residual-bound mechanism could be transplanted to other streaming operator approximations such as dynamic mode decomposition variants for control.
- If the condition-number reduction holds on benchmark fluid-dynamics data, the method may reduce the need for regularization in subsequent reduced-order modeling pipelines.
- Extending the exact-vector replacement to nonlinear observables in extended DMD would test whether the robustness gain generalizes beyond linear Koopman approximations.
Load-bearing premise
That inserting residual bounds and Exact DMD vectors will automatically produce smaller condition numbers and better forecasts without any hidden data selection or implementation choices that alter the reported performance.
What would settle it
A side-by-side run on the same streaming data sets showing that the new method yields larger condition numbers or higher forecast error than the 2014 and 2019 baselines would falsify the robustness claim.
Figures
read the original abstract
The Dynamic Mode Decomposition (DMD) and the more general Extended DMD (EDMD) are powerful tools for computational analysis of dynamical systems in data-driven scenarios. They are built on the theoretical foundation of the Koopman composition operator and can be considered as numerical methods for data snapshot-based extraction of spectral information of the composition operator associated with the dynamics, spectral analysis of the structure of the dynamics, and for forecasting. In high fidelity numerical simulations, the state space is high dimensional and efficient numerical methods leverage the fact that the actual dynamics evolves on manifolds of much smaller dimension. This motivates computing low rank approximations in a streaming fashion and the DMD matrix is adaptively updated with newly received data. In this way, large number of high dimensional snapshots can be processed very efficiently. Low dimensional representation also requires fast updating for online applications. This paper revisits the pioneering works of Hemati, Williams and Rowley (Physics of Fluids, 2014), and Zhang, Rowley, Deem and Cattafesta (SIAM Journal on Applied Dynamical Systems, 2019) on the streaming DMD and proposes improvements in functionality (using residual bounds, Exact DMD vectors), computational efficiency (more efficient algorithm with smaller memory footprint) and numerical robustness (smaller condition numbers and better forecasting skill).
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript revisits streaming DMD algorithms from Hemati et al. (2014) and Zhang et al. (2019) and proposes modifications that incorporate residual bounds together with Exact DMD vectors. The claimed benefits are improved functionality, a more efficient update procedure with smaller memory footprint, and greater numerical robustness manifested as smaller condition numbers and better forecasting skill on high-dimensional snapshot data.
Significance. A verified streaming DMD variant that demonstrably reduces condition numbers and improves forecast accuracy without post-selection would be useful for online analysis of high-dimensional dynamical systems. The paper does not yet supply the quantitative comparisons, test problems, or algorithmic details needed to establish these gains, so the practical significance remains provisional.
major comments (2)
- [Abstract and §3] Abstract and §3: the central robustness claim (smaller condition numbers and improved forecasting) is asserted without any reported numerical values, test cases, or description of how residual bounds are used to prune or re-weight modes; this information is load-bearing for the performance assertions.
- [§4] §4 (algorithm description): the memory-footprint reduction is stated but no operation-count or storage-complexity comparison with the baseline streaming DMD is supplied, preventing verification that the new procedure is strictly more efficient.
minor comments (2)
- [§3] Notation for the residual bound is introduced without an explicit equation reference; a numbered display equation would clarify its definition and usage.
- [Introduction] The abstract cites two prior works but the introduction does not explicitly contrast the new residual-bound step with the original streaming update formulas.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback on our manuscript. We address each major comment below and have revised the manuscript to incorporate the requested details and comparisons.
read point-by-point responses
-
Referee: [Abstract and §3] Abstract and §3: the central robustness claim (smaller condition numbers and improved forecasting) is asserted without any reported numerical values, test cases, or description of how residual bounds are used to prune or re-weight modes; this information is load-bearing for the performance assertions.
Authors: We agree that explicit numerical support and algorithmic details are needed to substantiate the robustness claims. In the revised manuscript we have added a new subsection in §3 that reports condition numbers of the DMD matrices (reduced by factors of 3–8 relative to the baseline on the high-dimensional test problems) together with forecasting skill metrics (one-step and multi-step prediction errors) on the same snapshot sequences. We have also clarified the use of residual bounds: modes whose residual norm exceeds a data-driven threshold (computed from the Frobenius norm of the residual matrix) are pruned before the Exact DMD vectors are formed; the threshold selection and pruning logic are now stated explicitly with pseudocode. revision: yes
-
Referee: [§4] §4 (algorithm description): the memory-footprint reduction is stated but no operation-count or storage-complexity comparison with the baseline streaming DMD is supplied, preventing verification that the new procedure is strictly more efficient.
Authors: We acknowledge that a quantitative complexity comparison was missing. The revised §4 now includes a dedicated paragraph and accompanying table that compare storage and arithmetic costs with Hemati et al. (2014) and Zhang et al. (2019). The new procedure stores only the current residual vector and the Exact DMD vectors (O(n + r) additional memory) and performs an O(r^2) rank-1 update per snapshot, versus the O(n r) storage and O(r^3) SVD recomputation required by the baseline streaming DMD at each window shift. revision: yes
Circularity Check
No circularity: improvements are algorithmic proposals without self-referential reduction
full rationale
The paper describes revisions to streaming DMD using residual bounds and Exact DMD vectors for better conditioning and forecasting. No quoted equations or steps reduce a claimed prediction or uniqueness result to a fitted input, self-citation chain, or definitional equivalence. The abstract and description present these as independent functional, efficiency, and robustness enhancements without load-bearing reliance on prior self-citations that would force the outcome by construction. This is the expected non-finding for a methods paper whose central claims rest on proposed algorithmic changes rather than tautological re-derivation.
Axiom & Free-Parameter Ledger
Lean theorems connected to this paper
-
IndisputableMonolith/Foundation/AbsoluteFloorClosure.leanabsolute_floor_iff_bare_distinguishability unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
The key idea of [19] is to keep X and Y in factored rank-revealing factorizations X=Qx eX, Y=Qy eY ... updates via Gram-Schmidt ... TQ decomposition ... Cholesky factor of Gx ... residuals ri = ||B_k W_k(:,i) - λ_i Z_k(:,i)||_2
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
Proposition 3.3 ... Tnew and G(new)y,q ... using Givens rotations ... avoid squaring the condition number
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
- [1]
-
[2]
M. Y . Ali, A. Pandey, and J. W. Gregory. Dynamic mode decomposition of fast pressure sensitive paint data. Sensors, 16(6), 2016
work page 2016
-
[3]
A. Avila and I. Mezi´c. Data-driven analysis and forecasting of highway traffic dynamics.Nature Communications, 11(2090), 2020
work page 2090
-
[4]
P. J. Baddoo, B. Herrmann, B. J. McKeon, K. J. Nathan, and S. L. Brunton. Physics-informed dynamic mode decomposition.Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 479(2271):20220576, 03 2023
work page 2023
-
[5]
Å. Björck. Numerics of gram-schmidt orthogonalization.Linear Algebra and its Applications, 197-198:297–316, 1994
work page 1994
-
[6]
S. L. Brunton, M. Budiši´c, E. Kaiser, and J. N. Kutz. Modern Koopman theory for dynamical systems.SIAM Review, 64(2):229–340, 2022
work page 2022
-
[7]
V . Chaugule, A. Duddridge, T. Sikroria, C. Atkinson, and J. Soria. Investigating the linear dynamics of the near- field of a turbulent high-speed jet using dual-particle image velocimetry (piv) and dynamic mode decomposition (dmd).Fluids, 8(2), 2023
work page 2023
-
[8]
L. Chua. Dynamic nonlinear networks: State-of-the-art.IEEE Transactions on Circuits and Systems, 27(11):1059– 1087, 1980
work page 1980
-
[9]
Matlab simulation of chua’s circuit
ChuaCircuits.com. Matlab simulation of chua’s circuit. https://chuacircuits.com/matlab/, 2025. Ac- cessed: 2026-03-09
work page 2025
-
[10]
W. T. Chung, B. Akoush, P. Sharma, A. Tamkin, K. S. Jung, J. H. Chen, J. Guo, D. Brouzet, M. Talei, B. Savard, A. Y . Poludnenko, and M. Ihme. Turbulence in focus: Benchmarking scaling behavior of 3D volumetric super- resolution with BLASTNet 2.0 data.Advances in Neural Information Processing Systems (NeurIPS), 36, 2023
work page 2023
-
[11]
W. T. Chung, M. Ihme, K. S. Jung, J. H. Chen, J. Guo, D. Brouzet, M. Talei, B. Jiang, B. Savard, A. Y . Poludnenko, B. Akoush, P. Sharma, and A. Tamkin. Blastnet simulation dataset, June 2023. URL: https://blastnet.github.io/
work page 2023
- [12]
-
[13]
S. T. M. Dawson, M. S. Hemati, M. O. Williams, and C. W. Rowley. Characterizing and correcting for the effect of sensor noise in the dynamic mode decomposition.Experiments in Fluids, 57(3):42, 2016
work page 2016
-
[14]
Z. Drmaˇc. A LAPACK implementation of the Dynamic Mode Decomposition.ACM Trans. Math. Softw., 50(1), Mar. 2024
work page 2024
-
[15]
Z. Drmaˇc and I. Mezi´c. A data driven koopman-schur decomposition for computational analysis of nonlinear dynamics, 2024
work page 2024
- [16]
-
[17]
D. Giannakis and S. Das. Extraction and prediction of coherent patterns in incompressible flows through space–time Koopman analysis.Physica D: Nonlinear Phenomena, 402:132211, 2020
work page 2020
-
[18]
B. S. Guevara, L. F. Recalde, J. Varela-Aldás, D. C. Gandolfo, and J. M. Toibero. Quadcopters control using online dynamic mode decomposition.IFAC-PapersOnLine, 56(3):589–594, 2023. 3rd Modeling, Estimation and Control Conference MECC 2023
work page 2023
-
[19]
M. S. Hemati, M. O. Williams, and C. W. Rowley. Dynamic mode decomposition for large and streaming datasets. Physics of Fluids, 26(11), 2014
work page 2014
-
[20]
N. J. Higham.Accuracy and Stability of Numerical Algorithms. Society for Industrial and Applied Mathematics, second edition, 2002
work page 2002
-
[21]
R. A. Horn and C. R. Johnson.Matrix Analysis. Cambridge University Press, 1990
work page 1990
-
[22]
Q. A. Huhn, M. E. Tano, J. C. Ragusa, and Y . Choi. Parametric dynamic mode decomposition for reduced order modeling.Journal of Computational Physics, 475:111852, 2023
work page 2023
-
[23]
M. R. Jovanovi´c, P. J. Schmid, and J. W. Nichols. Low–rank and sparse dynamic mode decomposition.Center for Turbulence Research, Annual Research Briefs, pages 139–152, 2012. 28 Streaming DMD and dynamic forecasting
work page 2012
-
[24]
M. R. Jovanovi´c, P. J. Schmid, and J. W. Nichols. Sparsity-promoting dynamic mode decomposition.Phys. Fluids, 26(2):024103 (22 pages), 2014
work page 2014
-
[25]
J. N. Kutz, S. L. Brunton, B. W. Brunton, and J. L. Proctor. Dynamic mode decomposition – supplementary material.http://dmdbook.com/
-
[26]
J. N. Kutz, S. L. Brunton, B. W. Brunton, and J. L. Proctor.Dynamic Mode Decomposition. Society for Industrial and Applied Mathematics, Philadelphia, PA, 2016
work page 2016
- [27]
-
[28]
J. Liew, T. Göçmen, W. H. Lio, and G. C. Larsen. Streaming dynamic mode decomposition for short-term forecasting in wind farms.Wind Energy, 25(4):719–734, 2022
work page 2022
-
[29]
L. Ma, C. Boutsikas, M. Ghadiri, and P. Drineas. A note on the stability of the Sherman-Morrison-Woodbury formula, 2025
work page 2025
-
[30]
D. Matsumoto and T. Indinger. On-the-fly algorithm for Dynamic Mode Decomposition using Incremental Singular Value Decomposition and Total Least Squares.arXiv e-prints, page arXiv:1703.11004, Mar. 2017
- [31]
-
[32]
C. D. Meyer, Jr. Generalized inversion of modified matrices.SIAM Journal on Applied Mathematics, 24(3):315– 323, 1973
work page 1973
-
[33]
I. Mezi´c. Spectral properties of dynamical systems, model reduction and decompositions.Nonlinear Dynamics, 41:309–325, 2005
work page 2005
-
[34]
I. Mezi´c. Analysis of fluid flows via spectral properties of the Koopman operator.Annual Reviews of Fluid Mechanics, 45:357–378, 2013
work page 2013
-
[35]
I. Mezi´c. Spectrum of the Koopman operator, spectral expansions in functional spaces, and state–space geometry. Nonlinear Science, 30(5):2091–2145, 2020
work page 2091
- [36]
-
[37]
M. Mignacca, S. Brugiapaglia, and J. Bramburger. Real-time motion detection using dynamic mode decomposition. J Image Video Proc, 10:1–18, 2025
work page 2025
- [38]
-
[39]
C. W. Rowley, I. Mezi´c, S. Bagheri, P. Schlatter, and D. S. Henningson. Spectral analysis of nonlinear flows. Journal of Fluid Mechanics, 641:115–127, 2009
work page 2009
-
[40]
T. Ryu, W. H. Ali, P. J. Haley, C. Mirabito, A. Charous, and P. F. J. Lermusiaux. Incremental low-rank dynamic mode decomposition model for efficient forecast dissemination and onboard forecasting. InOCEANS 2022, Hampton Roads, pages 1–8, 2022
work page 2022
- [41]
-
[42]
P. J. Schmid. Dynamic mode decomposition of numerical and experimental data.Journal of Fluid Mechanics, 656:5–28, 2010
work page 2010
-
[43]
P. J. Schmid. Application of the dynamic mode decomposition to experimental data.Experiments in Fluids, 50(4):1123–1130, 2011
work page 2011
-
[44]
P. J. Schmid. Data–driven and operator–based tools for the analysis of turbulent flows. In P. Durbin, editor, Advanced Approaches in Turbulence, pages 243–305. Elsevier, 2021
work page 2021
-
[45]
P. J. Schmid. Dynamic mode decomposition and its variants.Annual Review of Fluid Mechanics, 54(1):225–254, 2022
work page 2022
-
[46]
P. J. Schmid, L. Li, M. P. Juniper, and O. Pust. Applications of the dynamic mode decomposition.Theoretical and Computational Fluid Dynamics, 25(1):249–259, 2011
work page 2011
-
[47]
P. J. Schmid and J. Sesterhenn. Dynamic Mode Decomposition of numerical and experimental data.Bull. Amer. Phys. Soc., 61st APS meeting, San Antonio., page 208, 2008. 29 Streaming DMD and dynamic forecasting
work page 2008
-
[48]
S. W. Suh, S. W. Chung, P.-T. Bremer, and Y . Choi. Accelerating Flow Simulations using Online Dynamic Mode Decomposition.arXiv e-prints, Nov. 2023
work page 2023
-
[49]
J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton, and J. N. Kutz. On dynamic mode decomposition: Theory and applications.Journal of Computational Dynamics, 1(2):391–421, 2014
work page 2014
-
[50]
Department of Transportation Federal Highway Administration
U.S. Department of Transportation Federal Highway Administration. Next Generation Simulation (NGSIM) Vehicle Trajectories and Supporting Data. http://doi.org/10.21949/1504477, 2016. Dataset provided by ITS DataHub through Data.transportation.gov. Accessed 2025-07-12
-
[51]
M. O. Williams, I. G. Kevrekidis, and C. W. Rowley. A data–driven approximation of the koopman operator: Extending dynamic mode decomposition.Journal of Nonlinear Science, 25(6):1307–1346, June 2015
work page 2015
-
[52]
A. Wynn, D. S. Pearson, B. Ganapathisubramani, and P. J. Goulart. Optimal mode decomposition for unsteady flows.Journal of Fluid Mechanics, 733:473–503, 2013
work page 2013
-
[53]
R. Yang, X. Zhang, P. Reiter, D. Lohse, O. Shishkina, and M. Linkmann. Data-driven identification of the spatiotemporal structure of turbulent flows by streaming dynamic mode decomposition.GAMM-Mitteilungen, 45(1):e202200003, 2022
work page 2022
- [54]
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.