Higher-Order LaSDI: Reduced Order Modeling with Multiple Time Derivatives
Pith reviewed 2026-05-21 16:16 UTC · model grok-4.3
The pith
A high-order finite-difference scheme paired with rollout loss keeps reduced-order models accurate over long time horizons on PDEs.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
By introducing a flexible high-order yet inexpensive finite-difference scheme and a rollout loss that trains ROMs to make accurate predictions over arbitrary time horizons, higher-order LaSDI maintains predictive power on parameterized families of PDEs such as the 2D Burgers equation.
What carries the argument
The high-order finite-difference scheme that incorporates multiple time derivatives together with the rollout loss for training over extended sequences.
If this is right
- Reduced-order models trained this way sustain accuracy across longer simulation intervals without rapid error accumulation.
- The same scheme and loss apply to other parameterized PDEs beyond the Burgers equation.
- Training remains inexpensive while extending the reliable prediction window.
- The approach supports families of solutions rather than single instances.
Where Pith is reading between the lines
- Real-time control or design optimization loops could use these models for extended forecasts where full simulations are too slow.
- The method may transfer to other latent dynamical systems in machine learning that suffer from long-term drift.
- Testing on higher-dimensional or chaotic PDEs would clarify where the higher-order terms stop helping.
Load-bearing premise
The high-order finite-difference scheme combined with rollout training produces meaningfully better long-horizon accuracy than existing LaSDI or other ROM methods on the 2D Burgers equation.
What would settle it
A direct comparison of long-horizon prediction errors on the 2D Burgers equation where the new method shows no clear improvement over standard LaSDI baselines would falsify the central claim.
Figures
read the original abstract
Solving complex partial differential equations is vital in the physical sciences, but often requires computationally expensive numerical methods. Reduced-order models (ROMs) address this by exploiting dimensionality reduction to create fast approximations. While modern ROMs can solve parameterized families of PDEs, their predictive power degrades over long time horizons. We address this by (1) introducing a flexible, high-order, yet inexpensive finite-difference scheme and (2) proposing a Rollout loss that trains ROMs to make accurate predictions over arbitrary time horizons. We demonstrate our approach on the 2D Burgers equation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces Higher-Order LaSDI, extending Latent Space Dynamics Identification by incorporating multiple time derivatives through a flexible high-order finite-difference scheme and a rollout loss to train reduced-order models for accurate predictions over arbitrary time horizons. The approach is demonstrated on the 2D Burgers equation.
Significance. If the high-order scheme and rollout loss demonstrably improve long-horizon accuracy on nonlinear PDEs without introducing instability or excessive cost, the work would meaningfully advance ROM applicability in scientific computing for long-time simulations.
major comments (3)
- [Abstract] Abstract: The demonstration on the 2D Burgers equation is described without quantitative error metrics, baselines, implementation details, or comparisons to standard LaSDI, making it impossible to evaluate whether the proposed methods support the central claim of improved long-horizon predictions.
- [Finite-difference scheme description] Section on the finite-difference scheme: The claim that the high-order FD scheme is both accurate for multiple derivatives and inexpensive is load-bearing for the contribution, yet higher-order stencils increase stencil width and are known to amplify noise or require finer sampling; the manuscript provides no analysis of sensitivity to snapshot noise or discretization error in the 2D Burgers data.
- [Rollout loss formulation] Section on the rollout loss: The rollout loss is presented as addressing long-horizon degradation, but the manuscript does not show how it mitigates errors introduced by the high-order derivative estimation; without this, the combined method's robustness for arbitrary horizons remains unverified.
minor comments (2)
- [Notation and preliminaries] Clarify notation for the multiple time derivatives and how the FD scheme is embedded in the LaSDI framework.
- [Numerical results] Add a table or figure comparing computational cost and accuracy against baseline LaSDI variants.
Simulated Author's Rebuttal
We thank the referee for their constructive feedback on our manuscript. We have carefully considered each major comment and will make revisions to strengthen the presentation of our results.
read point-by-point responses
-
Referee: [Abstract] Abstract: The demonstration on the 2D Burgers equation is described without quantitative error metrics, baselines, implementation details, or comparisons to standard LaSDI, making it impossible to evaluate whether the proposed methods support the central claim of improved long-horizon predictions.
Authors: We agree that the abstract would benefit from more quantitative details to support our claims. In the revised manuscript, we will update the abstract to include specific quantitative error metrics for the 2D Burgers equation experiments, such as time-averaged relative errors, direct comparisons to standard LaSDI, and brief notes on the implementation details and baselines. This will allow readers to better assess the improvements in long-horizon predictions. revision: yes
-
Referee: [Finite-difference scheme description] Section on the finite-difference scheme: The claim that the high-order FD scheme is both accurate for multiple derivatives and inexpensive is load-bearing for the contribution, yet higher-order stencils increase stencil width and are known to amplify noise or require finer sampling; the manuscript provides no analysis of sensitivity to snapshot noise or discretization error in the 2D Burgers data.
Authors: The referee correctly identifies a potential limitation. Although our high-order scheme is formulated to be flexible and we chose orders that balance accuracy and cost for the smooth 2D Burgers solutions, we did not provide an explicit sensitivity analysis to noise or discretization errors. We will add this analysis in the revised manuscript, including a discussion of how the scheme performs under varying levels of snapshot noise and grid resolutions, to substantiate the claims of accuracy and computational inexpensiveness. revision: yes
-
Referee: [Rollout loss formulation] Section on the rollout loss: The rollout loss is presented as addressing long-horizon degradation, but the manuscript does not show how it mitigates errors introduced by the high-order derivative estimation; without this, the combined method's robustness for arbitrary horizons remains unverified.
Authors: We appreciate this point regarding the interplay between the rollout loss and derivative estimation. The rollout loss is intended to train the dynamics model to be robust over multiple time steps, thereby reducing the impact of any inaccuracies in the estimated derivatives. To make this clearer, we will revise the relevant section to include a more detailed explanation and additional results showing the effect of the rollout loss on mitigating derivative estimation errors, for example by ablating the rollout component and comparing long-term prediction accuracy. revision: yes
Circularity Check
No circularity: new FD scheme and rollout loss are independent proposals
full rationale
The paper introduces a high-order finite-difference scheme for multiple time derivatives and a Rollout loss for long-horizon ROM training as methodological contributions, then demonstrates them on the 2D Burgers equation. These elements do not reduce to self-definitional fits, renamed inputs, or load-bearing self-citations by construction. The derivation chain presents the schemes as novel and inexpensive additions to LaSDI-style ROMs without equating any prediction or result to its own fitted parameters or prior author work in a circular manner. The central claims remain independent of the inputs.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al.{TensorFlow}: a system for {Large-Scale} machine learning. In12th USENIX symposium on operating systems design and implementation (OSDI 16), pages 265–283, 2016
work page 2016
-
[2]
Robert Anderson, Julian Andrej, Andrew Barker, Jamie Bramwell, Jean-Sylvain Camier, Jakub Cerveny, Veselin Dobrev, Yohann Dudouit, Aaron Fisher, Tzanio Kolev, et al. Mfem: A modular finite element methods library.Computers & Mathematics with Applications, 81:42–74, 2021
work page 2021
-
[3]
mLaSDI: Multi-stage latent space dynamics identification
William Anderson, Seung Whan Chung, and Youngsoo Choi. mlasdi: Multi-stage latent space dynamics identification.arXiv preprint arXiv:2506.09207, 2025
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[4]
Julian Andrej, Nabil Atallah, Jan-Phillip Bäcker, Jean-Sylvain Camier, Dylan Copeland, Veselin Dobrev, Yohann Dudouit, Tobias Duswald, Brendan Keith, Dohyun Kim, et al. High-performance finite elements with mfem.The International Journal of High Performance Computing Applications, 38(5):447–467, 2024
work page 2024
-
[5]
Gal Berkooz, Philip Holmes, and John L Lumley. The proper orthogonal decomposition in the analysis of turbulent flows.Annual review of fluid mechanics, 25(1):539–575, 1993
work page 1993
-
[6]
Christophe Bonneville, Youngsoo Choi, Debojyoti Ghosh, and Jonathan L Belof. Data-driven autoen- coder numerical solver with uncertainty quantification for fast physical simulations.arXiv preprint arXiv:2312.01021, 2023
-
[7]
Christophe Bonneville, Youngsoo Choi, Debojyoti Ghosh, and Jonathan L Belof. GPLaSDI: Gaussian process-based interpretable latent space dynamics identification through deep autoencoder.Computer Methods in Applied Mechanics and Engineering, 418:116535, 2024
work page 2024
-
[8]
Christophe Bonneville, Xiaolong He, April Tran, Jun Sur Park, William Fries, Daniel A Messenger, Siu Wun Cheung, Yeonjong Shin, David M Bortz, Debojyoti Ghosh, et al. A comprehensive review of latent space dynamics identification algorithms for intrusive and non-intrusive reduced-order-modeling. arXiv preprint arXiv:2403.10748, 2024
-
[9]
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020
work page 1901
-
[10]
Steven L Brunton, Joshua L Proctor, and J Nathan Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical systems.Proceedings of the national academy of sciences, 113(15):3932–3937, 2016
work page 2016
-
[11]
Numerical analysis, brooks, 1997
Richard L Burden and J Douglas Faires. Numerical analysis, brooks, 1997
work page 1997
-
[12]
Kathleen Champion, Bethany Lusch, J Nathan Kutz, and Steven L Brunton. Data-driven discovery of coordinates and governing equations.Proceedings of the National Academy of Sciences, 116(45): 22445–22451, 2019
work page 2019
-
[13]
Neural ordinary differential equations.Advances in neural information processing systems, 31, 2018
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations.Advances in neural information processing systems, 31, 2018
work page 2018
-
[14]
Accelerating design optimization using reduced order models.arXiv preprint arXiv:1909.11320, 2019
Youngsoo Choi, Geoffrey Oxberry, Daniel White, and Trenton Kirchdoerfer. Accelerating design optimization using reduced order models.arXiv preprint arXiv:1909.11320, 2019
-
[15]
Seung Whan Chung, Christopher Miller, Youngsoo Choi, Paul Tranquilli, H Keo Springer, and Kyle Sullivan. Latent space dynamics identification for interface tracking with application to shock-induced pore collapse.arXiv preprint arXiv:2507.10647, 2025. 30
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[16]
LaSDI:Parametriclatentspacedynamicsidentification
WilliamDFries, XiaolongHe, andYoungsooChoi. LaSDI:Parametriclatentspacedynamicsidentification. Computer Methods in Applied Mechanics and Engineering, 399:115436, 2022
work page 2022
-
[17]
Understanding the difficulty of training deep feedforward neural networks
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. InProceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249–256. JMLR Workshop and Conference Proceedings, 2010
work page 2010
-
[18]
Alexander N Gorban and Ivan Yu Tyukin. Blessing of dimensionality: mathematical foundations of the statistical physics of data.Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 376(2118):20170237, 2018
work page 2018
-
[19]
Greiner.Relativistic Quantum Mechanics: Wave Equations
W. Greiner.Relativistic Quantum Mechanics: Wave Equations. Springer Berlin Heidelberg, 2012. ISBN 9783642880827
work page 2012
-
[20]
W.H. Hayt and J.A. Buck.Engineering Electromagnetics. McGraw-Hill, 2011. ISBN 9780071317078
work page 2011
-
[21]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016
work page 2016
-
[22]
Xiaolong He, Youngsoo Choi, William D Fries, Jonathan L Belof, and Jiun-Shyan Chen. gLaSDI: Parametric physics-informed greedy latent space dynamics identification.Journal of Computational Physics, 489:112267, 2023
work page 2023
-
[23]
Xiaolong He, April Tran, David M Bortz, and Youngsoo Choi. Physics-informed active learning with simultaneous weak-form latent space dynamics identification.International Journal for Numerical Methods in Engineering, 126(1):e7634, 2025
work page 2025
-
[24]
A fast learning algorithm for deep belief nets
Geoffrey E Hinton, Simon Osindero, and Yee-Whye Teh. A fast learning algorithm for deep belief nets. Neural computation, 18(7):1527–1554, 2006
work page 2006
-
[25]
Traian Iliescu and Zhu Wang. Variational multiscale proper orthogonal decomposition: Navier-stokes equations.Numerical Methods for Partial Differential Equations, 30(2):641–663, 2014
work page 2014
-
[26]
Openfoam: A c++ library for complex physics simulations
Hrvoje Jasak, Aleksandar Jemcov, Zeljko Tukovic, et al. Openfoam: A c++ library for complex physics simulations. InInternational workshop on coupled methods in numerical dynamics, volume 1000, pages 1–20. Dubrovnik, Croatia), 2007
work page 2007
-
[27]
Eurika Kaiser, J Nathan Kutz, and Steven L Brunton. Sparse identification of nonlinear dynamics for model predictive control in the low-data limit.Proceedings of the Royal Society A, 474(2219):20180335, 2018
work page 2018
-
[28]
Youngkyu Kim, Karen Wang, and Youngsoo Choi. Efficient space–time reduced order model for linear dynamical systems in python using less than 120 lines of code.Mathematics, 9(14):1690, 2021
work page 2021
-
[29]
Adam: A Method for Stochastic Optimization
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014
work page internal anchor Pith review Pith/arXiv arXiv 2014
-
[30]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks.Communications of the ACM, 60(6):84–90, 2017
work page 2017
-
[31]
Kookjin Lee and Eric J Parish. Parameterized neural ordinary differential equations: Applications to computational physics problems.Proceedings of the Royal Society A, 477(2253):20210162, 2021
work page 2021
-
[32]
Bethany Lusch, J Nathan Kutz, and Steven L Brunton. Deep learning for universal linear embeddings of nonlinear dynamics.Nature communications, 9(1):4950, 2018
work page 2018
-
[33]
Component-wise reduced order model lattice-type structure design
Sean McBane and Youngsoo Choi. Component-wise reduced order model lattice-type structure design. Computer methods in applied mechanics and engineering, 381:113813, 2021. 31
work page 2021
-
[34]
Benjamin McLaughlin, Janet Peterson, and Ming Ye. Stabilized reduced order models for the advection– diffusion–reaction equation using operator splitting.Computers & Mathematics with Applications, 71 (11):2407–2420, 2016
work page 2016
-
[35]
Jun Sur Richard Park, Siu Wun Cheung, Youngsoo Choi, and Yeonjong Shin. tlasdi: Thermodynamics- informed latent space dynamics identification.Computer Methods in Applied Mechanics and Engineering, 429:117144, 2024
work page 2024
-
[36]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library.Advances in neural information processing systems, 32, 2019
work page 2019
-
[37]
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and É. Duchesnay. Scikit-learn: Machine learning in python.J. Mach. Learn. Res., 12(null):2825–2830, November 2011. ISSN 1532-4435
work page 2011
-
[38]
Universal Differential Equations for Scientific Machine Learning
Christopher Rackauckas, Yingbo Ma, Julius Martensen, Collin Warner, Kirill Zubov, Rohit Supekar, Dominic Skinner, Ali Ramadhan, and Alan Edelman. Universal differential equations for scientific machine learning.arXiv preprint arXiv:2001.04385, 2020
work page internal anchor Pith review Pith/arXiv arXiv 2001
-
[39]
Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.Journal of Computational physics, 378:686–707, 2019
work page 2019
-
[40]
Yulia Rubanova, Ricky TQ Chen, and David K Duvenaud. Latent ordinary differential equations for irregularly-sampled time series.Advances in neural information processing systems, 32, 2019
work page 2019
-
[41]
Data-driven discovery of partial differential equations.Science advances, 3(4):e1602614, 2017
Samuel H Rudy, Steven L Brunton, Joshua L Proctor, and J Nathan Kutz. Data-driven discovery of partial differential equations.Science advances, 3(4):e1602614, 2017
work page 2017
-
[42]
Peter J Schmid. Dynamic mode decomposition of numerical and experimental data.Journal of fluid mechanics, 656:5–28, 2010
work page 2010
-
[43]
Giovanni Stabile and Gianluigi Rozza. Finite volume pod-galerkin stabilised reduced order methods for the parametrised incompressible navier–stokes equations.Computers & Fluids, 173:273–284, 2018
work page 2018
-
[44]
Robert Stephany. Dde-find: learning delay differential equations from noisy, limited data.Proceedings of the Royal Society A, 481(2310):20240403, 2025
work page 2025
-
[45]
Robert Stephany and Youngsoo Choi. Rollout-lasdi: Enhancing the long-term accuracy of latent space dynamics.arXiv preprint arXiv:2509.08191, 2025
-
[46]
Robert Stephany and Christopher Earls. Pde-read: Human-readable partial differential equation discovery using deep learning.Neural Networks, 154:360–382, 2022
work page 2022
-
[47]
Robert Stephany and Christopher Earls. Pde-learn: Using deep learning to discover partial differential equations from noisy, limited data.Neural Networks, 174:106242, 2024
work page 2024
-
[48]
Cambridge university press, 2007
Jos Thijssen.Computational physics. Cambridge university press, 2007
work page 2007
-
[49]
April Tran, Xiaolong He, Daniel A Messenger, Youngsoo Choi, and David M Bortz. Weak-form latent space dynamics identification.Computer Methods in Applied Mechanics and Engineering, 427:116998, 2024
work page 2024
-
[50]
Attention is all you need.Advances in neural information processing systems, 30, 2017
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017
work page 2017
-
[51]
Christopher KI Williams and Carl Edward Rasmussen.Gaussian processes for machine learning, volume 2. MIT press Cambridge, MA, 2006. 32 A Derivation of Finite Difference Schemes In this appendix, we derive the finite difference schemes we introduced in section 4.1. Throughout this section, we will letS⊆Rbe an open subset ofR. Likewise, letf:S→R. A.1 Three-...
work page 2006
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.