pith. sign in

arxiv: 2605.26039 · v2 · pith:FSJ33WHKnew · submitted 2026-05-25 · 🧮 math.NA · cs.NA

Fast Quadratic Manifold Learning For Nonlinear Dimensionality Reduction in Large-scale Systems using Riemannian Optimization

Gavin Paxton , Seunghee Cheon , Rudy Geelen , Shane A. McQuarrie This is my paper

Pith reviewed 2026-06-29 20:17 UTC · model grok-4.3

classification 🧮 math.NA cs.NA
keywords quadratic manifold learningRiemannian optimizationStiefel manifolddimensionality reductionreduced-order modelingnonlinear approximationsingular vectorslarge-scale simulation
0
0 comments X

The pith

FastQM optimizes quadratic manifold bases by rotating them on the Stiefel manifold to outperform principal component choices.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces FastQM to address the suboptimality of standard principal component bases for quadratic manifold approximation in dimensionality reduction. It formulates basis alignment as a continuous Riemannian optimization problem on the Stiefel manifold over a span of candidate singular vectors, sidestepping the combinatorial expense of greedy selection methods. A feature-space formulation keeps the computational cost independent of the full state dimension. The approach is demonstrated on a turbulent airfoil-wake large-eddy simulation to show gains in representational capacity for nonlinear reduced-order modeling.

Core claim

FastQM learns an ideal coordinate alignment for quadratic manifold approximation by rotating the reduced basis within a candidate span of singular vectors using Riemannian optimization on the Stiefel manifold. The feature-space formulation ensures the optimization cost scales independently of the full state-space dimension, enabling better quadratic approximations than those from leading principal components or greedy selection in large-scale systems.

What carries the argument

Riemannian optimization on the Stiefel manifold to rotate a reduced basis inside the span of candidate singular vectors for quadratic manifold learning, implemented via a feature-space formulation.

If this is right

  • Quadratic manifold approximations achieve higher accuracy for fixed basis size by using optimized alignments rather than leading principal components.
  • The method remains practical for high-dimensional systems because optimization cost stays independent of the original state dimension.
  • Continuous optimization replaces combinatorial greedy search and allows exploration of larger candidate pools without prohibitive cost.
  • Better bases directly improve the fidelity of reduced-order models for nonlinear dynamics such as fluid turbulence.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The Stiefel manifold rotation technique could extend to basis selection in other nonlinear approximation schemes beyond quadratics.
  • The approach indicates that many reduced-order modeling tasks may benefit from treating basis choice as a manifold optimization problem rather than discrete search.
  • Validation on additional large-scale datasets would test whether the observed gains generalize beyond the airfoil-wake example.

Load-bearing premise

An optimal or near-optimal basis for the quadratic manifold exists inside the span of the chosen candidate singular vectors and the optimizer will locate a rotation that improves performance.

What would settle it

If the quadratic manifold reconstruction error on held-out simulation data shows no improvement when using the Riemannian-optimized basis versus the leading principal components, the central claim would be refuted.

read the original abstract

The effectiveness of dimensionality reduction with quadratic manifolds hinges on the choice of a reduced basis and the associated quadratic correction terms. Existing approaches typically rely on subspaces spanned by the leading principal components of the training data. Although optimal for linear approximation, such bases are inherently suboptimal for quadratic manifold learning. Greedy basis-selection methods can significantly improve the representational capacity of quadratic manifolds by searching over a larger pool of candidate principal components, but the combinatorial cost limits the basis sizes that can be used in practice. This work proposes FastQM, an approach that treats the identification of an optimal quadratic approximation as a continuous optimization problem on the Stiefel manifold. By rotating the reduced basis within a candidate span of singular vectors, FastQM learns an ideal coordinate alignment tailored to quadratic manifold approximation. A feature-space formulation ensures that the optimization cost scales independently of the full state-space dimension. The efficacy of the proposed method is demonstrated on a turbulent airfoil-wake large-eddy simulation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper introduces FastQM, which formulates quadratic manifold basis selection as a Riemannian optimization problem on the Stiefel manifold. The method rotates a reduced basis inside the span of candidate singular vectors to improve quadratic approximation quality over leading principal components, while a feature-space formulation keeps the cost independent of the full state dimension. Efficacy is shown via demonstration on a turbulent airfoil-wake large-eddy simulation.

Significance. If the optimization consistently locates rotations that meaningfully outperform both leading PCs and feasible greedy selections, the approach would address a recognized limitation in quadratic manifold methods for large-scale nonlinear reduction. The feature-space scaling claim, if substantiated, would be a practical strength for high-dimensional systems.

major comments (3)
  1. [Method formulation (inferred from abstract)] The manuscript provides no argument establishing that an optimal (or near-optimal) quadratic correction basis must lie inside the span of the candidate singular vectors; this span-containment assumption is load-bearing for the claim that Stiefel optimization will improve representational capacity. Without such justification or a counter-example showing the assumption can fail, the central improvement guarantee remains unanchored.
  2. [Optimization procedure] No convergence analysis, local-minima characterization, or guarantee is supplied for the non-convex Stiefel optimization; the paper therefore offers no evidence that the procedure reliably escapes the leading-PC initialization or outperforms it by more than marginal amounts.
  3. [Numerical results] The demonstration on the airfoil-wake LES is described only qualitatively; the absence of quantitative error metrics, basis-size comparisons, or tables against leading PCs and greedy selection prevents verification that the claimed improvement is realized in practice.
minor comments (2)
  1. [Abstract / Feature-space formulation] The abstract states that the feature-space formulation removes dependence on full state dimension, but the precise mapping from state-space inner products to feature-space quantities is not shown; a short derivation or pseudocode would clarify the scaling claim.
  2. [Method] Notation for the quadratic correction terms and the Stiefel constraint should be introduced with explicit definitions before the optimization problem is stated.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive comments on our manuscript. We address each major point below, clarifying the method's scope and agreeing where revisions are warranted to strengthen the presentation.

read point-by-point responses

  1. Referee: The manuscript provides no argument establishing that an optimal (or near-optimal) quadratic correction basis must lie inside the span of the candidate singular vectors; this span-containment assumption is load-bearing for the claim that Stiefel optimization will improve representational capacity. Without such justification or a counter-example showing the assumption can fail, the central improvement guarantee remains unanchored.

    Authors: FastQM does not assert that a globally optimal basis must lie in the candidate span; the formulation deliberately restricts the search to rotations within an expanded pool of singular vectors (larger than the leading PCs) to enable continuous optimization as a tractable alternative to combinatorial greedy search. This is motivated by the observation that quadratic manifold quality depends on coordinate alignment within the data-driven subspace, and the feature-space cost remains independent of state dimension. We do not claim a theoretical guarantee of global optimality, which would require solving an intractable combinatorial problem over all possible bases. revision: no

  2. Referee: No convergence analysis, local-minima characterization, or guarantee is supplied for the non-convex Stiefel optimization; the paper therefore offers no evidence that the procedure reliably escapes the leading-PC initialization or outperforms it by more than marginal amounts.

    Authors: The optimization problem is non-convex, and we employ standard Riemannian trust-region or conjugate-gradient methods on the Stiefel manifold, which are guaranteed to reach a stationary point from the leading-PC initialization. Numerical results on the airfoil-wake data show consistent improvement in approximation quality, though we acknowledge the absence of a full local-minima analysis or escape guarantee. Such analysis lies beyond the paper's scope, which focuses on the practical formulation and scaling. revision: no

  3. Referee: The demonstration on the airfoil-wake LES is described only qualitatively; the absence of quantitative error metrics, basis-size comparisons, or tables against leading PCs and greedy selection prevents verification that the claimed improvement is realized in practice.

    Authors: We agree that quantitative metrics and explicit comparisons would improve verifiability. The revised manuscript will add tables reporting reconstruction errors for varying basis sizes, including direct comparisons to leading principal components and feasible greedy selections on the turbulent airfoil-wake dataset. revision: yes

Circularity Check

0 steps flagged

No circularity: optimization method is independent of its inputs

full rationale

The paper presents FastQM as a continuous optimization procedure on the Stiefel manifold that rotates a candidate basis to improve quadratic manifold approximation. No equations, fitted parameters, or self-citations are shown that reduce a claimed prediction or uniqueness result back to the input data or prior author work by construction. The feature-space formulation addresses computational scaling but does not create a definitional loop. The derivation therefore remains self-contained against external benchmarks and does not trigger any of the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review; no explicit free parameters, axioms, or invented entities are stated. The method implicitly relies on standard concepts from Riemannian optimization and singular value decomposition.

pith-pipeline@v0.9.1-grok · 5706 in / 1125 out tokens · 22738 ms · 2026-06-29T20:17:05.613013+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 30 canonical work pages

  1. [1]

    A survey of projection-based model reduction methods for parametric dynamical systems,

    Benner, P., Gugercin, S., and Willcox, K., “A survey of projection-based model reduction methods for parametric dynamical systems,”SIAM Review, Vol. 57, No. 4, 2015, pp. 483–531. https://doi.org/10.1137/130932715

    work page doi:10.1137/130932715 2015

  2. [2]

    Survey of multifidelity methods in uncertainty propagation, inference, and optimization,

    Peherstorfer, B., Willcox, K., and Gunzburger, M., “Survey of multifidelity methods in uncertainty propagation, inference, and optimization,”SIAM Review, Vol. 60, No. 3, 2018, pp. 550–591. https://doi.org/10.1137/16M1082469

    work page doi:10.1137/16m1082469 2018

  3. [3]

    The structure of inhomogeneous turbulent flows,

    Lumley, J. L., “The structure of inhomogeneous turbulent flows,”Atmospheric Turbulence and Radio Wave Propagation, 1967, pp. 166–178

    1967

  4. [4]

    Turbulence and the dynamics of coherent structures. I. Coherent structures,

    Sirovich, L., “Turbulence and the dynamics of coherent structures. I. Coherent structures,”Quarterly of Applied Mathematics, Vol. 45, No. 3, 1987, pp. 561–571. https://doi.org/10.1090/qam/910462

    work page doi:10.1090/qam/910462 1987

  5. [5]

    The proper orthogonal decomposition in the analysis of turbulent flows,

    Berkooz, G., Holmes, P., and Lumley, J. L., “The proper orthogonal decomposition in the analysis of turbulent flows,”Annual Review of Fluid Mechanics, Vol. 25, No. 1, 1993, pp. 539–575. https://doi.org/10.1146/annurev.fl.25.010193.002543

    work page doi:10.1146/annurev.fl.25.010193.002543 1993

  6. [6]

    A quadratic manifold for model order reduction of nonlinear structural dynamics,

    Jain, S., Tiso, P., Rutzmoser, J. B., and Rixen, D. J., “A quadratic manifold for model order reduction of nonlinear structural dynamics,”Computers & Structures, Vol. 188, 2017, pp. 80–94. https://doi.org/10.1016/j.compstruc.2017.04.005

    work page doi:10.1016/j.compstruc.2017.04.005 2017

  7. [7]

    Generalizationofquadraticmanifoldsforreducedordermodelingofnonlinear structural dynamics,

    Rutzmoser,J.B.,Rixen,D.J.,Tiso,P.,andJain,S.,“Generalizationofquadraticmanifoldsforreducedordermodelingofnonlinear structural dynamics,”Computers & Structures, Vol. 192, 2017, pp. 196–209. https://doi.org/10.1016/j.compstruc.2017.06.003

    work page doi:10.1016/j.compstruc.2017.06.003 2017

  8. [8]

    Quadratic approximation manifold for mitigating the Kolmogorov barrier in nonlinear projection- based model order reduction,

    Barnett, J., and Farhat, C., “Quadratic approximation manifold for mitigating the Kolmogorov barrier in nonlinear projection- based model order reduction,”Journal of Computational Physics, Vol. 464, 2022, p. 111348. https://doi.org/10.1016/j.jcp.2022. 111348

    work page doi:10.1016/j.jcp.2022 2022

  9. [9]

    Operator inference for non-intrusive model reduction with quadratic manifolds,

    Geelen, R., Wright, S., and Willcox, K., “Operator inference for non-intrusive model reduction with quadratic manifolds,” Computer Methods in Applied Mechanics and Engineering, Vol. 403, 2023, p. 115717. https://doi.org/10.1016/j.cma.2022. 115717

    work page doi:10.1016/j.cma.2022 2023

  10. [10]

    Symplectic model reduction of Hamiltonian systems using data-driven quadratic manifolds,

    Sharma, H., Mu, H., Buchfink, P., Geelen, R., Glas, S., and Kramer, B., “Symplectic model reduction of Hamiltonian systems using data-driven quadratic manifolds,”Computer Methods in Applied Mechanics and Engineering, Vol. 417, 2023, p. 116402. https://doi.org/10.1016/j.cma.2023.116402

    work page doi:10.1016/j.cma.2023.116402 2023

  11. [11]

    Greedy construction of quadratic manifolds for nonlinear dimensionality reduction and nonlinear model reduction,

    Schwerdtner, P., and Peherstorfer, B., “Greedy construction of quadratic manifolds for nonlinear dimensionality reduction and nonlinear model reduction,” arXiv:2403.06732, 2024. https://doi.org/10.48550/arXiv.2403.06732

    work page doi:10.48550/arxiv.2403.06732 2024

  12. [12]

    Online learning of quadratic manifoldsfromstreamingdatafornonlineardimensionalityreductionandnonlinearmodelreduction,

    Schwerdtner, P., Mohan, P., Pachalieva, A., Bessac, J., O’Malley, D., and Peherstorfer, B., “Online learning of quadratic manifoldsfromstreamingdatafornonlineardimensionalityreductionandnonlinearmodelreduction,”ProceedingsoftheRoyal SocietyA:Mathematical,PhysicalandEngineeringSciences,Vol.481,No.2314,2025. https://doi.org/10.1098/rspa.2024.0670

    work page doi:10.1098/rspa.2024.0670 2025

  13. [13]

    Empirical sparse regression on quadratic manifolds,

    Schwerdtner, P., Gugercin, S., and Peherstorfer, B., “Empirical sparse regression on quadratic manifolds,”SIAM Journal on Scientific Computing, Vol. 47, No. 6, 2025, pp. A3085–A3107. https://doi.org/10.1137/24M1717270

    work page doi:10.1137/24m1717270 2025

  14. [14]

    Unified LSPG model reduction framework and assessment for hypersonic computational fluid dynamics,

    Chmiel, M. R., Barnett, J., and Farhat, C., “Unified LSPG model reduction framework and assessment for hypersonic computational fluid dynamics,”AIAA Journal, Vol. 63, No. 1, 2025, pp. 72–90. https://doi.org/10.2514/1.J063840

    work page doi:10.2514/1.j063840 2025

  15. [15]

    Learning latent representations in high-dimensional state spaces using polynomial manifold constructions,

    Geelen, R., Balzano, L., and Willcox, K., “Learning latent representations in high-dimensional state spaces using polynomial manifold constructions,”2023 62nd IEEE Conference on Decision and Control (CDC), IEEE, 2023, pp. 4960–4965. https://doi.org/10.1109/CDC49753.2023.10384209

    work page doi:10.1109/cdc49753.2023.10384209 2023

  16. [16]

    Learning physics-based reduced-order models from data using nonlinear manifolds,

    Geelen, R., Balzano, L., Wright, S., and Willcox, K., “Learning physics-based reduced-order models from data using nonlinear manifolds,”Chaos: AnInterdisciplinaryJournalofNonlinearScience, Vol.34, No.3, 2024. https://doi.org/10.1063/5.0170105

    work page doi:10.1063/5.0170105 2024

  17. [17]

    Approximation bounds for model reduction on polynomially mapped manifolds,

    Buchfink, P., Glas, S., and Haasdonk, B., “Approximation bounds for model reduction on polynomially mapped manifolds,” Comptes Rendus. Mathématique, Vol. 362, 2024, pp. 1881–1891. https://doi.org/10.5802/crmath.632

    work page doi:10.5802/crmath.632 2024

  18. [18]

    Entropy-stable model reduction of one-dimensional hyperbolic systems using rational quadratic manifolds,

    Klein, R., Sanderse, B., Costa, P., Pecnik, R., and Henkes, R., “Entropy-stable model reduction of one-dimensional hyperbolic systems using rational quadratic manifolds,”Journal of Computational Physics, Vol. 528, 2025, p. 113817. https://doi.org/10.1016/j.jcp.2025.113817

    work page doi:10.1016/j.jcp.2025.113817 2025

  19. [19]

    Learning latent space dynamics with model-form uncertainties: A stochastic reduced-order modeling approach,

    Yong, J. Y., Geelen, R., and Guilleminot, J., “Learning latent space dynamics with model-form uncertainties: A stochastic reduced-order modeling approach,”Computer Methods in Applied Mechanics and Engineering, Vol. 435, 2025, p. 117638. https://doi.org/10.1016/j.cma.2024.117638. 14

    work page doi:10.1016/j.cma.2024.117638 2025

  20. [20]

    Nonlinear model reduction by probabilistic manifold decomposition,

    Guo, J., and Xiao, D., “Nonlinear model reduction by probabilistic manifold decomposition,”SIAM Journal on Scientific Computing, Vol. 48, No. 1, 2026, pp. A209–A235. https://doi.org/10.1137/25M1738863

    work page doi:10.1137/25m1738863 2026

  21. [21]

    Parametric probabilistic manifold decomposition for nonlinear model reduction,

    Guo, J., and Xiao, D., “Parametric probabilistic manifold decomposition for nonlinear model reduction,” arXiv:2601.07278,

    arXiv

  22. [22]

    https://doi.org/10.48550/arXiv.2601.07278

    work page doi:10.48550/arxiv.2601.07278

  23. [23]

    Mitigating the Kolmogorov barrier for the reduction of aerodynamic models using neural-network-augmented reduced-order models,

    Barnett, J. L., Farhat, C., and Maday, Y., “Mitigating the Kolmogorov barrier for the reduction of aerodynamic models using neural-network-augmented reduced-order models,”AIAA SciTech Forum, 2023, p. 0535. https://doi.org/10.2514/6.2023-0535

    work page doi:10.2514/6.2023-0535 2023

  24. [24]

    Nonlinear projection-based model order reduction with machine learning regression for closure error modeling in the latent space,

    Ares de Parga, S., Tezaur, R., Hernández, C. G., and Farhat, C., “Nonlinear projection-based model order reduction with machine learning regression for closure error modeling in the latent space,”Computer Methods in Applied Mechanics and Engineering, Vol. 448, 2026, p. 118443. https://doi.org/10.1016/j.cma.2025.118443

    work page doi:10.1016/j.cma.2025.118443 2026

  25. [25]

    Machine learning-based quadratic closures for non-intrusive reduced order models,

    Codega, G., Ivagnes, A., Demo, N., and Rozza, G., “Machine learning-based quadratic closures for non-intrusive reduced order models,”SIAM Journal on Scientific Computing, Vol. 48, No. 3, 2026, pp. C505–C525. https://doi.org/10.1137/25M1766759

    work page doi:10.1137/25m1766759 2026

  26. [26]

    Interpretable and flexible non-intrusive reduced-order models usingreproducingkernelHilbertspaces,

    Diaz, A. N., McQuarrie, S. A., Tencer, J. T., and Blonigan, P. J., “Interpretable and flexible non-intrusive reduced-order models usingreproducingkernelHilbertspaces,”ComputerMethodsinAppliedMechanicsandEngineering,Vol.452,2026,p.118734. https://doi.org/10.1016/j.cma.2026.118734

    work page doi:10.1016/j.cma.2026.118734 2026

  27. [27]

    Kernel manifolds: Nonlinear-augmentation dimensionality reduction using reproducing kernel Hilbert spaces,

    Diaz, A. N., Needels, J. T., Tezaur, I. K., and Blonigan, P. J., “Kernel manifolds: Nonlinear-augmentation dimensionality reduction using reproducing kernel Hilbert spaces,”International Journal for Numerical Methods in Engineering, Vol. 126, No. 24, 2025, p. e70230. https://doi.org/10.1002/nme.70230

    work page doi:10.1002/nme.70230 2025

  28. [28]

    Onclosuresforreducedordermodels–Aspectrumof first-principletomachine-learnedavenues,

    Ahmed,S.E.,Pawar,S.,San,O.,Rasheed,A.,Iliescu,T.,andNoack,B.R.,“Onclosuresforreducedordermodels–Aspectrumof first-principletomachine-learnedavenues,”PhysicsofFluids,Vol.33,No.9,2021,p.091301.https://doi.org/10.1063/5.0061577

    work page doi:10.1063/5.0061577 2021

  29. [29]

    Nonlinear model order reduction on quadratic manifolds via greedy algorithms with dimension-dependent regularization,

    Ji, L., Rashid, S., Chen, Y., and Wang, Z., “Nonlinear model order reduction on quadratic manifolds via greedy algorithms with dimension-dependent regularization,” arXiv:2603.24962, 2026. https://doi.org/10.48550/arXiv.2603.24962

    work page doi:10.48550/arxiv.2603.24962 2026

  30. [30]

    Pymanopt: A Python toolbox for optimization on manifolds using automatic differentiation,

    Townsend, J., Koep, N., and Weichwald, S., “Pymanopt: A Python toolbox for optimization on manifolds using automatic differentiation,”Journal of Machine Learning Research, Vol. 17, No. 137, 2016, pp. 1–5. URL http://jmlr.org/papers/v17/16- 177.html

    2016

  31. [31]

    Data-driven parametric reduced-order models: Operator inference for reactive flow applications,

    McQuarrie, S. A., “Data-driven parametric reduced-order models: Operator inference for reactive flow applications,” Ph.D. thesis, The University of Texas at Austin, 2023. https://doi.org/10.26153/tsw/50172

    work page doi:10.26153/tsw/50172 2023

  32. [32]

    JAX: Composable transformations of Python+NumPy programs,

    Bradbury,J.,Frostig,R.,Hawkins,P.,Johnson,M.J.,Katariya,Y.,Leary,C.,Maclaurin,D.,Necula,G.,Paszke,A.,VanderPlas, J., Wanderman-Milne, S., and Zhang, Q., “JAX: Composable transformations of Python+NumPy programs,” , 2018. URL http://github.com/jax-ml/jax

    2018

  33. [33]

    Nonlinear dimensionality reduction then and now: AIMs for dissipative PDEs in the ML era,

    Koronaki, E. D., Evangelou, N., Martin-Linares, C. P., Titi, E. S., and Kevrekidis, I. G., “Nonlinear dimensionality reduction then and now: AIMs for dissipative PDEs in the ML era,”Journal of Computational Physics, Vol. 506, 2024, p. 112910. https://doi.org/https://doi.org/10.1016/j.jcp.2024.112910

    work page doi:10.1016/j.jcp.2024.112910 2024

  34. [34]

    A database for reduced-complexity modeling of fluid flows,

    Towne, A., Dawson, S. T., Brès, G. A., Lozano-Durán, A., Saxton-Fox, T., Parthasarathy, A., Jones, A. R., Biler, H., Yeh, C.-A., Patel, H. D., et al., “A database for reduced-complexity modeling of fluid flows,”AIAA Journal, Vol. 61, No. 7, 2023, pp. 2867–2892. https://doi.org/10.2514/1.J062203

    work page doi:10.2514/1.j062203 2023

  35. [35]

    Turbulent airfoil wake large eddy simulation [Data set], University of Michigan - Deep Blue Data,

    Towne, A., Yeh, C., Patel, H., and Taira, K., “Turbulent airfoil wake large eddy simulation [Data set], University of Michigan - Deep Blue Data,” http://deepblue.lib.umich.edu/data/collections/kk91fk98z, 2022. 15

    2022